Image processing method and electronic equipment

Abstract

The embodiment of the invention provides an image processing method and electronic equipment, relates to the technical field of images, and is used for reducing the brightness and color change of a displayed image when a camera is switched in a shooting process. The method is applied to the electronic equipment, the electronic equipment comprises a first camera and a second camera, and the shooting directions of the first camera and the second camera are the same. The method comprises the steps that in response to switching from a first camera to a second camera, a first image and a second image are acquired, the first image is an image acquired by the first camera, and the second image is an image to be displayed acquired by the second camera after the camera is switched; obtaining color information of the first image and color information of the second image; based on the color information of the first image and the color information of the second image, performing color compensation on the second image in a red-green-blue (RGB) space to obtain a compensated second image; and displaying the compensated second image.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image technology, and in particular, to an image processing method and an electronic device. Background Art

[0002] With the development of technology, users have higher and higher requirements for taking pictures using electronic devices such as mobile phones. The number of cameras set on a large number of electronic devices such as mobile phones has increased from one to multiple. The combination of multiple cameras can meet the shooting requirements in different scenarios.

[0003] In some shooting scenarios, the electronic device needs to switch cameras. When switching to another camera, the image displayed on the electronic device will also switch from the image collected by one camera to the image collected by another camera. However, there may be differences in the shooting angles, the light input amount of the aperture, and the response of the image sensor to color and brightness of different cameras. Therefore, when the electronic device switches cameras, there may be problems such as jumps in the brightness and color of the images displayed before and after the switch. Summary of the Invention

[0004] Embodiments of the present application provide an image processing method and an electronic device for reducing the changes in the brightness and color of the displayed image when switching cameras during the shooting process.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an image processing method is provided. The method is applied to an electronic device, and the electronic device includes a first camera and a second camera, and the shooting directions of the first camera and the second camera are the same. The method includes:

[0007] The electronic device responds to the switch from the first camera to the second camera, and obtains a first image and a second image. The first image is the image collected by the first camera, and the second image is the image to be displayed collected by the second camera after switching the camera. Due to the differences between different cameras, when the electronic device switches from displaying the image collected by the first camera to displaying the image collected by the second camera, there are large differences in the brightness and color of the front and rear images, resulting in the problem of jumps. Therefore, after obtaining the first image and the second image, the electronic device can obtain the color information of the first image and the color information of the second image. Then, based on the color information of the first image and the color information of the second image, color compensation is performed on the second image in the RGB (Red, Green, Blue) space to obtain the compensated second image. Finally, at the time point when the second image needs to be displayed, the compensated second image is displayed.

[0008] In this solution, the image captured by the front camera before switching (i.e., the first image) is used to perform color compensation on the image captured by the camera after switching (i.e., the second image), and then the image is displayed. Thus, the color change of the image displayed by the electronic device before and after switching the camera will not jump, but can be consistent or similar to the brightness and color of the image captured by the camera before switching.

[0009] In a possible implementation manner of the first aspect, the second image is a frame of the N frames of images captured by the second camera after the camera is switched, where N is a positive integer. In this solution, after the camera is switched, the first image is used to compensate the first N frames of images captured by the camera after switching (the second camera), so that the color change of the image displayed by the electronic device before and after switching the camera will not jump, but can smoothly and naturally transition to the brightness and color of the second image.

[0010] In a possible implementation manner of the first aspect, the first image is the last frame of the image captured and displayed by the first camera before the electronic device switches from the first camera to the second camera. In this way, after the electronic device switches the camera, the brightness and color of the image captured by the second camera used after switching can be similar to the brightness and color of the image captured by the first camera used before switching, thus ensuring that the color change will not jump.

[0011] In a possible implementation manner of the first aspect, both the first camera and the second camera are in the startup state, and the first camera and the second camera operate independently. In this embodiment, the first image is the image captured in real time by the first camera after the electronic device switches from the first camera to the second camera. In this way, after the electronic device switches the camera, the brightness and color of the image captured by the second camera used after switching can be similar to the brightness and color of the image captured by the first camera used before switching, thus ensuring that the color change will not jump.

[0012] In a possible implementation manner of the first aspect, before obtaining the first image and the second image in response to switching from the first camera to the second camera, the method may further include: in response to the current shooting magnification of the electronic device switching from the first shooting magnification to the second shooting magnification, switching from the first camera to the second camera. Wherein, the first shooting magnification is less than the preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification. In this way, in the scenario where the camera switching is triggered by switching from a smaller shooting magnification to a larger shooting magnification, the electronic device can use the image captured by the camera before switching to perform color compensation on the image captured by the camera after switching, and then display it. Thus, it is ensured that the color change of the displayed image will not jump before and after switching the camera.

[0013] In a possible implementation of the first aspect, before obtaining the first image and the second image in response to the switch from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to the current shooting magnification of the electronic device switching from the first shooting magnification to the second shooting magnification. Wherein, the first shooting magnification is greater than or equal to a preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification. In this way, in a scenario where the camera switch is triggered by switching from a larger shooting magnification to a smaller shooting magnification, the electronic device can use the image captured by the pre-switch camera to perform color compensation on the image captured by the post-switch camera, and then display it. Thus, it is ensured that there is no jump in the color change of the displayed image before and after switching the camera.

[0014] In a possible implementation of the first aspect, before obtaining the first image and the second image in response to the switch from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to the distance between the first camera and the object to be photographed switching from the first shooting distance to the second shooting distance. Wherein, the first shooting distance is less than a preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance. In this way, in a scenario where the camera switch is triggered by switching from a smaller shooting distance to a larger shooting distance, the electronic device can use the image captured by the pre-switch camera to perform color compensation on the image captured by the post-switch camera, and then display it. Thus, it is ensured that there is no jump in the color change of the displayed image before and after switching the camera.

[0015] In a possible implementation of the first aspect, before obtaining the first image and the second image in response to the switch from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to the distance between the first camera and the object to be photographed switching from the first shooting distance to the second shooting distance. Wherein, the first shooting distance is greater than or equal to a preset shooting distance, and the second shooting distance is less than the preset shooting distance. In this way, in a scenario where the camera switch is triggered by switching from a larger shooting distance to a smaller shooting distance, the electronic device can use the image captured by the pre-switch camera to perform color compensation on the image captured by the post-switch camera, and then display it. Thus, it is ensured that there is no jump in the color change of the displayed image before and after switching the camera.

[0016] In a possible implementation of the first aspect, the electronic device stores the preset calibrated internal and external parameters of each second camera. In this implementation, before obtaining the color information of the first image and the color information of the second image, the above method may further include: based on the preset calibrated internal and external parameters, first perform first pixel alignment processing on the first image and the second image to obtain the first image and the second image after the first pixel alignment processing. Thereby, the pixel points of the first image and the second image can be kept aligned. Perform second pixel alignment processing on the first image and the second image after the first pixel alignment processing to obtain the first image after the second pixel alignment processing and the second image after the second pixel alignment processing. Among them, the second pixel alignment processing includes blurring processing. The blurring processing can make the pixels of the image smoother, thereby further aligning the pixel points of the first image and the second image to obtain a better pixel alignment effect. Or, the second pixel alignment processing includes downsampling processing and blurring processing. Performing downsampling processing and blurring processing on the image can reduce the error caused by inaccurate image alignment and can obtain a better pixel alignment effect. At the same time, the downsampling processing can also reduce the calculation amount of the electronic device. And the blurring processing can make the pixels of the image smoother.

[0017] In a possible implementation of the first aspect, the above-mentioned color compensation of the second image in the red, green, and blue (RGB) space based on the color information of the first image and the color information of the second image to obtain the compensated second image may specifically include: Since the first image and the second image have been subjected to pixel alignment processing, the pixel points of the images after the pixel alignment processing should be one-to-one aligned. Furthermore, an RGB mapping relationship can be generated based on the color information of the first image after the second pixel alignment processing and the color information of the second image after the second pixel alignment processing. Then, according to the RGB mapping relationship, color compensation is performed on the second image to obtain the compensated second image. In this way, it can be ensured that when color compensation is performed on the second image, accurate compensation can be carried out.

[0018] In a possible implementation of the first aspect, the RGB mapping relationship is a three-dimensional GRB mapping table. In this way, it is convenient to perform color compensation on the second image quickly and conveniently. Moreover, using the three-dimensional GRB mapping table for mapping has stronger expression ability and more accurate mapping. Using the three-dimensional GRB mapping table for mapping can align the brightness and color of the first image and the second image at the same time and adapt to richer scenarios.

[0019] In a possible implementation of the first aspect, the above-mentioned three-dimensional GRB mapping table can be specifically stored in a way of reducing sampling. In this way, the space occupied by the three-dimensional GRB mapping table in the electronic device can be reduced.

[0020] In a possible implementation of the first aspect, the second image is the i-th image among the N images collected by the second camera after switching the camera, where N is a positive integer and i ≤ N. The electronic device stores a basic three-dimensional RGB mapping table. In this implementation, generating the RGB mapping relationship based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process may specifically include: when i does not exceed S (i.e., i ≤ S), determining the color mapping relationship between each pixel point of the first image and the second image based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process. Here, S is a positive integer and S < N. Then, updating the previous three-dimensional RGB mapping table based on the color mapping relationship between each pixel point to obtain the i-th updated three-dimensional RGB mapping table. The RGB mapping relationship includes the i-th updated three-dimensional RGB mapping table; where when i = 1, the previous three-dimensional RGB mapping table is the basic three-dimensional RGB mapping table. That is, updating the basic three-dimensional GRB mapping table with the first S images collected by the switched-back camera can obtain the three-dimensional RGB mapping table when the 3A of the mobile phone has converged. When using this updated three-dimensional RGB mapping table to perform color compensation on the second image, a better compensation effect can be obtained.

[0021] In a possible implementation of the first aspect, the second image is the i-th image among the N images collected by the second camera after switching the camera, where N is a positive integer and i ≤ N. The electronic device stores a basic three-dimensional RGB mapping table. In this implementation, generating the RGB mapping relationship based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process may specifically include: when i does not exceed S, determining the color mapping relationship between each pixel point of the first image and the second image based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process. Here, S is a positive integer and S < N. Then, updating the basic three-dimensional RGB mapping table based on the color mapping relationship between each pixel point to obtain the i-th updated three-dimensional RGB mapping table. The RGB mapping relationship includes the i-th updated three-dimensional RGB mapping table. That is, updating the basic three-dimensional RGB mapping table with the first S images collected by the switched-back camera can obtain the three-dimensional RGB mapping table when the 3A of the mobile phone has converged. When using this updated three-dimensional RGB mapping table to perform color compensation on the second image, a better compensation effect can be obtained.

[0022] In a possible implementation of the first aspect, the above-mentioned color compensation of the second image according to the RGB mapping relationship to obtain the compensated second image may specifically include: applying the updated three-dimensional RGB mapping table of the i-th frame to the second image to obtain the intermediate image of the i-th frame. Wherein, when i > S, the updated three-dimensional RGB mapping table of the i-th frame is the updated three-dimensional RGB mapping table of the S-th frame. Then, using a preset weight, the intermediate image of the i-th frame and the second image are weighted and fused to obtain the compensated second image. Wherein, the magnitude of the preset weight is inversely correlated with i.

[0023] In this solution, when performing color compensation on the second image, as i gradually increases, the preset weight for weighted fusion using the color of the first image gradually decreases, which can enable the electronic device to smoothly and naturally transition from the image captured by the front camera before the display switch to the brightness and color of the image captured by the rear camera after the display switch. Thus, avoiding the color jump of the displayed image before and after switching the camera. At the same time, using the three-dimensional RGB mapping table to implement color lookup for the images before and after switching the camera facilitates convenient and fast color compensation for the second image. Moreover, using the three-dimensional GRB mapping table for mapping has stronger expressive ability and more accurate mapping. Using the three-dimensional GRB mapping table for mapping can align the brightness and color of the first image and the second image simultaneously, adapting to richer scenarios.

[0024] In a possible implementation of the first aspect, the second image is the i-th frame image among the N frame images captured by the second camera after switching the camera, N is a positive integer, and i ≤ N. Based on the color information of the first image and the color information of the second image, color compensation is performed on the second image in the red-green-blue RGB space to obtain the compensated second image, which may specifically include: in the RGB space, using a preset weight to weight and fuse the color information of the first image into the color information of the second image to obtain the compensated second image. Wherein, the magnitude of the preset weight is inversely correlated with i. In this solution, when performing color compensation on the second image, as i gradually increases, the preset weight for weighted fusion using the color of the first image gradually decreases, which can enable the electronic device to smoothly and naturally transition from the image captured by the front camera before the display switch to the brightness and color of the image captured by the rear camera after the display switch. Thus, avoiding the color jump of the displayed image before and after switching the camera.

[0025] In a possible implementation of the first aspect, before obtaining the color information of the first image and the color information of the second image, the above method further includes: determining whether the first image and / or the second image meet a preset condition. The preset condition includes: the exposure corresponding to the first image and / or the second image exceeds a preset exposure range, or the brightness difference between the first image and the second image is greater than a preset brightness difference. And, in the case where the first image and / or the second image do not meet the preset condition, perform the steps of obtaining the color information of the first image and the color information of the second image, and, based on the color information of the first image and the color information of the second image, perform color compensation on the second image in the Red-Green-Blue (RGB) space to obtain the compensated second image.

[0026] In a possible implementation of the first aspect, when the first image and / or the second image meet the above preset condition, the first image may not be used to perform color compensation on the second image, but the second image may be directly displayed. In this way, when there is an abnormal situation in the image captured by the camera, the problem of poor display effect caused by performing color compensation on the second image can be avoided.

[0027] In a possible implementation of the first aspect, the electronic device may respectively obtain the Y components of the first image and the second image in the YUV space, and determine the brightness of the first image and the brightness of the second image. Thus, the brightness difference between the first image and the second image can be calculated. In this way, the brightness difference between the first image and the second image can be determined conveniently and quickly.

[0028] In a possible implementation of the first aspect, the electronic device includes an Image Processing Hardware Abstraction Layer (HAL). The above steps of, in response to switching from the first camera to the second camera, obtaining the first image and the second image; obtaining the color information of the first image and the second image, and performing color compensation on the second image based on the color information of the first image and the color information of the second image, can be implemented by this Image Processing HAL. The Image Processing HAL sends the obtained compensated second image to the camera application, and the camera application displays the compensated second image.

[0029] In a possible implementation of the first aspect, the first image and the second image obtained by the Image Processing HAL in response to the first camera switching to the second camera are both images processed by an Image Signal Processor (ISP) (including Image Front End (IFE) and Image Processing Engine (IPE)). In this way, it is convenient to perform color compensation processing on the images.

[0030] In a second aspect, the present application further provides an electronic device. The electronic device may include: a display, a processor, and a memory. The memory is used to store computer-executable instructions. When the electronic device runs, the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the image processing method according to any one of the above first aspect.

[0031] In a third aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When it runs on a computer, it enables the computer to execute the image processing method according to any one of the above first aspect.

[0032] In a fourth aspect, a computer program product containing instructions is provided. When it runs on an electronic device, it enables the electronic device to execute the image processing method according to any one of the above first aspect.

[0033] In a fifth aspect, a device (for example, the device may be a chip system) is provided. The device includes a processor for supporting the electronic device to implement the functions involved in the above first aspect. In a possible design, the device further includes a memory for storing the necessary program instructions and data of the electronic device. When the device is a chip system, it may be composed of chips or may include chips and other discrete devices.

[0034] Among them, for the technical effects brought by any one of the design manners in the second aspect to the fifth aspect, reference may be made to the technical effects brought by different design manners in the first aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the interface of a mobile phone provided by an embodiment of the present application;

[0036] Figure 2 It is a schematic flowchart of the solution of the related art provided by an embodiment of the present application;

[0037] Figure 3 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application;

[0038] Figure 4 It is a schematic software and hardware structure diagram of an electronic device provided by an embodiment of the present application;

[0039] Figure 5 It is a schematic flowchart of an image processing method provided by an embodiment of the present application;

[0040] Figure 6 It is a schematic diagram of the effect of pixel alignment processing on a first image and a second image provided by an embodiment of the present application;

[0041] Figure 7 Schematic diagram of updating a 3D LUT table provided by an embodiment of the present application;

[0042] Figure 8 Schematic diagram of the process of using the updated 3D LUT table to find the output result provided by an embodiment of the present application;

[0043] Figure 9 Schematic flow chart of an image processing method provided by an embodiment of the present application;

[0044] Figure 10 Schematic flow chart of an image processing method provided by an embodiment of the present application;

[0045] Figure 11 Schematic diagram of the image displayed before and after the mobile phone switches the camera provided by an embodiment of the present application;

[0046] Figure 12 Schematic diagram of the structure of a chip system provided by an embodiment of the present application. Detailed implementation manners

[0047] In practical applications, there may be differences in the hardware configurations among multiple cameras of an electronic device. Exemplarily, there may be differences in the shooting angles of different cameras, the light input amount of the aperture, and the response of the image sensor to color brightness, etc. Due to the existence of these differences, when the electronic device switches from the first camera to the second camera for shooting, even if the shooting ranges are similar, when the displayed image switches from the image collected by the first camera to the image collected by the second camera, there may still be jumps in brightness and color.

[0048] Taking the electronic device as a mobile phone as an example, common cameras of a mobile phone include a wide-angle camera, a main camera, and a telephoto camera. To obtain better shooting effects, corresponding shooting magnification ranges can be set for different cameras in advance inside the mobile phone. In some embodiments, the shooting magnification range corresponding to the wide-angle camera includes: [0.6, 1); the shooting magnification range corresponding to the main camera includes: [1, 4); the shooting magnification range corresponding to the telephoto camera includes: greater than 4 times. Whichever camera's corresponding shooting magnification range the current shooting magnification of the mobile phone is in, the mobile phone will display the image collected by the corresponding camera. Therefore, during the process of the mobile phone switching the current shooting magnification, it may trigger the camera selected by the mobile phone, that is, trigger the image displayed by the mobile phone to switch from the image collected by one camera to the image collected by another camera. At the moment of camera switching, the image displayed by the mobile phone may have the jumps in brightness and color as described in the above scenario.

[0049] It should be noted that the camera of the mobile phone and the shooting magnification ranges corresponding to each camera in the above embodiments are all examples. In other embodiments, the mobile phone may include other cameras, and the shooting magnification ranges corresponding to each camera may also be other ranges.

[0050] Figure 1 Fig. shows the shooting interface 101 of the camera application of the mobile phone. This shooting interface 101 includes a magnification control 102, and the user can select the shooting magnification through the magnification control 102. Figure 1 In the shown shooting interface 101, the current shooting magnification of the mobile phone is 1 (corresponding to the icon 103 of "1X" shown in the figure). In one example, the user can slide the magnification control 102 left and right to control the switching of the current shooting magnification of the mobile phone. As Figure 1 shown, the mobile phone can switch the current shooting magnification of the mobile phone in response to the user's operation of swiping the magnification control 102 to the left.

[0051] At the same time, the shooting interface of the mobile phone is updated accordingly. For example, the mobile phone is updated to display the shooting interface 104. The magnification control in the shooting interface 104 is displayed as the magnification control 105. This magnification control 105 is used to indicate the magnification selected by the user during the sliding process; as shown in the shooting interface 104, the current shooting magnification selected by the user is 2.0 (which can also be called 2.0X). In addition, the object to be photographed in the shooting interface 104 is magnified and displayed compared with the object to be photographed in the shooting interface 101. Taking the shooting magnification ranges corresponding to each camera of the above mobile phone as an example, the images displayed in the shooting interface 101 and the shooting interface 104 are both images collected by the main camera.

[0052] Combined with the above description, it can be seen that the shooting magnification of 4X belongs to the shooting magnification range corresponding to the telephoto camera. When the current shooting magnification of the mobile phone is switched to 4X (and above), the mobile phone will switch from the main camera to the telephoto camera. Specifically, the image displayed on the mobile phone is switched from the image collected by the main camera to the image collected by the telephoto camera. As Figure 1 shown in the shooting interface 106 displayed on the mobile phone. The magnification control of this shooting interface 106 shows that the current shooting magnification of the mobile phone is 4X, as Figure 1 shown by the icon 107. The object to be photographed in this shooting interface 106 is further magnified and displayed compared with the object to be photographed in the shooting interface 104.

[0053] Due to the differences in hardware configurations between different cameras, there are differences in the brightness and color of the images collected by the telephoto camera and the main camera. Therefore, before and after the mobile phone switches from displaying the image collected by the main camera to displaying the image collected by the telephoto camera, there are jumps in the brightness and color of the displayed image. And the jumps in the brightness and color of the image displayed on the mobile phone are likely to affect the visual experience of the user during shooting, and further affect the user experience.

[0054] It is understandable that Figure 1 only one implementation process of switching the current shooting magnification of the mobile phone in the photo-taking interface of the mobile phone camera application is shown. During the process of switching the current shooting magnification of the mobile phone in the video recording and other interfaces of the mobile phone camera application, or when the mobile phone switches the current shooting magnification in other ways, similar problems as above also exist; they will not be elaborated here.

[0055] As Figure 2 shown, in the related art, the above problems are solved in the following way: First step, calculate the mean and variance of two histograms; Second step, align the histogram of the target image to obtain a reference image; Third step, obtain the corrected image, and perform the above operations on the Y, U, and V channels in sequence. In this method, the brightness information of the Y channel of the target image is adjusted after alignment. This method uses the histogram to align. When using histogram alignment, it is necessary to assume that the histogram distributions of each channel of the image are approximately normal distributions, and calculate the mean and variance of the normal distributions to align the histograms of different images. However, this method will cause problems such as image blurring and color cast, and there are certain limitations.

[0056] Based on this, an embodiment of the present application proposes an image processing method, which can be applied to an electronic device (such as a mobile phone) including at least two cameras (such as a first camera and a second camera). It should be noted that the shooting directions of the first camera and the second camera in the embodiment of the present application are the same, that is, both are rear cameras or both are front cameras. Specifically, this method can be used to solve the problem that the brightness and color of the image displayed on the electronic device jump when switching between cameras.

[0057] In some embodiments, for the image processing method in the embodiments of the present application, when the electronic device responds to the switching from the first camera to the second camera, it acquires the image captured by the first camera (denoted as the first image) and the image captured by the second camera (denoted as the second image). Moreover, the second image is the image to be displayed captured by the second camera after the camera is switched. Then, the electronic device respectively acquires the color information of the first image and the second image. And, based on the color information of the first image and the color information of the second image, the electronic device performs color compensation on the second image in the red (R) green (G) blue (B) (RGB) space to obtain the compensated second image. Finally, the electronic device displays the compensated second image at the target display moment. In this solution, the first image is used to perform color compensation on the image captured by the second camera after the camera is switched, so that at the target display moment, the original second image is replaced by the compensated second image for display, ensuring that when the electronic device switches cameras, the color change of the displayed image does not jump, but remains consistent or similar in brightness and color to the image captured by the first camera.

[0058] In some embodiments, the above-mentioned second image is specifically any one of the first N frames of images captured by the second camera after the camera is switched. In this way, after the electronic device switches cameras, performing color compensation on the first N frames of images captured by the second camera and then displaying them can ensure that the color change of the displayed image before and after the camera switch does not jump, but smoothly and naturally transitions from the color of the image captured by the first camera to the color of the image captured by the second camera.

[0059] The above-mentioned electronic device can specifically be any electronic device with two or more cameras. Exemplarily, the electronic device can specifically be a mobile phone, a tablet computer, a personal computer (PC), a smart screen, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, wearable devices such as smart watches, artificial intelligence (AI) speakers, and in-vehicle devices. It can also be various teaching aids (such as learning machines, early education machines), smart toys, portable robots, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, media players, etc. Devices. It can also be a device with mobile office functions, a device with smart home functions, a device with audio-visual entertainment functions, a device supporting smart travel, etc. The embodiments of the present application do not impose special restrictions on the specific form of this device.

[0060] Figure 3 The structural schematic diagram of the electronic device 100 provided in some embodiments of the present application is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, a camera 192, a display screen 193, and a subscriber identification module (SIM) card interface 194, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a touch sensor 180B, etc.

[0061] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0062] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. For example, the processor 110 is used to execute the video processing method in the embodiments of the present application. Among them, the ISP may specifically include: an image signal processor frontend (IFE), and an image signal processor post end (IPE).

[0063] Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0064] A memory can also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] The USB interface 130 is an interface that conforms to the USB standard specification, and can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices.

[0066] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve the data storage function. For example, files such as music and videos are saved in the external memory card.

[0067] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function (such as a sound playback function, an image playback function, etc.).

[0068] In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0069] The charge management module 140 is used to receive a charging input from a charger. Among them, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charge management module 140 can receive the charging input from a wired charger through the USB interface 130.

[0070] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 193, the camera 192, the wireless communication module 160, etc.

[0071] In some other embodiments, the power management module 141 may also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 may also be disposed in the same device.

[0072] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0073] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0074] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through the antenna 1 and radiate it out.

[0075] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as Wi-Fi networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0076] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, enabling the electronic device 100 to communicate with the network and other devices through wireless communication technologies.

[0077] The electronic device 100 may implement audio functions through the audio module 170 and the application processor, etc. For example, music playback, recording, etc.

[0078] The audio module 170 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 170 may also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

[0079] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 193. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 193, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0080] The touch sensor 180B, also known as the "touch panel". The touch sensor 180B can be disposed on the display screen 193. The touch sensor 180B and the display screen 193 together form a touch screen, also known as the "touch display screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual outputs related to the touch operations can be provided through the display screen 193. In some other embodiments, the touch sensor 180B can also be disposed on the surface of the electronic device 100, at a different position from where the display screen 193 is located.

[0081] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys. They can also be touch keys. The electronic device 100 can receive key inputs and generate key signal inputs related to the user settings and function controls of the electronic device 100.

[0082] The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback.

[0083] The camera 192 is used to capture still images or videos. In some embodiments, the electronic device 100 can include one or N cameras 192, where N is a positive integer greater than 1.

[0084] The electronic device 100 realizes the display function through the GPU, the display screen 193, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0085] The display screen 193 is used to display images, videos, etc. In some embodiments, the electronic device 100 can include one or N display screens 193, where N is a positive integer greater than 1.

[0086] The SIM card interface 194 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 194 to achieve contact and separation from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1.

[0087] The image processing methods involved in the following embodiments can all be executed in the electronic device 100 having the above hardware structure.

[0088] The following briefly explains the technical terms that may be involved in the embodiments of this application.

[0089] RGB is a color model that obtains various colors by changing the three color channels of red, green, and blue and their superposition with each other.

[0090] YUV is a color encoding mode, where Y represents luminance, that is, the grayscale value, and UV represent chrominance and chroma respectively. Its function is to describe the color and saturation of an image and is used to specify the color of a pixel.

[0091] The three-dimensional (3D) lookup table (LUT) is an algorithm for retuning the hue of an image by establishing a three-dimensional color mapping table. The input of the 3D LUT includes RGB values, and the output is the RGB values corresponding to the input RGB values in the 3D LUT.

[0092] 3A of a mobile phone usually refers to the algorithm functions of the camera. 3A can include automatic focus (AF), auto exposure (AE), and auto white balance (AWB).

[0093] The image processing method proposed in the embodiments of this application can be specifically applied to the scenario of camera switching that occurs during the shooting process of an electronic device. This method can be used to reduce the differences in brightness and color between the images when the displayed image switches from the image captured by the first camera to the image captured by the second camera before and after the electronic device switches the camera, thereby avoiding the situation of jumps in brightness and color.

[0094] The following will describe in detail the image processing method proposed in the embodiments of this application with reference to the accompanying drawings.

[0095] Figure 4 The schematic diagram of the software and hardware architecture of the mobile phone is shown. At the same time, in Figure 4 the data flow of the above image processing method in some embodiments is also shown. In combination with Figure 4 , the image processing method proposed in the embodiments of this application will be introduced.

[0096] The mobile phone includes an application layer, a hardware abstraction layer (HAL), and a hardware layer. Among them, HAL at least includes a camera HAL and an image processing HAL. The camera HAL is used for the camera application to call the camera. The image processing HAL is used to process the images captured by the camera.

[0097] In combination with Figure 5 for the basis of Figure 4Describe the process of the image processing method implemented by the mobile phone shown. This method includes the following steps:

[0098] S301. The camera application is started.

[0099] In some embodiments, starting the camera application can specifically be that the mobile phone responds to the user's operation of opening the camera application and controls the camera application to start.

[0100] S302. The camera application calls the first camera and the second camera through the camera HAL.

[0101] In some embodiments, the camera application can send an activation instruction to the camera (including the first camera and the second camera) by calling the interface provided by the camera HAL. Correspondingly, the camera receives the activation instruction and starts in response to the activation instruction, and then can start image acquisition.

[0102] The first camera and the second camera are two different cameras in the mobile phone respectively. Exemplarily, the first camera is a wide-angle camera, and the second camera can be the main camera or the telephoto camera. The first camera is the main camera, and the second camera can be the wide-angle camera or the telephoto camera. The first camera is the telephoto camera, and the second camera can be the main camera or the wide-angle camera.

[0103] In some embodiments, after the camera application is started, multiple cameras can be started simultaneously for image acquisition, and the multiple cameras can operate independently. However, in the interface corresponding to the camera application, only the image acquired by one of the cameras is displayed. For example, the camera application simultaneously calls the first camera and the second camera to respectively acquire corresponding images, and the mobile phone selects the image acquired by one of the cameras according to the current shooting magnification and displays it on the interface. It should be noted that in the scenario where the mobile phone includes three cameras, the camera application can simultaneously call the three cameras to respectively perform image acquisition and display the image acquired by one of the cameras according to the current shooting magnification.

[0104] In some other embodiments, the camera application can also only start one camera for image acquisition. Further, the camera application can start the camera to be switched on for image acquisition again when detecting a camera switch. For example, the camera application calls the first camera for image acquisition. When the camera application detects a switch from the first camera to the second camera, the camera application starts the second camera. This can reduce the power consumption of the mobile phone.

[0105] S303. Save the image 1 acquired by the first camera and the image 2 acquired by the second camera to the cache.

[0106] In some embodiments, the above-mentioned Image 1 and Image 2 are both images processed by IFE and IPE. The process in which the first camera and the second camera transmit the collected and output image signals to the IFE and IPE, and the IFE and IPE process the image signals respectively and output Image 1 and Image 2 is not shown in Figure 4 The image signal collected by the first camera is processed by the IFE and IPE to generate Image 1, and the image signal collected by the second camera is processed by the IFE and IPE to generate Image 2.

[0107] As can be seen from the description of the above embodiments, in some embodiments, the mobile phone can only use one camera to collect images at the same time. In this embodiment, when saving the image, only the currently collected image can be saved to the cache. That is, the above S303 can also be replaced by saving Image 1 collected by the first camera to the cache, or saving the image collected by the second camera to the cache.

[0108] In other embodiments, the camera application of the mobile phone can call multiple cameras to collect images at the same time, but only display the images collected by one of the cameras. In this embodiment, when saving the image, the images collected by each camera can be saved to the cache respectively. Then, the image collected by the camera currently used by the mobile phone in the cache is transmitted to the camera application for display. Exemplarily, if the mobile phone is currently using the first camera, the image collected by the first camera is transmitted to the camera application for display.

[0109] S304. The Image Processing HAL determines whether an event of switching the camera occurs on the mobile phone.

[0110] Taking the process of S301 - S303 in which the camera used by the mobile phone is the first camera as an example, the event of switching the camera can specifically correspond to the mobile phone switching from the first camera to the second camera.

[0111] The mobile phone can switch the camera when certain conditions are met. Exemplarily, when the current shooting magnification of the mobile phone changes, the mobile phone may need to switch the currently used camera. As can be seen from the above description, the shooting magnification ranges corresponding to different cameras are different. Therefore, when the current shooting magnification of the mobile phone switches from the shooting magnification range corresponding to one camera to the shooting magnification range corresponding to another camera, the mobile phone can switch the camera.

[0112] In some embodiments, the above S304 can specifically include: the mobile phone determines whether it detects that the current shooting magnification switches from the first shooting magnification to the second shooting magnification. The first shooting magnification and the second shooting magnification respectively belong to the shooting magnification ranges corresponding to two different cameras. Generally speaking, which camera the mobile phone is currently using, the image displayed on the mobile phone is the image collected by that camera.

[0113] The switching of the camera on the mobile phone can be performed between any two cameras. Then, the switching of the above shooting magnification may be from a smaller shooting magnification to a larger shooting magnification. In some embodiments, the above first shooting magnification and second shooting magnification respectively belong to the shooting magnification ranges corresponding to two different cameras, and can be specifically expressed as: the first shooting magnification is less than a preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification.

[0114] Among them, the above preset shooting magnification may be the critical value between the shooting magnification ranges respectively corresponding to the first camera and the second camera. Taking the first camera as the main camera and the second camera as the telephoto camera as an example, the preset shooting magnification may be 4X. Or, the above preset shooting magnification may also be an intermediate value between the shooting magnification ranges respectively corresponding to the first camera and the second camera. Exemplarily, taking the first camera as the wide-angle camera and the second camera as the telephoto camera as an example, the preset shooting magnification may be 5X.

[0115] In another example, the switching of the above shooting magnification may also be from a larger shooting magnification to a smaller shooting magnification. In some embodiments, the above first shooting magnification and second shooting magnification respectively belong to the shooting magnification ranges corresponding to two different cameras, and can be specifically expressed as: the first shooting magnification is greater than or equal to the preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification. Among them, the above preset shooting magnification may be the critical value between the shooting magnification ranges respectively corresponding to the first camera and the second camera. Taking the first camera as the telephoto camera and the second camera as the main camera as an example, the preset shooting magnification may be 4X. Or, the above preset shooting magnification may also be an intermediate value between the shooting magnification ranges respectively corresponding to the first camera and the second camera. Exemplarily, taking the first camera as the telephoto camera and the second camera as the wide-angle camera as an example, the preset shooting magnification may be 5X.

[0116] In some mobile phones, the mobile phone can achieve autofocus according to the distance between the object to be photographed and the camera. That is to say, the mobile phone can automatically select a suitable shooting magnification as the current shooting magnification of the mobile phone in combination with the distance between the object to be photographed and the camera. Different shooting magnifications correspond to different focal lengths of the camera, and different focal lengths are suitable for photographing objects within different distances. Therefore, when the distance between the object to be photographed and the camera is different, the corresponding suitable shooting magnification is also different. In some embodiments, the distance between the object to be photographed and the camera can be referred to as the shooting distance.

[0117] In some embodiments, the mobile phone pre-stores the mapping relationship between different shooting distances and shooting magnifications. The mapping relationship stores the shooting magnifications suitable for use at multiple shooting distances respectively. In this embodiment, the mobile phone can select a suitable shooting magnification according to the current shooting distance, and at the same time determine the shooting magnification range corresponding to the camera to which the shooting magnification belongs. If the shooting magnifications corresponding to the two shooting distances before and after the switch belong to the shooting magnification ranges corresponding to two different cameras respectively, it indicates that an event of switching cameras occurs on the mobile phone.

[0118] Since different lenses of the mobile phone correspond to different shooting magnification ranges, combined with the shooting magnifications suitable for different shooting distances, the mobile phone can convert the shooting magnification ranges corresponding to different cameras into the shooting distance ranges corresponding to different cameras. That is, in some other embodiments, the mobile phone can pre-store the shooting distance ranges corresponding to different cameras. In this embodiment, to determine whether the camera is switched, the mobile phone can specifically determine whether the two shooting distances before and after belong to the shooting distance ranges corresponding to two different cameras respectively. If the two shooting distances before and after belong to the shooting distance ranges corresponding to two different cameras respectively, it indicates that an event of switching cameras occurs on the mobile phone.

[0119] In some other embodiments, the above S304 may specifically include: the mobile phone determines whether it detects that the distance between the first camera and the object to be photographed switches from the first shooting distance to the second shooting distance. And, the first shooting distance and the second shooting distance respectively correspond to the shooting distance ranges of different cameras.

[0120] The switch of the shooting distance can be from a smaller shooting distance to a larger shooting distance. In some embodiments, that the first shooting distance and the second shooting distance respectively correspond to the shooting distance ranges of different cameras can be specifically expressed as: the first shooting distance is less than the preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance.

[0121] Among them, the preset shooting distance can be the critical value between the shooting distance ranges corresponding to the first camera and the second camera respectively. Taking the first camera as the main camera and the second camera as the telephoto camera as an example, the preset shooting distance can be the shooting distance suitable for 4X. Or, the above preset shooting magnification may also be an intermediate value between the shooting magnification ranges corresponding to the first camera and the second camera respectively. Exemplarily, taking the first camera as the wide-angle camera and the second camera as the telephoto camera as an example, the preset shooting distance can be the shooting distance suitable for 5X.

[0122] The switching of the shooting distance may also be from a larger shooting distance to a smaller shooting distance. Therefore, in some other embodiments, the first shooting distance and the second shooting distance respectively correspond to the shooting distance ranges of different cameras, which can be specifically expressed as: the first shooting distance is greater than or equal to the preset shooting distance, and the second shooting distance is less than the preset shooting distance. In some embodiments, the preset shooting distance may be the critical value between the shooting distance ranges corresponding to the first camera and the second camera respectively. Taking the first camera as a telephoto camera and the second camera as a main camera as an example, the preset shooting distance may be the shooting distance suitable for 4X. Or, the above preset shooting magnification may also be an intermediate value between the shooting magnification ranges corresponding to the first camera and the second camera respectively. Exemplarily, taking the first camera as a telephoto camera and the second camera as a wide-angle camera as an example, the preset shooting distance may be the shooting distance suitable for 5X.

[0123] In the technical solution proposed in the embodiments of the present application, the mobile phone can determine whether a camera switching event occurs by changing the shooting magnification or the shooting distance. In this way, it is convenient to perform color compensation on the image to be displayed in a timely manner when a camera switching event occurs, and avoid the jump of the brightness and color of the displayed image before and after the camera switching.

[0124] In some embodiments, the Image Processing HAL can obtain the current shooting magnification or shooting distance through the camera application, and determine whether a camera switching event occurs by detecting the change of the shooting magnification or shooting distance.

[0125] In some other embodiments, when the camera application detects that the change of the shooting magnification or shooting distance meets the condition for switching cameras, it can switch the cameras, and then send camera switching indication information to the Image Processing HAL. In this embodiment, the Image Processing HAL can determine whether a camera switching event occurs by judging whether it receives the camera switching indication information sent by the camera application. Specifically, the Image Processing HAL can determine that a camera switching event occurs when it receives the camera switching indication information sent by the camera application.

[0126] In some examples, if no camera switching event occurs on the mobile phone, the Image Processing HAL does not need to perform color compensation on the image to be displayed, but can directly transmit the image obtained from the cache to the camera application for display. Taking the current camera used by the mobile phone as the first camera as an example, the image to be displayed transmitted by the Image Processing HAL to the camera application is Image 1.

[0127] In some other examples, if a camera-switching event occurs on a mobile phone, the Image Processing HAL needs to perform color compensation on the image to be displayed and then transmit it to the camera application for display. In this embodiment, the image to be displayed transmitted by the Image Processing HAL to the camera application is the image after color compensation.

[0128] In some embodiments of the present application, for the sake of distinction, the image 1 that needs to be used when performing color compensation after the mobile phone switches cameras is denoted as the first image; the image 2 that needs to be color-compensated after the camera is switched is denoted as the second image. In the description of the following embodiments, when the judgment result of S304 is yes, the first image is used to replace image 1, and the second image is used to replace image 2 for illustration.

[0129] In some embodiments, the second image is the image to be displayed at a certain subsequent display moment (such as the target display moment). In some embodiments, the target display moment may be the next display moment after the current display moment; that is to say, the second image is the next frame of the image displayed by the mobile phone at the current display moment to be displayed. In some other embodiments, the target display moment may also be a certain display moment after that; that is to say, the second image is a certain frame of the image displayed by the mobile phone after the current display moment to be displayed.

[0130] In the embodiments of the present application, the image processing method is to solve the problem that when the mobile phone switches from displaying the image collected by the first camera to the image collected by the second camera before and after the camera is switched, the brightness and color of the image change abruptly. Therefore, it is necessary to perform color compensation on the image collected by the second camera after switching to the second camera and then display it. In some embodiments, the above-mentioned second image may specifically be the image collected by the second camera after the mobile phone switches from the first camera to the second camera.

[0131] Furthermore, in the method according to the embodiment of the present application, the first image collected by the first camera is used to perform color compensation on the second image, and then the second image after color compensation is displayed. In this way, after the mobile phone switches to the second camera, the displayed image will not have jumps in brightness and color compared with the image displayed before the switch, but will smoothly and naturally transition to display the image collected by the second camera. Therefore, the image collected by the second camera within a period of time after the mobile phone switches to the second camera can be color-compensated and then displayed. That is, the above-mentioned second image may specifically refer to any one of the first N frames of images collected by the second camera after the mobile phone switches from the first camera to the second camera. That is to say, in the image processing method proposed in the embodiment of the present application, for the first N frames of images collected by the camera used by the mobile phone after the camera switch (i.e., the second camera, hereinafter referred to as the post-switch camera), color compensation will be performed first and then the images will be displayed. For the images after the Nth frame collected by the post-switch camera, color compensation will no longer be performed, that is, the images after the Nth frame will be displayed with the original brightness and color of the images collected by the post-switch camera. In this way, color compensation is performed on the brightness and color of the images displayed on the mobile phone before and after the camera switch to achieve smooth and natural transition, and it will not consume too much power of the mobile phone.

[0132] The second image is introduced in detail in the above embodiment. Next, the selection of the first image will be described. In order to reduce the large difference in brightness and color of the images displayed on the mobile phone before and after the camera switch, the image already displayed on the mobile phone can be used to perform certain processing (such as color compensation) on the to-be-displayed second image, and then the processed image is displayed. In some embodiments, the above-mentioned first image may specifically refer to: the last frame of image collected and displayed by the first camera before the mobile phone switches from the first camera to the second camera. In this way, the second image displayed after the camera switch will not have a large difference in brightness and color compared with the first image displayed before the camera switch, and there will be no jump situation.

[0133] As can be seen from the above embodiments, in some embodiments, the camera application can start multiple cameras simultaneously for image acquisition, but only display the image captured by one of the cameras. Therefore, when performing color compensation on the image to be displayed after switching cameras, it is also possible to obtain the image captured in real time by the camera used by the mobile phone before switching (i.e., the first camera, which can be denoted as the pre-switching camera), and perform color compensation on the image to be displayed after switching cameras. In this embodiment, the above first image may also refer to: the image captured in real time by the first camera after the mobile phone switches from the first camera to the second camera. Exemplarily, when the second image is the i-th (i≤N) frame image captured by the second camera after the mobile phone switches from the first camera to the second camera, the first image may be the i-th frame image captured by the first camera after the mobile phone switches from the first camera to the second camera. In this way, the second image displayed after switching cameras can have no significant difference in brightness and color compared with the first image displayed before switching, and there will be no jumping situation.

[0134] In some embodiments, after the image processing HAL determines the event of switching cameras in S304 above, S305 may be executed.

[0135] S305. The image processing HAL obtains the first image and the second image from the cache.

[0136] In some embodiments of the present application, the image processing HAL can be used to perform color compensation on the image captured by the camera after switching (i.e., the second image) based on the image captured before switching (i.e., the first image) when the event of switching cameras occurs on the mobile phone.

[0137] S306. The image processing HAL respectively obtains the color information of the first image and the color information of the second image.

[0138] In some embodiments, the color information of the image can specifically be represented by the RGB values of each pixel of the image. The color information of the first image includes the RGB values of each pixel in the first image, and the color information of the second image includes the RGB values of each pixel in the second image.

[0139] Since the first image and the second image are captured by two cameras respectively, and the capture times may be different, the pixels of the first image and the second image cannot be guaranteed to correspond one by one. In this case, if the first image is directly used for color compensation of the second image, it may cause abnormal color problems in the compensated image. In some embodiments, before using the first image for color compensation of the second image, pixel alignment processing may be performed on the first image and the second image first.

[0140] Due to the different spatial positions of different cameras in the mobile phone settings, there are certain differences in the images collected for the same object to be photographed. Usually, before leaving the factory, the mobile phone can calibrate the internal and external parameters of each camera and store the calibrated internal and external parameters in the mobile phone. In some embodiments, the first image and the second image can be subjected to pixel alignment processing based on the calibrated internal and external parameters. Among them, the pixel alignment processing may include the first pixel alignment processing. Before S306, the above method further includes: based on the preset calibrated internal and external parameters, performing the first pixel alignment processing on the first image and the second image to obtain the first image and the second image after the first pixel alignment processing.

[0141] In the embodiment of the present application, the image to be displayed is the second image. In order to ensure that the finally displayed image (i.e., the compensated second image) is as pixel-aligned as possible with the original second image, the above first pixel alignment processing is specifically performed on the first image. In some embodiments, the above performing the first pixel alignment processing on the first image and the second image based on the preset calibrated internal and external parameters may specifically include: based on the preset calibrated internal and external parameters, mapping the first image to the second image to obtain the first image after the first pixel alignment processing. It should be noted that the specific implementation process of mapping the first image to the second image based on the preset calibrated internal and external parameters to obtain the first image after the first pixel alignment processing can refer to the description in the related art and will not be elaborated in the embodiment of the present application.

[0142] In some embodiments, after the mobile phone obtains the first image after the first pixel alignment processing, it can perform color compensation on the second image according to the color information of the first image after the first pixel alignment processing and the color information of the second image.

[0143] Since the preset calibrated internal and external parameters are pre-stored, compared with the method of registering using real-time collected images, the technical solution provided in the embodiment of the present application has higher processing efficiency. Moreover, the registration method in the related art requires relatively prominent feature points in the image to obtain a better pixel alignment effect. However, in the embodiment of the present application, the preset calibrated internal and external parameters are used for pixel alignment processing. Regardless of the content of the image, even if the image does not have relatively prominent feature points, a better pixel alignment effect can still be achieved.

[0144] After performing the first pixel alignment process on the first image and the second image based on the preset calibrated internal and external parameters, it cannot be guaranteed that the image pixels correspond one by one. Therefore, after performing the first pixel alignment process on the first image and the second image, a second pixel alignment process can be performed to further align the pixels between the first image and the second image. In some embodiments, after obtaining the first image after the first pixel alignment process, the method further includes: performing a second pixel alignment process on the second image and the first image after the first pixel alignment process respectively, to obtain the first image after the second pixel alignment process and the second image after the second pixel alignment process. The above pixel alignment process includes: the second pixel alignment process.

[0145] Further, in an embodiment of performing the second pixel alignment process on the first image and the second image, after the mobile phone obtains the first image after the first pixel alignment process, it can perform color compensation on the second image according to the first image after the second pixel alignment process and the second image after the second pixel alignment process.

[0146] In some embodiments, the second pixel alignment process may include performing a blur process on the image. Among them, any image blur process method can be used to perform the blur process on the image. Exemplarily, specifically performing a Gaussian blur process on the image can be used to perform the blur process on the image. In this way, after performing the first pixel alignment process on the first image and the second image based on the preset calibrated internal and external parameters, further performing the second pixel alignment process in a blur process manner can make the pixels of the image smoother, so as to obtain a better pixel alignment effect.

[0147] In other embodiments, the second pixel alignment process includes downsampling and blurring. Among them, performing downsampling on the image can reduce the video memory and computational amount of the image. In this way, after performing the first pixel alignment process on the first image and the second image based on the preset calibrated internal and external parameters, further performing the second pixel alignment process in a downsampling and blurring manner can reduce the error caused by inaccurate image alignment, and a better pixel alignment effect can be obtained. At the same time, the downsampling process can also reduce the computational amount of the mobile phone. The blur process can make the pixels of the image smoother.

[0148] Figure 6 The effect schematic diagram of performing the pixel alignment process on the first image and the second image is shown. Figure 6 The images 50 and 51 shown respectively correspond to the above first image and second image. The images 52 and 53 respectively correspond to the first image after the second pixel alignment process and the second image after the second pixel alignment process.

[0149] S307. The image processing HAL performs color compensation on the second image in the RGB space based on the color information of the first image and the color information of the second image, and obtains the compensated second image.

[0150] In some embodiments, S307 may specifically include: generating an RGB mapping relationship based on the color information of the first image and the color information of the second image; performing color compensation on the second image based on the RGB mapping relationship, and obtaining the compensated second image. In this way, it can be ensured that when performing color compensation on the second image, the compensation can be accurately performed.

[0151] In the embodiment of performing pixel alignment processing on the first image and the second image, an RGB mapping relationship may be generated for the first image after pixel alignment processing and the second image after pixel alignment processing in the above S307.

[0152] The above RGB mapping relationship can be used to represent the corresponding relationship between the pixels of the first image and the second image. In some embodiments, the above RGB mapping relationship may specifically be a 3D RGB mapping relationship. Exemplarily, the three-dimensional RGB mapping relationship can be implemented by a 3D LUT. In this way, it is convenient and fast to perform color compensation on the second image. Moreover, using a three-dimensional GRB mapping table for mapping has stronger expressive ability and more accurate mapping. Using a three-dimensional GRB mapping table for mapping can align the brightness and color of the first image and the second image at the same time, adapting to richer scenarios.

[0153] In some embodiments, a basic three-dimensional RGB mapping table (such as a basic 3D LUT) is stored in the mobile phone. Among them, the basic 3D LUT table may specifically be an initialized LUT table that is predefined and has no mapping effect. The above S307 may specifically include: updating the basic 3D LUT table based on the color information of the first image and the second image, and obtaining the updated 3D LUT table. Then, applying the updated 3D LUT table obtained this time to the second image to obtain the compensated second image.

[0154] After the mobile phone switches the camera, in the first few frames of images captured by the switched rear camera, it usually has not fully converged. If only one frame of image captured by the switched front camera and one frame of image captured by the switched rear camera are used to update the basic 3D LUT table, and the updated 3D LUT table is used to perform color compensation on the first N frames of images captured by the switched rear camera, problems of inaccurate compensation may occur. As a result, compared with the last frame of image captured by the first camera displayed before switching the camera, there are still problems of brightness and color jumps in the compensated second image. Therefore, in some embodiments, the first few frames of images captured by the switched rear camera (such as the first S frames of images, S < N), and the first image can be used to update the 3D LUT table multiple times. After the 3A of the mobile phone is converged, the update of the 3D LUT table can be stopped. For the images from the (S + 1)-th frame to the N-th frame captured by the switched rear camera, the 3D LUT table obtained from the last update (i.e., the 3D LUT table updated after the S-th frame) can be directly used for color compensation.

[0155] In some embodiments, generating the RGB mapping relationship based on the color information of the first image and the color information of the second image may specifically include: when i ≤ S, based on the color information of the first image and the color information of the second image, determining the color mapping relationship between each pixel point of the first image and the second image. Then, based on the color mapping relationship between each pixel point, updating the previous three-dimensional RGB mapping table to obtain the three-dimensional RGB mapping table updated after the i-th frame. Among them, when i = 1, the previous three-dimensional RGB mapping table is the basic three-dimensional RGB mapping table. In this solution, using the first S frames of images captured by the switched rear camera to update the basic three-dimensional GRB mapping table can update the three-dimensional RGB mapping table when the 3A of the mobile phone has been converged. When using the updated three-dimensional RGB mapping table to perform color compensation on the second image, a better compensation effect can be obtained.

[0156] Figure 7 Shows a schematic diagram of the update of the 3D LUT table in some embodiments. In Figure 7 it, the basic 3D LUT table (RGB) is updated to obtain the updated 3D LUT table (R1G1B1).

[0157] The three-dimensional RGB mapping table can be the above 3D LUT table. The output of the 3D LUT table can be expressed as: Output RGB value = LUT(R input, G input, B input). That is to say, according to the input color (R value, G value, B value), in the updated 3D LUT table, a unique corresponding another color (R value, G value, B value) can be found.

[0158] In addition, taking an 8-bit bitmap as an example, there are 256 possible values for the corresponding RGB values. Correspondingly, by setting the size of the 3D LUT table to 257*257*257, the corresponding RGB value can be found for each color. In this way, the storage space occupied by the 3D LUT table is relatively large. Therefore, in order to reduce the storage space occupied by the 3D LUT table, the data volume of the 3D LUT table can be reduced by reducing sampling. Exemplarily, the size of the 3D LUT table can be set to 33*33*33. In some other embodiments, the size of the 3D LUT table can be set to 17*17*17.

[0159] Please refer to Figure 8 , which shows the process of finding the output result (i.e., color 2) through the 3D LUT table, taking the size of the 3D LUT table as 17*17*17 and the input color 1 (RGB value (0, 191, 255)) as an example. Since a 3D LUT table with a size of 17*17*17 is used to represent 256*256*256 colors, the input RGB value needs to be normalized first: RGB / 255 = (0, 0.749016, 1). Then it is mapped to the grid of 17*17*17: (0, 0.749016, 1)*16 = (0, 11.9843136, 16). Then calculate the position of the grid: 1 + 12*17 + 16*17*17 = 4829. Finally, look up the RGB value stored at the position corresponding to this grid in the 3D LUT table as the output RGB value, that is Figure 8 the color 2 (145, 163, 193) shown in

[0160] In Figure 8 the embodiment shown, during the process of calculating the grid position according to the mapped values of the grid (0, 11.9843136, 16), in order to simplify the calculation process, the non-integer values are directly rounded and then calculated. This method is prone to color deviation problems. In some other embodiments, for the non-integer values in the mapped values of the grid (0, 11.9843136, 16), the grid position can be directly calculated. Since the mapped values of the grid are not rounded, the calculated grid position is also non-integer. Finally, when looking up the RGB value corresponding to this grid position in the 3D LUT table, linear interpolation can be used to find the RGB value corresponding to the grid position as the output result. In this way, a more accurate RGB value can be found. Thus, when performing color compensation on the second image, a better and more accurate compensation effect can be obtained.

[0161] In the technical solution provided in the above embodiments, by storing the 3D LUT table in a way of reducing sampling, the occupied storage space can be reduced.

[0162] In some embodiments, applying the updated 3D LUT table to the second image to obtain the compensated second image may specifically include: in the RGB space, weighted-fusing the color information of the first image into the color information of the second image using a preset weight to obtain the compensated second image. Wherein, the magnitude of the preset weight is inversely correlated with i.

[0163] Wherein, when weighted-fusing the color information of the first image into the color information of the second image using a preset weight, it is necessary to weight-fuse the color information of each pixel point of the first image into the color information of the corresponding pixel point of the second image according to the one-to-one correspondence between each pixel point of the first image and the second image. In this way, the brightness and color of the first image and the second image can be aligned simultaneously to adapt to a richer scenario.

[0164] The smaller the value of i, the more forward the image to be displayed of the corresponding second image after switching the camera. To achieve a smooth and natural transition, when performing color compensation, for the second image that is more forward after switching the camera, the color proportion of the first image used for weighted fusion should be heavier. And for the second image that is more backward after switching the camera, the color proportion of the first image used for weighted fusion should be lighter. Only in this way can the color of the first frame image displayed after switching the camera be closest to the color of the first image, and the color of the image after switching the camera gradually changes towards the original color of the second image. Until the (N + 1)th frame image after switching the camera, the mobile phone starts to display the original color of the image collected by the switched camera. And the larger the preset weight, the higher the color proportion of the first image during weighted fusion. Therefore, the smaller i is, the larger the preset weight; the larger i is, the preset weight can gradually decrease.

[0165] Exemplarily, when the mobile phone performs weighted fusion on the ith frame of the second image, the preset weight used can be expressed as: 1 - i / N. In other embodiments, the preset weight can also be expressed by other expressions.

[0166] In the embodiments where color compensation is performed on the second image using a 3D LUT table, specifically, the updated 3D LUT table can be applied to the second image to obtain an intermediate image. Then, the intermediate image and the second image are weighted-fused, and the weight used for fusion is the above-mentioned preset weight.

[0167] In some embodiments, the above-mentioned operation of weighted-fusing the color information of the first image into the color information of the second image in the RGB space to obtain the compensated second image can be specifically implemented by means of alpha blending.

[0168] In the technical solution provided by the embodiments of the present application, when performing color compensation on the second image, weighted fusion can be performed. And according to the order of the images collected by the switched rear camera after switching the camera for the second image, the size of the preset weight is determined, so that smooth and natural transitions of the brightness and color of the images before and after switching the camera can be achieved, avoiding jump situations. Furthermore, the visual experience of the user during the shooting process can be improved.

[0169] S308. The Image Processing HAL transmits the compensated second image to the camera application.

[0170] Correspondingly, the camera application receives the compensated second image sent by the Image Processing HAL.

[0171] S309. The camera application displays the compensated second image.

[0172] As can be seen from the description in the above embodiments, the second image is the image to be displayed at the target display moment. Therefore, in some embodiments, the camera application displays the compensated second image at the target display moment.

[0173] In the technical solution provided by the embodiments of the present application, the first image is used to perform color compensation on the first N frames of images collected by the second camera after switching the camera, and then display is performed. Thus, when the images displayed before and after the mobile phone switches the camera are switched, the color change of the images will not jump, but can change smoothly and naturally.

[0174] In addition, please continue to refer to Figure 5 , when the judgment result of S304 above is no, the mobile phone can execute S310 - S312.

[0175] S310. The Image Processing HAL obtains the first image from the cache.

[0176] S311. The Image Processing HAL transmits the first image to the camera application.

[0177] Correspondingly, the camera application receives the first image sent by the Image Processing HAL.

[0178] If no camera switching event occurs on the mobile phone, the Image Processing HAL can directly transmit the first image collected by the first camera to the camera application for display.

[0179] S312. The camera application displays the first image.

[0180] In some other embodiments, the image processing HAL of the mobile phone can also first obtain the first image and the second image from the cache, and then determine whether an event of switching the camera occurs on the mobile phone. In this embodiment, if it is determined that an event of switching the camera occurs on the mobile phone, after performing color compensation on the image captured by the switched camera (i.e., the second image), it is then transmitted to the camera application for display. If it is determined that the mobile phone does not have an event of switching the camera, no color compensation is performed on the image, but it is directly transmitted to the camera application for display.

[0181] In the technical solution proposed in the embodiments of the present application, when an event of switching the camera does not occur on the mobile phone, the image processing HAL may not process the image captured by the camera. Instead, according to the camera currently used by the mobile phone, the image captured by this camera is transmitted to the camera application for display.

[0182] In addition, in some abnormal situations, if color compensation is performed on the image after switching the camera, it is likely to affect the display effect of the image. Therefore, when an event of switching the camera occurs on the mobile phone, before performing color compensation on the second image, it is also possible to determine whether the first image and / or the second image is abnormal to determine whether color compensation needs to be performed on the second image. Please refer to Figure 9 In some embodiments, after S305 and before S306, the above method may further include S401:

[0183] S401. The image processing HAL determines whether the first image and / or the second image meets a preset condition.

[0184] If the determination result in S401 is negative, it means that the first image and / or the second image does not meet the preset condition, and there is no abnormal situation for the first image and / or the second image. At this time, the image processing HAL can perform color compensation on the second image and then display it. In this embodiment, S306 - S309 above can be executed when the determination result in S401 is positive.

[0185] In some embodiments, the above preset condition can be set in combination with the parameter information of the first image and / or the second image. Specifically, the preset condition can be set in combination with the parameter information related to the image brightness. Exemplarily, the parameter information related to the image brightness may include the exposure, brightness, etc. of the image.

[0186] The exposure of an image reflects the amount of light received by the camera's photosensitive element during the exposure time when the camera captures the image. Simply put, the higher the exposure of the image, the greater the brightness of the image. If the exposure of the first image is abnormal and / or the exposure of the second image is abnormal, when using the color information of the first image and the color information of the second image to perform color compensation on the second image in the RGB space, it is possible that the compensated image has a large difference from the original exposure of the second image. Therefore, in some embodiments, the above preset conditions may specifically include: the exposure corresponding to the first image and / or the second image exceeds the preset exposure range. Among them, the preset exposure range can be set according to the actual situation.

[0187] The brightness of an image represents the degree of brightness of the image. In some scenarios, when the mobile phone switches cameras, the brightness difference between the displayed images is large. Then, when using the color information of the first image and the color information of the second image to perform color compensation on the second image in the RGB space, it is possible that the compensated image has a large difference from the original brightness of the second image. That is to say, there is a large difference between the compensated second image and the true display effect of the image captured by the second camera, and it is easy to have the problem of image distortion. Thus, it brings a bad visual experience to the user. Therefore, in some other embodiments, the above preset conditions may specifically include: the brightness difference between the first image and the second image is greater than the preset brightness difference. Among them, the preset brightness difference can be set according to the actual situation.

[0188] In some embodiments, the brightness value of an image can be determined according to the Y component of the image in the YUV space. In some embodiments, calculating the brightness difference between the first image and the second image may specifically include: statistically analyzing the histogram information of the Y component of the first image in the YUV space, and determining the average value and / or variance of the Y component of the first image according to the histogram information of the Y component of the first image. Statistically analyzing the histogram information of the Y component of the second image in the YUV space, and determining the average value and / or variance of the Y component of the second image according to the histogram information of the Y component of the second image. Then, compare the average value and / or variance of the Y component of the first image with the average value and / or variance of the Y component of the second image respectively. When at least one parameter (the average value and variance of the Y component) exceeds the corresponding threshold, it is determined that the brightness difference between the first image and the second image is greater than the preset brightness difference. In this solution, using the Y component of the image in the YUV space to determine the brightness information of the image can conveniently and quickly determine the brightness difference between the first image and the second image.

[0189] In some other embodiments, when the judgment result of S401 is yes, it means that the first image and / or the second image meet the preset conditions, that is, there may be abnormal conditions in the first image and / or the second image. At this time, even if the mobile phone sends an event to switch the camera, the image processing HAL may not perform color compensation on the image to be displayed. Instead, it directly transmits the image collected by the second camera, that is, the second image, to the camera application. As Figure 9 shown in S402 - S404.

[0190] S402. The image processing HAL obtains the second image from the cache.

[0191] S403. The image processing HAL transmits the second image to the camera application.

[0192] S404. The camera application displays the second image.

[0193] In this way, when there are abnormal conditions in the image collected by the camera, the problem of poor display effect caused by color compensation for the second image can be avoided.

[0194] Next, in combination with Figure 10 the steps executed by the image processing HAL and the camera application in the image processing method proposed in this application will be described. After the mobile phone switches from the first camera to the second camera, the following steps can be executed.

[0195] S601_1. Obtain the image A collected by the first camera.

[0196] In some embodiments, the image A may be the last frame of the image collected and displayed by the first camera before the mobile phone switches the camera.

[0197] In some other embodiments, the mobile phone simultaneously collects images through the first camera and the second camera, and only displays the image collected by one of the cameras. The image A may also be the image collected in real time by the first camera after the mobile phone switches the camera. In this embodiment, the image A may also be represented as the image A_i. The image A_i and the image B_i are collected at the same time.

[0198] S601_2. Obtain the image B_i collected by the second camera.

[0199] It should be noted that the mobile phone may execute S601_1 first and then S601_2; it may also execute S601_2 first and then S601_1; or, the mobile phone may also execute S601_1 and S601_2 simultaneously.

[0200] S602. Judge whether i ≤ N.

[0201] Wherein, N is a positive integer; N ≥ 2.

[0202] If the judgment result of S602 is yes, color compensation for the image B_i is not required, and at the corresponding display moment, the image B_i can be directly displayed; as in S603.

[0203] S603. Display the image B_i.

[0204] If the judgment result of S602 is yes, that is, i ≤ N, it means that the image B_i is one of the first N frames of images captured by the switched - on camera (i.e., the second camera) after switching the camera, and color compensation for the image B_i is required. At this time, S604 can be executed.

[0205] S604. Calculate the brightness of the image A and the image B_i respectively.

[0206] S605. Determine whether the image A and / or the image B_i has abnormal exposure, or whether the brightness difference between the image A and the image B_i is greater than a preset brightness difference.

[0207] If the judgment result of S605 is yes, it means that the image A and / or the image B_i is abnormal. At this time, color compensation for the image B_i can be not performed, but the image B_i can be directly displayed, that is, S603.

[0208] If the judgment result of S605 is no, it means that the image A and / or the image B_i is not abnormal. At this time, color compensation for the image B_i can be performed; exemplarily, S606 can be executed.

[0209] S606. Determine whether i ≤ S.

[0210] Where S is a positive integer and S < N. In this embodiment, after the mobile phone switches the camera, the first S frames of images captured by the second camera are used to update the basic 3D LUT table. Therefore, when the judgment result of S606 is yes, S607 - S609 can be executed.

[0211] If the judgment result of S606 is no, it means that the image B_i does not belong to the first S frames of images captured by the second camera after switching the camera. At this time, it may not be necessary to use this frame of image B_i to update the 3D LUT table. Therefore, the 3D LUT table obtained by directly using the image B_S can be used to directly perform color compensation on the current frame B_i. As Figure 10 shown, when the judgment result of S606 is no, it can jump to S610.

[0212] S607. After registering the image A and the image B based on the preset calibration internal and external parameters, perform downsampling processing, and then perform Gaussian blur processing.

[0213] S608. Traverse the Gaussian-blurred images A and B_i pixel by pixel to determine the color mapping relationship of each pixel point between the images.

[0214] S609. Update the previous 3D LUT table based on the color mapping relationship of each pixel point to obtain the updated 3D LUT table for the i-th frame.

[0215] Among them, when i = 1, the previous 3D LUT table is the basic 3D LUT table.

[0216] Exemplarily, updating the basic 3D LUT table based on the color mapping relationship of each pixel point to obtain the updated 3D LUT table for the i-th frame may specifically include: when i = 1, update the basic 3D LUT table based on the color mapping relationship of each pixel point between image A and B_1 to obtain the 3D LUT table 1. When i = 2, update the 3D LUT table 1 based on the color mapping relationship of each pixel point between image A and B_2 to obtain the 3D LUT table 2. And so on, until when i = S, update the 3D LUT table (S - 1) to obtain the 3D LUT table S. Among them, the specific implementation process of updating the previous 3D LUT table based on the color mapping relationship of each pixel point may refer to the description in the related art.

[0217] In another example, updating the basic 3D LUT table based on the previous mapping relationship of each pixel point to obtain the updated 3D LUT table for the i-th frame may also be implemented in the following way: when i = 1, update the basic 3D LUT table based on the color mapping relationship of each pixel point between image A and B_1 to obtain the 3D LUT table 1. When i = 2, update the basic 3D LUT table based on the color mapping relationship of each pixel point between image A and B_2 to obtain the 3D LUT table 2. And so on, until when i = S, update the basic 3D LUT table based on the color mapping relationship of each pixel point between image A and B_S to obtain the 3D LUT table S.

[0218] S610. Apply the updated 3D LUT table for the i-th frame to the image B_i to obtain the intermediate image B_A.

[0219] It can be understood that when i > S, the updated 3D LUT table for the i-th frame is the updated 3D LUT table for the S-th frame.

[0220] In some examples, when i = 1, the above S610 may specifically include: applying the 3D LUT table 1 to the image B_1 to obtain the intermediate image B_A.

[0221] When i = 2, the above S610 may specifically include: applying the 3D LUT table 2 to the image B_2 to obtain an intermediate image B_A.

[0222] When i = S, the above S610 may specifically include: applying the 3D LUT table S to the image B_S to obtain an intermediate image B_A.

[0223] When i = S + 1, the above S610 may specifically include: applying the 3D LUT table S to the image B_S+1 to obtain an intermediate image B_A.

[0224] When i = N, the above S610 may specifically include: applying the 3D LUT table S to the image B_N to obtain an intermediate image B_A. And so on.

[0225] S611. Perform weighted fusion on the intermediate image B_A and the image B_i to obtain a compensated image B_i; the weight is a preset weight.

[0226] S612. Display the compensated image B_i.

[0227] The specific implementation process of the above S601 - S612 may refer to the description in the above embodiments.

[0228] In the technical solution provided by the embodiments of the present application, the image A collected by the first camera is used to perform color compensation on the first N frames of images B_i collected by the second camera after switching the camera, and then display. Thus, the image switching and color change of the image caused by the mobile phone switching the camera will not jump, but can change smoothly and naturally.

[0229] Figure 11 Shows a schematic diagram of the images displayed by the mobile phone before and after switching the camera. Before the mobile phone switches the camera, the currently used camera of the mobile phone is the first camera, and the displayed image is image 0. After the mobile phone switches the camera, the currently used camera of the mobile phone switches from the first camera to the second camera. Under certain conditions, the images displayed by the mobile phone are sequentially the compensated image 1, the compensated image 2,... the compensated image N, and the image N + 1, etc.

[0230] In some embodiments, the above algorithm for performing color compensation on the images collected after switching the camera and then displaying after the mobile phone switches the camera may be named the MCC algorithm. This method is applied after IPE and before sending for display.

[0231] In the above embodiments, the image processing method is described by taking its application to electronic devices such as mobile phones as an example. In some other embodiments, the above image processing method can also be applied to cloud devices such as servers. In this embodiment, cloud devices such as servers can obtain a first image and a second image from electronic devices such as mobile phones. After performing color compensation on the second image in the RGB space based on the color information of the first image and the color information of the second image to obtain the compensated second image, the compensated second image can be returned to the electronic devices such as mobile phones for display. The specific implementation processes of the steps of the above image processing method implemented by cloud devices such as servers are similar to the descriptions of the steps in the above embodiments and will not be elaborated here.

[0232] Some other embodiments of the present application provide an electronic device (such as a mobile phone). The electronic device may include: a display screen, a memory, and one or more processors. The display screen and the memory are coupled to the processor. The display screen is used to display the interface of the electronic device, such as the image preview interface collected by the camera in the camera application. The memory is further used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform each function or step executed by the mobile phone in the above method embodiments. The structure of the electronic device may refer to Figure 3 the structure of the electronic device 100 shown.

[0233] Embodiments of the present application also provide a chip system, as Figure 12 shown. The chip system 1000 includes at least one processor 1001 and at least one interface circuit 1002. The processor 1001 and the interface circuit 1002 can be interconnected by a line. For example, the interface circuit 1002 can be used to receive signals from other devices (such as the memory of a computer). For another example, the interface circuit 1002 can be used to send signals to other devices (such as the processor 1001). Exemplarily, the interface circuit 1002 can read the instructions stored in the memory and send the instructions to the processor 1001. When the instructions are executed by the processor 1001, the computer can perform each step in the above embodiments. Of course, the chip system may further include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0234] Embodiments of the present application also provide a computer-readable storage medium, which includes computer instructions. When the computer instructions run on the above electronic device (such as a mobile phone), the electronic device is enabled to perform each function or step executed by the mobile phone in the above method embodiments.

[0235] The embodiments of the present application also provide a computer program product. When the computer program product runs on a computer, it enables the computer to execute each function or step that the mobile phone executes in the above method embodiments. Among them, the computer may be an electronic device, such as a mobile phone.

[0236] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0237] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0238] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0239] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0240] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the embodiments of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0241] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that, The method is applied to an electronic device, which includes a first camera and a second camera with the same shooting direction; the method includes: In response to switching from the first camera to the second camera, obtain a first image and a second image, where the first image is an image captured by the first camera, and the second image is an image to be displayed captured by the second camera after switching the camera; Obtain the color information of the first image and the color information of the second image; Based on the color information of the first image and the color information of the second image, perform color compensation on the second image in the Red-Green-Blue (RGB) space to obtain the compensated second image; Display the compensated second image.

2. The method according to claim 1, wherein The second image is one of the N frames of images to be displayed captured by the second camera after switching the camera, where N is a positive integer.

3. The method according to claim 1 or 2, characterized in that The first image is the last frame of image captured and displayed by the first camera before the electronic device switches from the first camera to the second camera; Or, Both the first camera and the second camera are in the startup state and operate independently; the first image is an image captured in real time by the first camera after the electronic device switches from the first camera to the second camera.

4. The method according to any one of claims 1-3, characterized in that, Before the step of in response to switching from the first camera to the second camera, obtain a first image and a second image, the method further includes: In response to the current shooting magnification of the electronic device switching from a first shooting magnification to a second shooting magnification, switch from the first camera to the second camera; where the first shooting magnification is less than a preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification; or the first shooting magnification is greater than or equal to the preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification; Or, In response to the distance between the first camera and the object to be photographed switching from a first shooting distance to a second shooting distance, switch from the first camera to the second camera; where the first shooting distance is less than a preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance; or the first shooting distance is greater than or equal to the preset shooting distance, and the second shooting distance is less than the preset shooting distance.

5. The method according to any one of claims 1-4, characterized in that The electronic device stores the preset calibrated internal and external parameters of each second camera; Before the step of obtain the color information of the first image and the color information of the second image, the method further includes: Based on the preset calibrated internal and external parameters, perform a first pixel alignment process on the first image and the second image to obtain the first image and the second image after the first pixel alignment process; Perform a second pixel alignment process on the first image and the second image after the first pixel alignment process to obtain the first image after the second pixel alignment process and the second image after the second pixel alignment process; the second pixel alignment process includes a blurring process, or the second pixel alignment process includes a downsampling process and a blurring process.

6. The method according to claim 5, wherein The performing color compensation on the second image in the red-green-blue RGB space based on the color information of the first image and the color information of the second image to obtain the compensated second image includes: Generating an RGB mapping relationship based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process; Performing color compensation on the second image according to the RGB mapping relationship to obtain the compensated second image.

7. The method according to claim 6, characterized in that, The RGB mapping relationship is a three-dimensional GRB mapping table.

8. The method according to claim 7, wherein The second image is the i-th image among N images collected by the second camera after switching the camera, N is a positive integer, and i ≤ N; the electronic device stores a basic three-dimensional RGB mapping table; The generating an RGB mapping relationship based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process includes: When i does not exceed S, determining the color mapping relationship between each pixel point of the first image and the second image based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process; where S is a positive integer and S < N; Updating the previous three-dimensional RGB mapping table based on the color mapping relationship between each pixel point to obtain the updated three-dimensional RGB mapping table of the i-th frame; the RGB mapping relationship includes the updated three-dimensional RGB mapping table of the i-th frame; where when i = 1, the previous three-dimensional RGB mapping table is the basic three-dimensional RGB mapping table.

9. The method according to claim 8, wherein The performing color compensation on the second image according to the RGB mapping relationship to obtain the compensated second image includes: Applying the updated three-dimensional RGB mapping table of the i-th frame to the second image to obtain the intermediate image of the i-th frame; when i > S, the updated three-dimensional RGB mapping table of the i-th frame is the updated three-dimensional RGB mapping table of the S-th frame; Performing weighted fusion of the intermediate image of the i-th frame and the second image using a preset weight to obtain the compensated second image; where the magnitude of the preset weight is inversely correlated with i.

10. The method according to any one of claims 1-5, characterized in that The second image is the i-th image among N images collected by the second camera after switching the camera, N is a positive integer, and i ≤ N; The performing color compensation on the second image in the red-green-blue RGB space based on the color information of the first image and the color information of the second image to obtain the compensated second image includes: In the RGB space, the color information of the first image is weighted and fused into the color information of the second image by using a preset weight to obtain the compensated second image; wherein, the magnitude of the preset weight is inversely correlated with i.

11. The method according to any one of claims 1-10, characterized in that, Before obtaining the color information of the first image and the color information of the second image, the method further includes: judging whether the first image and / or the second image meets a preset condition; the preset condition includes: the exposure corresponding to the first image and / or the second image exceeds a preset exposure range, or the brightness difference between the first image and the second image is greater than a preset brightness difference; The method further includes: in the case where the first image and / or the second image does not meet the preset condition, performing the steps of obtaining the color information of the first image and the color information of the second image, and performing color compensation on the second image in the red, green, blue (RGB) space based on the color information of the first image and the color information of the second image to obtain the compensated second image.

12. An electronic device, characterized in that, The electronic device includes a display screen, a processor, and a memory; the display screen and the memory are respectively coupled to the processor; The display screen is used for displaying the interface of the electronic device; the memory is used for storing computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used for storing computer instructions; when the computer instructions are executed by the processor of the electronic device, the electronic device executes the method according to any one of claims 1-11.

Images