Image processing method and electronic device

Abstract

Embodiments of the present application provide an image processing method and an electronic device, and relate to the technical field of images, which are used to reduce the changes in brightness and color of a displayed image when switching between cameras during shooting. The method is applied to an electronic device, which includes a first camera and a second camera, and the shooting directions of the first camera and the second camera are the same. The method includes: in response to switching from the first camera to the second camera, acquiring a first image and a second image, the first image being an image captured by the first camera, and the second image being an image to be displayed captured by the second camera after switching between cameras; acquiring color information of the first image and color information of the second image; based on the color information of the first image and the color information of the second image, performing color compensation on the second image in a red-green-blue (RGB) space to obtain a compensated second image; and displaying the compensated second image.

Description

Technical Field

[0001] This application relates to the field of image technology, and in particular to an image processing method and an electronic device. Background Technology

[0002] With the development of technology, users have increasingly higher requirements for shooting with mobile phones and other electronic devices. As a result, many mobile phones and other electronic devices now feature multiple cameras instead of just one. The combination of multiple cameras can meet the shooting needs in different scenarios.

[0003] In some shooting scenarios, electronic devices need to switch cameras. When switching to a different camera, the image displayed on the electronic device also changes from the image captured by the first camera to the image captured by the second. However, different cameras may have differences in shooting angle, aperture, light intake, and image sensor response to color and brightness. Therefore, when an electronic device switches cameras, there may be abrupt changes in the brightness and color of the displayed image before and after the switch. Summary of the Invention

[0004] This application provides an image processing method and electronic device for reducing changes in brightness and color of the displayed image when switching cameras during the shooting process.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, an image processing method is provided, applied to an electronic device including a first camera and a second camera, wherein the first camera and the second camera have the same shooting direction. The method includes:

[0007] The electronic device responds to switching from a first camera to a second camera, acquiring a first image and a second image. The first image is the image captured by the first camera, and the second image is the image to be displayed, captured by the second camera after the camera switch. Due to differences between the two cameras, when the electronic device switches from displaying the image captured by the first camera to displaying the image captured by the second camera, there are significant differences in brightness and color, causing abrupt transitions. Therefore, after acquiring the first and second images, the electronic device can obtain the color information of both the first and second images. Then, based on the color information of the first and second images, color compensation is performed on the second image in the RGB color space to obtain a compensated second image. Finally, at the time when the second image needs to be displayed, the compensated second image is displayed.

[0008] In this solution, the image captured by the camera before switching (i.e., the first image) is used, and the image captured by the camera after switching (i.e., the second image) is color-compensated before being displayed. This ensures that when the electronic device switches between images displayed before and after switching cameras, the color changes are not abrupt, but rather maintain a consistent or similar brightness and color to the image captured by the camera before switching.

[0009] In one possible implementation of the first aspect, the second image is a frame to be displayed from N frames captured by the second camera after switching cameras, where N is a positive integer. In this scheme, after switching cameras, the first image is used to compensate for the first N frames captured by the second camera after switching, so that the color change of the images displayed by the electronic device before and after switching cameras will not be abrupt, but can smoothly and naturally transition to the brightness and color of the second image.

[0010] In one possible implementation of the first aspect, the first image is the last frame captured and displayed by the first camera before the electronic device switches from the first camera to the second camera. This ensures that the brightness and color of the image captured by the second camera after the switch are similar to the brightness and color of the image captured by the first camera before the switch, thus preventing abrupt color changes.

[0011] In one possible implementation of the first aspect, both the first and second cameras are activated, and they operate independently. In this embodiment, the first image is an image captured in real-time by the first camera after the electronic device switches from the first camera to the second camera. This ensures that the brightness and color of the image captured by the second camera after the switch are similar to the brightness and color of the image captured by the first camera before the switch, thus preventing abrupt color changes.

[0012] In one possible implementation of the first aspect, before acquiring the first and second images in response to switching from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to the electronic device's current shooting magnification changing from a first shooting magnification to a second shooting magnification. Wherein, the first shooting magnification is less than a preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification. Thus, in scenarios where camera switching is triggered by switching from a lower shooting magnification to a higher shooting magnification, the electronic device can use the image acquired by the camera before switching, perform color compensation on the image acquired by the camera after switching, and then display it. This ensures that the color change of the displayed image does not abruptly change before and after switching cameras.

[0013] In one possible implementation of the first aspect, before acquiring the first image and the second image in response to switching from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to the current shooting magnification of the electronic device switching from the first shooting magnification to the second shooting magnification. Wherein, the first shooting magnification is greater than or equal to a preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification. Thus, in scenarios where camera switching is triggered by switching from a higher shooting magnification to a lower shooting magnification, the electronic device can use the image acquired by the camera before switching, perform color compensation on the image acquired by the camera after switching, and then display it. This ensures that the color change of the displayed image does not abruptly occur before and after switching cameras.

[0014] In one possible implementation of the first aspect, before acquiring the first and second images in response to switching from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to a change in the distance between the first camera and the subject from a first shooting distance to a second shooting distance. The first shooting distance is less than a preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance. Thus, in scenarios where a camera switch is triggered by a change from a smaller shooting distance to a larger shooting distance, the electronic device can use the image acquired by the camera before the switch, perform color compensation on the image acquired by the camera after the switch, and then display it. This ensures that the color changes in the displayed image do not abruptly change before and after the camera switch.

[0015] In one possible implementation of the first aspect, before acquiring the first image and the second image in response to switching from the first camera to the second camera, the method may further include: switching from the first camera to the second camera in response to a change in the distance between the first camera and the subject from a first shooting distance to a second shooting distance. The first shooting distance is greater than or equal to a preset shooting distance, and the second shooting distance is less than the preset shooting distance. Thus, in scenarios where a camera switch is triggered by a change from a larger shooting distance to a smaller shooting distance, the electronic device can use the image acquired by the camera before the switch, perform color compensation on the image acquired by the camera after the switch, and then display it. This ensures that the color change of the displayed image does not abruptly change before and after the camera switch.

[0016] In one possible implementation of the first aspect, the electronic device stores preset calibration intrinsic and extrinsic parameters for each of the second cameras. In this embodiment, before acquiring the color information of the first image and the color information of the second image, the method may further include: based on the preset calibration intrinsic and extrinsic parameters, performing a first pixel alignment process on the first image and the second image to obtain a first image and a second image after the first pixel alignment process. This ensures that the pixels of the first image and the second image are aligned. Performing a second pixel alignment process on the first image and the second image after the first pixel alignment process yields a first image and a second image after the second pixel alignment process. The second pixel alignment process includes blurring. Blurring can smooth the pixels of the image, thereby further aligning the pixels of the first image and the second image and obtaining a better pixel alignment effect. Alternatively, the second pixel alignment process includes downsampling and blurring. Performing downsampling and blurring on the image can reduce errors caused by inaccurate image alignment and obtain a better pixel alignment effect. Downsampling can also reduce the computational load of the electronic device. Blurring can make the pixels of the image smoother.

[0017] In one possible implementation of the first aspect, the aforementioned method of performing color compensation on the second image in the RGB color space based on the color information of the first and second images to obtain a compensated second image can specifically include: since pixel alignment processing has already been performed on the first and second images, the pixels of the image after pixel alignment processing should be aligned one-to-one. Furthermore, an RGB mapping relationship can be generated based on the color information of the first image after second pixel alignment processing and the color information of the second image after second pixel alignment processing. Then, color compensation is performed on the second image according to the RGB mapping relationship to obtain the compensated second image. This ensures that color compensation on the second image can be performed accurately.

[0018] In one possible implementation of the first aspect, the RGB mapping relationship is represented by a three-dimensional GRB mapping table. This facilitates convenient and quick color compensation of the second image. Furthermore, using a three-dimensional GRB mapping table provides stronger expressive power and more accurate mapping. Mapping using a three-dimensional GRB mapping table can simultaneously align the brightness and color of the first and second images, adapting to a wider range of scenarios.

[0019] In one possible implementation of the first aspect, the aforementioned three-dimensional GRB mapping table can be stored in a desampling manner. This reduces the space occupied by the three-dimensional GRB mapping table in the electronic device.

[0020] In one possible implementation of the first aspect, the second image is the i-th frame of N frames captured by the second camera after switching cameras, where N is a positive integer and i ≤ N. The electronic device stores a basic three-dimensional RGB mapping table. In this embodiment, generating the RGB mapping relationship based on the color information of the first image after second pixel alignment processing and the color information of the second image after second pixel alignment processing can specifically include: determining the color mapping relationship between each pixel of the first image and the second image based on the color information of the first image after second pixel alignment processing and the color information of the second image after second pixel alignment processing, provided that i does not exceed S (i.e., i ≤ S). Here, S is a positive integer and S < N. Then, the previous three-dimensional RGB mapping table is updated based on the color mapping relationship between each pixel to obtain the updated three-dimensional RGB mapping table for the i-th frame. The RGB mapping relationship includes the updated three-dimensional RGB mapping table for the i-th frame; where, when i = 1, the previous three-dimensional RGB mapping table is the basic three-dimensional RGB mapping table. In other words, the basic 3D GRB mapping table is updated using the first S frames captured by the switched camera. This allows for the acquisition of a new 3D RGB mapping table even after the phone's 3A convergence has been completed. Using this updated 3D RGB mapping table for color compensation of the second image yields better and more effective compensation results.

[0021] In one possible implementation of the first aspect, the second image is the i-th frame of N frames captured by the second camera after switching cameras, where N is a positive integer and i ≤ N. The electronic device stores a basic three-dimensional RGB mapping table. In this embodiment, the above-mentioned generation of RGB mapping relationship based on the color information of the first image after second pixel alignment processing and the color information of the second image after second pixel alignment processing can specifically include: determining the color mapping relationship between each pixel of the first image and the second image based on the color information of the first image after second pixel alignment processing and the color information of the second image after second pixel alignment processing, provided that i does not exceed S. Where S is a positive integer and S < N. Then, the basic three-dimensional RGB mapping table is updated based on the color mapping relationship between each pixel to obtain the updated three-dimensional RGB mapping table for the i-th frame. The RGB mapping relationship includes the updated three-dimensional RGB mapping table for the i-th frame. That is, updating the basic three-dimensional RGB mapping table using the first S frames of images captured by the camera after switching cameras can update the three-dimensional RGB mapping table even after the mobile phone 3A has converged. When using the updated 3D RGB mapping table to perform color compensation on the second image, better compensation results can be obtained.

[0022] In one possible implementation of the first aspect, the process of color compensation of the second image based on the RGB mapping relationship to obtain a compensated second image can specifically include: applying the updated 3D RGB mapping table of the i-th frame to the second image to obtain an intermediate image of the i-th frame. Where i > S, the updated 3D RGB mapping table of the i-th frame is the updated 3D RGB mapping table of the S-th frame. Then, using preset weights, the intermediate image of the i-th frame and the second image are weighted and fused to obtain the compensated second image. The magnitude of the preset weights is inversely correlated with i.

[0023] In this scheme, when performing color compensation on the second image, as i gradually increases, the preset weight for weighted fusion using the color of the first image gradually decreases. This allows the electronic device to smoothly and naturally transition from displaying the image captured by the camera before the switch to displaying the brightness and color of the image captured by the camera after the switch, thus avoiding abrupt color changes in the displayed images before and after the camera switch. Simultaneously, a three-dimensional RGB mapping table is used to look up the colors of the images before and after the camera switch, facilitating convenient and quick color compensation of the second image. Furthermore, using a three-dimensional GRB mapping table provides stronger expressive power and more accurate mapping. Mapping using a three-dimensional GRB mapping table can simultaneously align the brightness and color of the first and second images, adapting to a wider range of scenarios.

[0024] In one possible implementation of the first aspect, the second image is the i-th frame of N frames captured by the second camera after the camera switch, where N is a positive integer and i ≤ N. Based on the color information of the first image and the color information of the second image, color compensation is performed on the second image in the RGB color space to obtain the compensated second image. Specifically, this may include: in the RGB space, weighted fusion of the color information of the first image into the color information of the second image using preset weights to obtain the compensated second image. The magnitude of the preset weights is inversely correlated with i. In this scheme, when performing color compensation on the second image, as i gradually increases, the preset weights used for weighted fusion of the colors from the first image gradually decrease, allowing the electronic device to smoothly and naturally transition from displaying the image captured by the camera before the camera switch to displaying the brightness and color of the image captured by the camera after the camera switch. This avoids abrupt color changes in the displayed images before and after the camera switch.

[0025] In one possible implementation of the first aspect, before acquiring the color information of the first image and the color information of the second image, the method further includes: determining whether the first image and / or the second image meet preset conditions. The preset conditions include: the exposure of the first image and / or the second image exceeds a preset exposure range, or the brightness difference between the first image and the second image is greater than a preset brightness difference. Furthermore, if the first image and / or the second image does not meet the preset conditions, the steps of acquiring the color information of the first image and the second image, and of performing color compensation on the second image in the RGB color space based on the color information of the first image and the second image, to obtain a compensated second image.

[0026] In one possible implementation of the first aspect, when the first image and / or the second image meet the aforementioned preset conditions, color compensation for the second image can be waived using the first image, and the second image can be displayed directly. This avoids the problem of poor display quality caused by color compensation for the second image when there are abnormalities in the image captured by the camera.

[0027] In one possible implementation of the first aspect, the electronic device can acquire the Y components of the first image and the second image in the YUV space, respectively, and determine the brightness of the first image and the brightness of the second image. This allows for the calculation of the brightness difference between the first image and the second image. Thus, the brightness difference between the first image and the second image can be determined conveniently and quickly.

[0028] In one possible implementation of the first aspect, the electronic device includes an image processing HAL, which, in response to switching from a first camera to a second camera, acquires a first image and a second image; acquires color information of the first and second images, and performs color compensation on the second image based on the color information of the first and second images, which can be implemented by the image processing HAL. The image processing HAL sends the acquired compensated second image to a camera application, which then displays the compensated second image.

[0029] In one possible implementation of the first aspect, the image processing HAL responds to the switching from the first camera to the second camera, and both the acquired first and second images are images processed by the image signal processor ISP (including IFE and IPE). This facilitates color compensation processing of the images.

[0030] Secondly, this application also provides an electronic device. The electronic device may include a display, a processor, and a memory. The memory stores computer-executable instructions, and when the electronic device is running, the processor executes the computer-executable instructions stored in the memory to cause the electronic device to perform the image processing method as described in any of the first aspects above.

[0031] Thirdly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to perform the image processing method of any of the first aspects described above.

[0032] Fourthly, a computer program product containing instructions is provided, which, when run on an electronic device, enables the electronic device to perform the image processing method of any of the first aspects described above.

[0033] Fifthly, an apparatus (e.g., a system-on-a-chip) is provided, comprising a processor for supporting an electronic device in performing the functions described in the first aspect above. In one possible design, the apparatus further comprises a memory for storing program instructions and data necessary for the electronic device. When the apparatus is a system-on-a-chip, it may be composed of chips or may include chips and other discrete devices.

[0034] The technical effects of any of the design methods in aspects two through five can be found in the technical effects of different design methods in aspect one, and will not be repeated here. Attached Figure Description

[0035] Figure 1 A schematic diagram of a mobile phone interface provided for an embodiment of this application;

[0036] Figure 2 A flowchart illustrating the solutions provided by the related technologies in the embodiments of this application;

[0037] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0038] Figure 4 A schematic diagram of the hardware and software structure of an electronic device provided in an embodiment of this application;

[0039] Figure 5 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0040] Figure 6 This application provides a schematic diagram illustrating the effect of pixel alignment processing on a first image and a second image, as shown in the embodiments of this application.

[0041] Figure 7 A schematic diagram illustrating the updating of a 3D LUT table provided in an embodiment of this application;

[0042] Figure 8 This application provides a schematic diagram illustrating a process for finding output results using an updated 3D LUT table, as provided in an embodiment of the present application.

[0043] Figure 9 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0044] Figure 10 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0045] Figure 11 This is a schematic diagram showing the images displayed on a mobile phone before and after switching cameras, as provided in an embodiment of this application.

[0046] Figure 12 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0047] In practical applications, the hardware configurations of multiple cameras in an electronic device may differ. For example, the shooting angle, the amount of light entering the camera through the aperture, and the response of the image sensor to color and brightness may all vary between different cameras. Due to these differences, when the electronic device switches from the first camera to the second camera for shooting, even if the framing range is similar, the displayed image may still exhibit abrupt changes in brightness and color when switching from the image captured by the first camera to the image captured by the second camera.

[0048] Taking a mobile phone as an example, common mobile phone cameras include wide-angle cameras, main cameras, and telephoto cameras. To achieve better shooting results, the phone can pre-set corresponding magnification ranges for different cameras. In some embodiments, the magnification range for a wide-angle camera includes [0.6, 1); the magnification range for a main camera includes [1, 4); and the magnification range for a telephoto camera includes greater than 4x. The phone displays the image captured by the camera whose current magnification falls within that range. Therefore, when the phone switches its current magnification, it may trigger the selected camera, switching the displayed image from one camera to another. During this camera switching, the image displayed on the phone may experience the brightness and color changes described above.

[0049] It should be noted that the mobile phone cameras and the corresponding magnification ranges of each camera mentioned in the above embodiments are just examples. In other embodiments, the mobile phone may include other cameras, and the corresponding magnification ranges of each camera may also be other ranges.

[0050] Figure 1 The image shows the camera interface 101 of a mobile phone's camera application. This camera interface 101 includes a magnification control 102, through which the user can select the shooting magnification. Figure 1 In the camera interface 101 shown, the phone's current zoom level is 1 (corresponding to the "1X" icon 103 shown in the figure). In one example, the user can slide the zoom control 102 left or right to switch the phone's current zoom level. Figure 1 As shown, the phone can switch the current shooting magnification in response to the user's leftward swipe operation on the magnification control 102.

[0051] Simultaneously, the phone's camera interface updates accordingly. For example, the phone updates to display camera interface 104. The magnification control in camera interface 104 is displayed as magnification control 105. This magnification control 105 is used to indicate the magnification selected by the user during the swipe; as shown in camera interface 104, the user's currently selected shooting magnification is 2.0 (also referred to as 2.0X). Furthermore, the subject in camera interface 104 is magnified compared to the subject in camera interface 101. Taking the shooting magnification ranges corresponding to the various cameras on the phone as an example, the images displayed in both camera interface 101 and camera interface 104 are images captured by the main camera.

[0052] Based on the above explanation, a 4X zoom level falls within the zoom range corresponding to telephoto cameras. When the phone's current zoom level is switched to 4X (or higher), the phone will switch from the main camera to the telephoto camera. Specifically, the image displayed on the phone will switch from the image captured by the main camera to the image captured by the telephoto camera. Figure 1 The mobile phone displays the camera interface 106. The magnification control on this camera interface 106 shows that the phone's current shooting magnification is 4X. Figure 1 As shown in icon 107, the subject in this camera interface 106 is further magnified compared to the subject in the camera interface 104.

[0053] Due to differences in hardware configurations between different cameras, the brightness and color of images captured by the telephoto camera differ from those captured by the main camera. Consequently, when the phone switches between displaying the image from the main camera and the image from the telephoto camera, the brightness and color of the displayed image change abruptly. This abrupt change in brightness and color can negatively impact the user's visual experience when taking photos, thus affecting the overall user experience.

[0054] Understandable. Figure 1 This only illustrates one implementation process for switching the current shooting magnification of a mobile phone within the camera app's photo-taking interface. Similar issues exist when switching the current shooting magnification in other interfaces such as video recording within the camera app, or when the phone switches the current shooting magnification through other means; these will not be elaborated upon here.

[0055] like Figure 2 As shown, related technologies address the aforementioned problems using the following method: First, calculate the mean and variance of two histograms; second, align the target image histogram to obtain a reference image; third, obtain the corrected image and perform the above operations sequentially on the Y, U, and V channels. In this method, the brightness information of the Y channel of the target image is adjusted after alignment. This method uses histograms for alignment. However, histogram alignment requires assuming that the histogram distribution of each channel of the image is approximately normally distributed, and aligning the histograms of different images is done by calculating the mean and variance of the normally distributed distribution. However, this method can cause problems such as image blurring and color cast, and has certain limitations.

[0056] Based on this, this application proposes an image processing method that can be applied to electronic devices (such as mobile phones) including at least two cameras (such as a first camera and a second camera). It should be noted that in this application embodiment, the first camera and the second camera have the same shooting direction, i.e., both are rear cameras or both are front cameras. Specifically, this method can be used to solve the problem of brightness and color jumps in the image displayed by the electronic device before and after camera switching.

[0057] In some embodiments, the image processing method of this application involves an electronic device, in response to switching from a first camera to a second camera, acquiring an image captured by the first camera (denoted as the first image) and an image captured by the second camera (denoted as the second image). The second image is the image to be displayed, captured by the second camera after the camera switch. Then, the electronic device acquires the color information of the first and second images respectively. Based on the color information of the first and second images, the electronic device performs color compensation on the second image in the red-green-blue (RGB) space to obtain a compensated second image. Finally, the electronic device displays the compensated second image at the target display time. In this scheme, the first image is used to perform color compensation on the image captured by the second camera after the camera switch, thereby replacing the original second image with the compensated second image at the target display time. This ensures that the color change of the displayed image does not jump when the electronic device switches cameras, but remains consistent with the color of the image captured by the first camera, maintaining or closely resembling the brightness and color of the image.

[0058] In some embodiments, the second image is specifically any one of the first N frames captured by the second camera after switching cameras. Thus, after the electronic device switches cameras, color compensation is applied to the first N frames captured by the second camera before displaying them, ensuring that the color changes of the images displayed before and after the camera switch do not abruptly change, but rather smoothly and naturally transition from the colors of the images captured by the first camera to the colors of the images captured by the second camera.

[0059] The aforementioned electronic device can be any electronic device with two or more cameras. For example, the electronic device can be a mobile phone, tablet computer, personal computer (PC), smart screen, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, wearable devices such as smartwatches, artificial intelligence (AI) speakers, and in-vehicle devices. It can also be various teaching aids (such as learning machines and early education machines), smart toys, portable robots, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, media players, etc. Furthermore, it can be a device with mobile office functions, a device with smart home functions, a device with audio-visual entertainment functions, or a device supporting smart travel. This application does not impose any special limitations on the specific form of the device.

[0060] Figure 3 The diagram illustrates the structure of an electronic device 100 provided in some embodiments of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, buttons 190, a motor 191, a camera 192, a display screen 193, and a subscriber identification module (SIM) card interface 194, etc. The sensor module 180 may include a pressure sensor 180A, a touch sensor 180B, etc.

[0061] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0062] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. For example, processor 110 is used to execute the video processing method described in this embodiment. Specifically, the ISP may include an image signal processor frontend (IFE) and an image signal processor post-end (IPE).

[0063] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0064] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc. USB interface 130 can be used to connect a charger to charge electronic device 100, and it can also be used for data transfer between electronic device 100 and peripheral devices.

[0066] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0067] Internal memory 121 can be used to store executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function (such as sound playback, image playback, etc.).

[0068] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0069] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130.

[0070] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 193, camera 192, and wireless communication module 160, etc.

[0071] In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be located in the same device.

[0072] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0073] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0074] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0075] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as Wi-Fi), Bluetooth, Global Navigation Satellite System (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0076] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0077] Electronic device 100 can implement audio functions through audio module 170 and application processor, such as music playback and recording.

[0078] The audio module 170 is used to convert digital audio signals into analog audio signals for output, and also to convert analog audio inputs into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0079] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A may be disposed on display screen 193. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 193, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A.

[0080] Touch sensor 180B, also known as a "touch panel," can be located on display screen 193. The touch sensor 180B and display screen 193 together form a touchscreen, also known as a "touch screen." Touch sensor 180B is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 193. In other embodiments, touch sensor 180B may also be located on the surface of electronic device 100, in a different position than display screen 193.

[0081] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0082] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0083] Camera 192 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 192, where N is a positive integer greater than 1.

[0084] Electronic device 100 implements display functions through a GPU, a display screen 193, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0085] The display screen 193 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 193, where N is a positive integer greater than 1.

[0086] The SIM card interface 194 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 194 to achieve contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1.

[0087] The image processing methods described in the following embodiments can all be executed in the electronic device 100 having the above-described hardware structure.

[0088] The following is a brief explanation of the technical terms that may be involved in the embodiments of this application.

[0089] RGB is a color model that uses variations in the red, green, and blue color channels and their superposition to create a variety of colors.

[0090] YUV is a color coding mode where Y represents luminance, or grayscale value, and UV represents chrominance and chroma, respectively. It describes the color and saturation of an image and is used to specify the color of a pixel.

[0091] A 3D lookup table (LUT) is an algorithm that readjusts the hue of an image by creating a three-dimensional color mapping table. The input to a 3D LUT includes RGB values, and the output is the RGB values ​​in the 3D LUT that correspond to the input RGB values.

[0092] In mobile phones, "3A" usually refers to the camera's algorithm functions. 3A can include automatic focus (AF), automatic exposure (AE), and automatic white balance (AWB).

[0093] The image processing method proposed in this application can be specifically applied to scenarios where camera switching occurs during the shooting process of an electronic device. This method can reduce the differences in brightness and color between images when the displayed image changes from an image captured by a first camera to an image captured by a second camera, thereby avoiding abrupt changes in brightness and color.

[0094] The image processing method proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0095] Figure 4 A schematic diagram of the phone's hardware and software architecture is shown. Meanwhile, in... Figure 4 The diagram also illustrates the data flow of the image processing methods described above in some embodiments. Combined with... Figure 4 The image processing method proposed in the embodiments of this application will be introduced.

[0096] A mobile phone consists of an application layer, a hardware abstraction layer (HAL), and a hardware layer. The HAL includes at least a camera HAL and an image processing HAL. The camera HAL is used by camera applications to access the camera. The image processing HAL is used to process images captured by the camera.

[0097] Combination Figure 5 Based on Figure 4The flow of the image processing method implemented on the mobile phone shown is explained. The method includes the following steps:

[0098] S301. Camera application launched.

[0099] In some embodiments, camera app launch can specifically be achieved by the phone controlling the camera app to launch in response to the user's action of opening the camera app.

[0100] S302. The camera application calls the first and second cameras through the camera HAL.

[0101] In some embodiments, the camera application can send an activation command to the cameras (including a first camera and a second camera) by calling the interface provided by the camera HAL. Accordingly, upon receiving the activation command, the camera starts up and can then begin image acquisition.

[0102] The first camera and the second camera are two different cameras in the phone. For example, the first camera can be a wide-angle camera, and the second camera can be either a main camera or a telephoto camera. Alternatively, the first camera can be the main camera, and the second camera can be either a wide-angle camera or a telephoto camera. Or, the first camera can be a telephoto camera, and the second camera can be either the main camera or a wide-angle camera.

[0103] In some embodiments, the camera application can simultaneously activate multiple cameras for image acquisition after startup, and these cameras can operate independently. However, the interface displayed by the camera application only shows the image captured by one of the cameras. For example, the camera application can simultaneously use the first and second cameras to capture corresponding images, and the phone will select the image captured by one camera to display on the interface based on the current zoom level. It should be noted that in scenarios where the phone has three cameras, the camera application can simultaneously use all three cameras to capture images and display the image captured by one camera based on the current zoom level.

[0104] In other embodiments, the camera application may also activate only one camera for image acquisition. Furthermore, the camera application can activate the camera to be switched to only after detecting a switch in camera functionality. For example, the camera application may use the first camera for image acquisition. If the camera application detects a switch from the first camera to the second camera, it then activates the second camera. This reduces the phone's power consumption.

[0105] S303. Save image 1 captured by the first camera and image 2 captured by the second camera to the cache.

[0106] In some embodiments, both Image 1 and Image 2 are images processed by IFE and IPE. The first and second cameras transmit the captured and output image signals to IFE and IPE, and the IFE and IPE process the image signals and output Image 1 and Image 2 respectively. Figure 4 Image 1 is generated by processing the image signal acquired by the first camera through IFE and IPE, and image 2 is generated by processing the image signal acquired by the second camera through IFE and IPE.

[0107] As can be seen from the description of the above embodiments, in some embodiments, the mobile phone can simultaneously use only one camera to capture images. In this embodiment, when saving images, only the currently captured image can be saved to the cache. That is, S303 above can also be replaced by saving the image 1 captured by the first camera to the cache, or saving the image captured by the second camera to the cache.

[0108] In other embodiments, the phone's camera application can simultaneously access multiple cameras to capture images, but only display the image captured by one of the cameras. In this embodiment, when saving images, the image captured by each camera can be saved to a cache separately. Then, the image captured by the camera currently used by the phone in the cache is transmitted to the camera application for display. For example, if the phone is currently using the first camera, the image captured by the first camera is transmitted to the camera application for display.

[0109] S304. Image Processing HAL determines whether the phone has switched cameras.

[0110] Taking the process from S301 to S303, where the phone uses the first camera as an example, the event of switching cameras can specifically correspond to the phone switching from the first camera to the second camera.

[0111] A mobile phone can switch cameras under certain conditions. For example, when the phone's current zoom level changes, the phone may need to switch the currently used camera. As explained above, different cameras correspond to different zoom levels. Therefore, when the phone's current zoom level changes from the zoom level of one camera to the zoom level of another camera, the phone can switch cameras.

[0112] In some embodiments, S304 may specifically include: the mobile phone determining whether it has detected a switch from a first shooting magnification to a second shooting magnification. The first shooting magnification and the second shooting magnification belong to the shooting magnification ranges corresponding to two different cameras. Generally speaking, the image displayed by the mobile phone is the image captured by the camera currently being used.

[0113] Switching between cameras on a mobile phone can be done between any two cameras. Therefore, the aforementioned switching of shooting magnification may involve switching from a lower shooting magnification to a higher shooting magnification. In some embodiments, the aforementioned first shooting magnification and second shooting magnification belong to the shooting magnification ranges corresponding to two different cameras, specifically: the first shooting magnification is less than a preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification.

[0114] The preset shooting magnification can be a critical value between the shooting magnification ranges corresponding to the first camera and the second camera, respectively. For example, if the first camera is a main camera and the second camera is a telephoto camera, the preset shooting magnification could be 4X. Alternatively, the preset shooting magnification could also be an intermediate value between the shooting magnification ranges corresponding to the first camera and the second camera, respectively. For instance, if the first camera is a wide-angle camera and the second camera is a telephoto camera, the preset shooting magnification could be 5X.

[0115] In another example, the aforementioned switching of shooting magnification might also involve switching from a larger shooting magnification to a smaller one. In some embodiments, the first and second shooting magnifications belong to the shooting magnification ranges corresponding to two different cameras, specifically: the first shooting magnification is greater than or equal to a preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification. The preset shooting magnification can be a critical value between the shooting magnification ranges corresponding to the first and second cameras. For example, if the first camera is a telephoto camera and the second camera is a main camera, the preset shooting magnification could be 4X. Alternatively, the preset shooting magnification could also be an intermediate value between the shooting magnification ranges corresponding to the first and second cameras. For example, if the first camera is a telephoto camera and the second camera is a wide-angle camera, the preset shooting magnification could be 5X.

[0116] In some mobile phones, the phone can automatically focus based on the distance between the subject and the camera. In other words, the phone can automatically select an appropriate magnification as the current shooting magnification based on the distance between the subject and the camera. Different shooting magnifications correspond to different focal lengths of the camera, and different focal lengths are suitable for shooting objects at different distances. Therefore, different distances between the subject and the camera correspond to different suitable shooting magnifications. In some embodiments, the distance between the subject and the camera can be referred to as the shooting distance.

[0117] In some embodiments, the mobile phone pre-stores a mapping relationship between different shooting distances and shooting magnification. This mapping relationship stores multiple shooting magnifications suitable for each shooting distance. In this embodiment, the mobile phone can select a suitable shooting magnification based on the current shooting distance, and simultaneously determine the shooting magnification range corresponding to the camera to which that shooting magnification belongs. If the shooting magnifications corresponding to the two shooting distances before and after the switch belong to the shooting magnification ranges of two different cameras, it indicates that the mobile phone has performed a camera switching event.

[0118] Since different lenses on a mobile phone correspond to different magnification ranges, and considering the appropriate magnification for different shooting distances, the phone can convert the magnification ranges corresponding to different cameras into shooting distance ranges corresponding to different cameras. That is, in some embodiments, the phone can pre-store the shooting distance ranges corresponding to different cameras. In this embodiment, the phone determines whether a camera switch has occurred by judging whether the two preceding and following shooting distances belong to the shooting distance ranges corresponding to two different cameras. If the two preceding and following shooting distances belong to the shooting distance ranges corresponding to two different cameras, it indicates that a camera switch event has occurred.

[0119] In other embodiments, S304 may specifically include: the mobile phone determining whether it detects that the distance between the first camera and the object being photographed has switched from a first shooting distance to a second shooting distance. Furthermore, the first shooting distance and the second shooting distance correspond to different shooting distance ranges for different cameras.

[0120] Switching the shooting distance can be from a smaller shooting distance to a larger shooting distance. In some embodiments, the first shooting distance and the second shooting distance correspond to different shooting distance ranges of different cameras, specifically: the first shooting distance is less than a preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance.

[0121] The preset shooting distance can be a critical value between the shooting distance ranges corresponding to the first camera and the second camera, respectively. For example, if the first camera is a main camera and the second camera is a telephoto camera, the preset shooting distance could be 4X, corresponding to a suitable shooting distance. Alternatively, the aforementioned preset shooting magnification could also be an intermediate value between the shooting magnification ranges corresponding to the first camera and the second camera, respectively. For instance, if the first camera is a wide-angle camera and the second camera is a telephoto camera, the preset shooting distance could be 5X, corresponding to a suitable shooting distance.

[0122] Switching shooting distances can also involve changing from a larger shooting distance to a smaller shooting distance. Therefore, in some embodiments, the first shooting distance and the second shooting distance correspond to different shooting distance ranges of different cameras, specifically: the first shooting distance is greater than or equal to a preset shooting distance, and the second shooting distance is less than the preset shooting distance. In some embodiments, the preset shooting distance can be a critical value between the shooting distance ranges corresponding to the first camera and the second camera, respectively. Taking a telephoto camera as an example and the second camera as a main camera, the preset shooting distance could be 4X, corresponding to a suitable shooting distance. Alternatively, the aforementioned preset shooting magnification could also be an intermediate value between the shooting magnification ranges corresponding to the first camera and the second camera, respectively. For example, taking a telephoto camera as an example and the second camera as a wide-angle camera, the preset shooting distance could be 5X, corresponding to a suitable shooting distance.

[0123] In the technical solution proposed in this application embodiment, the mobile phone can determine whether a camera switching event has occurred by changing the shooting magnification or shooting distance. This facilitates timely color compensation of the image to be displayed when a camera switching event occurs, avoiding abrupt changes in brightness and color of the displayed image before and after the camera switch.

[0124] In some embodiments, the image processing HAL can obtain the current shooting magnification or shooting distance through the camera application, and determine whether a camera switching event has occurred by detecting changes in the shooting magnification or shooting distance.

[0125] In other embodiments, the camera application can switch cameras when it detects a change in magnification or shooting distance that meets the conditions for switching cameras, and then send camera switching instruction information to the image processing HAL. In this embodiment, the image processing HAL can determine whether a camera switching event has occurred by judging whether it has received the camera switching instruction information sent by the camera application. Specifically, the image processing HAL can determine that a camera switching event has occurred when it receives the camera switching instruction information sent by the camera application.

[0126] In some examples, if the phone does not switch cameras, the image processing HAL does not need to perform color compensation on the image to be displayed. Instead, it can directly transmit the image retrieved from the cache to the camera application for display. Taking the phone's currently used camera as an example, the image to be displayed transmitted by the image processing HAL to the camera application is image 1.

[0127] In other examples, if the phone switches cameras, the image processing HAL needs to perform color compensation on the image to be displayed before transmitting it to the camera application for display. In this embodiment, the image to be displayed transmitted by the image processing HAL to the camera application is the color-compensated image.

[0128] In some embodiments of this application, for the purpose of distinction, image 1, which is needed for color compensation after switching cameras on the mobile phone, is designated as the first image; and image 2, which requires color compensation after switching cameras, is designated as the second image. In the following description of embodiments, when the determination result of S304 is yes, image 1 is replaced by the first image and image 2 is replaced by the second image.

[0129] In some embodiments, the second image is an image to be displayed at a subsequent display moment (such as a target display moment). In some embodiments, the target display moment can be the next display moment after the current display moment; that is, the second image is the next frame to be displayed after the image displayed by the phone at the current display moment. In other embodiments, the target display moment can also be a subsequent display moment; that is, the second image is a frame to be displayed after the image displayed by the phone at the current display moment.

[0130] In the embodiments of this application, the image processing method is used to address the issue of abrupt changes in image brightness and color when the phone switches from displaying an image captured by the first camera to an image captured by the second camera before switching cameras. Therefore, it is necessary to perform color compensation on the image captured by the second camera after switching to the second camera before displaying it. In some embodiments, the aforementioned second image may specifically be the image captured by the second camera after the phone switches from the first camera to the second camera.

[0131] Furthermore, in the method of this application embodiment, the first image captured by the first camera is used to perform color compensation on the second image before displaying the color-compensated second image. This ensures that after the phone switches to the second camera, the displayed image does not exhibit abrupt changes in brightness and color compared to the image displayed before the switch; instead, it smoothly and naturally transitions to the image captured by the second camera. Therefore, the images captured by the second camera within a certain period after the phone switches to the second camera can be color-compensated before being displayed. Specifically, the aforementioned second image can refer to any one of the first N frames captured by the second camera after the phone switches from the first camera to the second camera. In other words, in the image processing method proposed in this application embodiment, the first N frames captured by the camera used by the phone after switching (i.e., the second camera, hereinafter referred to as the camera before switching) are color-compensated before being displayed. Images captured by the camera after the switch, from the Nth frame onwards, will not undergo color compensation; that is, images from the Nth frame onwards will display with the original brightness and color of the image captured by the camera after the switch. This achieves a smooth and natural transition by color-compensating the brightness and color of the images displayed on the phone before and after camera switching, without consuming excessive power.

[0132] The second image has been described in detail in the above embodiments. The selection of the first image will now be explained. To reduce the significant differences in brightness and color between the images displayed on the phone before and after camera switching, the second image to be displayed can be processed (e.g., color compensation) using an image already displayed on the phone, and then the processed image can be displayed. In some embodiments, the first image may specifically refer to the last frame image captured and displayed by the first camera before the phone switches from the first camera to the second camera. This ensures that the second image displayed after camera switching will not have significant differences in brightness and color compared to the first image displayed before camera switching, thus avoiding any abrupt changes.

[0133] As can be seen from the above embodiments, in some embodiments, the camera application can simultaneously activate multiple cameras for image acquisition, but only display the image acquired by one of the cameras. Therefore, when performing color compensation on the image to be displayed after switching cameras, the image captured in real time by the camera used by the phone before switching (i.e., the first camera, which can be referred to as the camera before switching) can also be obtained, and color compensation can be performed on the image to be displayed after switching cameras. In this embodiment, the aforementioned first image can also refer to the image captured in real time by the first camera after the phone switches from the first camera to the second camera. For example, when the second image is the i-th (i≤N) frame image captured by the second camera after the phone switches from the first camera to the second camera, the first image can be the i-th frame image captured by the first camera after the phone switches from the first camera to the second camera. In this way, the brightness and color of the second image displayed after switching cameras will not differ significantly from the first image displayed before switching, and there will be no abrupt changes.

[0134] In some embodiments, after the image processing HAL determines in S304 that a camera switching event has occurred, S305 can be executed.

[0135] S305. Image Processing HAL retrieves the first and second images from the cache.

[0136] In some embodiments of this application, the image processing HAL can be used to perform color compensation on the image captured by the camera after switching (i.e., the second image) based on the image captured before switching (i.e., the first image) when a camera switching event occurs on the mobile phone.

[0137] S306. Image Processing HAL: Obtain the color information of the first image and the color information of the second image respectively.

[0138] In some embodiments, the color information of an image can be specifically represented by the RGB values ​​of each pixel in the image. The color information of the first image includes the RGB values ​​of each pixel in the first image, and the color information of the second image includes the RGB values ​​of each pixel in the second image.

[0139] Since the first and second images are captured by two separate cameras at potentially different times, the pixels of the first and second images cannot be guaranteed to correspond one-to-one. In this case, directly using the first image to compensate for the color of the second image may result in abnormal colors in the compensated image. In some embodiments, pixel alignment processing can be performed on the first and second images before using the first image to compensate for the color of the second image.

[0140] Different cameras, due to their different spatial positions within the phone's settings, capture images of the same subject with certain differences. Typically, the phone's internal and external parameters are calibrated before leaving the factory, and these calibrated parameters are stored in the phone. In some embodiments, pixel alignment processing can be performed on a first image and a second image using the calibrated internal and external parameters. This pixel alignment processing may include a first pixel alignment process. Prior to S306, the method further includes: performing a first pixel alignment process on the first image and the second image based on preset calibrated internal and external parameters to obtain a first pixel aligned first image and a second image.

[0141] In this embodiment, the image to be displayed is a second image. To ensure that the final displayed image (i.e., the compensated second image) maintains pixel alignment with the original second image as much as possible, the aforementioned first pixel alignment processing specifically processes the first image. In some embodiments, the aforementioned first pixel alignment processing of the first image and the second image based on preset calibration intrinsic and extrinsic parameters may specifically include: mapping the first image to the second image based on preset calibration intrinsic and extrinsic parameters to obtain the first image after first pixel alignment processing. It should be noted that the specific implementation process of mapping the first image to the second image based on preset calibration intrinsic and extrinsic parameters to obtain the first image after first pixel alignment processing can be referred to the description in related technologies, and will not be repeated in this embodiment.

[0142] In some embodiments, after obtaining the first image after the first pixel alignment processing, the mobile phone can perform color compensation on the second image based on the color information of the first image after the first pixel alignment processing and the color information of the second image.

[0143] Since the preset calibration intrinsic and extrinsic parameters are pre-stored, the technical solution provided in this application is more efficient than registration using real-time acquired images. Furthermore, registration methods in related technologies require images to have prominent feature points to achieve good pixel alignment. However, the pixel alignment process using preset calibration intrinsic and extrinsic parameters in this application achieves good pixel alignment regardless of the image content or even if the image lacks prominent feature points.

[0144] After performing a first pixel alignment process on the first image and the second image based on preset calibration intrinsic and extrinsic parameters, a one-to-one correspondence between image pixels cannot be guaranteed. Therefore, after performing the first pixel alignment process on the first image and the second image, a second pixel alignment process can be performed to further align the pixels between the first image and the second image. In some embodiments, after obtaining the first image after the first pixel alignment process, the method further includes: performing a second pixel alignment process on the second image and the first image after the first pixel alignment process, respectively, to obtain a first image after the second pixel alignment process and a second image after the second pixel alignment process. The pixel alignment process includes the second pixel alignment process.

[0145] Furthermore, in an embodiment where a second pixel alignment process is performed on the first image and the second image, after obtaining the first image after the first pixel alignment process, the mobile phone can perform color compensation on the second image based on the first image after the second pixel alignment process and the second image after the second pixel alignment process.

[0146] In some embodiments, the second pixel alignment process may include blurring the image. The image blurring can employ any image blurring method. For example, the image blurring may specifically involve applying a Gaussian blur. Thus, after performing the first pixel alignment process on the first and second images based on preset calibrated intrinsic and extrinsic parameters, further performing the second pixel alignment process using blurring can make the image pixels smoother, thereby achieving a better pixel alignment effect.

[0147] In other embodiments, the second pixel alignment process includes downsampling and blurring. Downsampling the image reduces the amount of video memory and computation required. Thus, after performing first pixel alignment on the first and second images based on preset calibrated intrinsic and extrinsic parameters, further second pixel alignment using downsampling and blurring reduces errors caused by inaccurate image alignment, resulting in better pixel alignment. Downsampling also reduces the computational load on the mobile phone. Blurring makes the image pixels smoother.

[0148] Figure 6 This diagram illustrates the effect of pixel alignment processing on the first and second images. Figure 6 Images 50 and 51 shown correspond to the first and second images described above, respectively. Images 52 and 53 correspond to the first image after second pixel alignment processing and the second image after second pixel alignment processing, respectively.

[0149] S307. Image Processing HAL performs color compensation on the second image in the RGB space based on the color information of the first image and the color information of the second image to obtain the compensated second image.

[0150] In some embodiments, S307 may specifically include: generating an RGB mapping relationship based on the color information of the first image and the color information of the second image; and performing color compensation on the second image based on the RGB mapping relationship to obtain a compensated second image. This ensures that color compensation of the second image is accurate.

[0151] In the embodiment of performing pixel alignment processing on the first image and the second image, in the above-described S307, an RGB mapping relationship can be generated between the pixel-aligned first image and the pixel-aligned second image.

[0152] The aforementioned RGB mapping relationship can be used to characterize the correspondence between pixels in the first and second images. In some embodiments, the RGB mapping relationship can specifically be a 3D RGB mapping relationship. For example, a three-dimensional RGB mapping relationship can be implemented using a 3D LUT. This facilitates convenient and quick color compensation of the second image. Furthermore, using a three-dimensional GRB mapping table provides stronger expressive power and more accurate mapping. Using a three-dimensional GRB mapping table allows for simultaneous alignment of brightness and color between the first and second images, adapting to a wider range of scenarios.

[0153] In some embodiments, the mobile phone stores a basic three-dimensional RGB mapping table (such as a basic 3D LUT). Specifically, the basic 3D LUT table may be a predefined initial LUT table without mapping functionality. S307 may specifically include: updating the basic 3D LUT table based on the color information of the first and second images to obtain an updated 3D LUT table. Then, applying the updated 3D LUT table to the second image to obtain a compensated second image.

[0154] After a phone switches cameras, the first few frames captured by the new camera are usually not fully converged. If only one frame captured by the previous camera and its color information are used to update the basic 3D LUT table, and then this updated 3D LUT table is used to perform color compensation on the first N frames captured by the new camera, inaccurate compensation may occur. This results in the compensated second image still having brightness and color abrupt changes compared to the last frame captured by the first camera before the camera switch. Therefore, in some embodiments, the first few frames captured by the new camera (e.g., the first S frames, S < N) and the first image can be used to update the 3D LUT table multiple times. Once the phone's 3A convergence is complete, updating the 3D LUT table can be stopped. The images from frame (S+1) to frame N captured by the new camera can then be used directly with the 3D LUT table obtained from the last update (i.e., the 3D LUT table updated at frame S).

[0155] In some embodiments, generating the RGB mapping relationship based on the color information of the first image and the second image may specifically include: when i ≤ S, determining the color mapping relationship between each pixel of the first image and the second image based on the color information of the first image and the second image. Then, updating the previous three-dimensional RGB mapping table based on the color mapping relationship between each pixel to obtain the updated three-dimensional RGB mapping table for the i-th frame. Where i = 1, the previous three-dimensional RGB mapping table is the basic three-dimensional RGB mapping table. In this scheme, updating the basic three-dimensional RGB mapping table using the S frames of images captured by the camera after switching can be achieved even after the phone's 3A has converged. Using this updated three-dimensional RGB mapping table for color compensation of the second image can yield better compensation results.

[0156] Figure 7 Schematic diagrams illustrating the updating of 3D LUT tables in some embodiments are shown. Figure 7 In this process, the basic 3D LUT table (RGB) is updated to obtain the updated 3D LUT table (R1G1B1).

[0157] A 3D RGB mapping table can be the 3D LUT table described above. The output of the 3D LUT table can be represented as: Output RGB value = LUT(R input, G input, B input). That is, based on the input color (R value, G value, B value), a unique corresponding color (R value, G value, B value) can be found in the updated 3D LUT table.

[0158] Furthermore, taking an 8-bit bitmap as an example, there are 256 possible RGB values. Accordingly, setting the size of the 3D LUT table to 257*257*257 allows finding the corresponding RGB value for each color. This results in a large storage space occupied by the 3D LUT table. Therefore, to reduce the storage space occupied by the 3D LUT table, the data volume can be reduced by lowering the sampling rate. For example, the size of the 3D LUT table can be set to 33*33*33. In other embodiments, the size of the 3D LUT table can be set to 17*17*17.

[0159] Please refer to Figure 8 This example demonstrates the process of finding the output (color 2) using a 3D LUT table with dimensions of 17*17*17 and an input color 1 (RGB value (0, 191, 255)). Since a 17*17*17 3D LUT table is used to represent 256*256*256 colors, the input RGB values ​​need to be normalized first: RGB / 255 = (0, 0.749016, 1). Then, this is mapped to a 17*17*17 grid: (0, 0.749016, 1)*16 = (0, 11.9843136, 16). The position of this grid cell is then calculated: 1 + 12*17 + 16*17*17 = 4829. Finally, the corresponding RGB value stored at this grid position is found in the 3D LUT table and used as the output RGB value. Figure 8 The color shown is 2 (145, 163, 193).

[0160] exist Figure 8 In the illustrated embodiment, during the calculation of the grid position based on the mapped grid value (0, 11.9843136, 16), non-integer values ​​are directly rounded before calculation to simplify the process. This method is prone to color deviation issues. In other embodiments, for non-integer values ​​in the mapped grid value (0, 11.9843136, 16), the grid position can be calculated directly. Since the mapped grid value is not rounded, the calculated grid position is also a non-integer. Finally, when looking up the RGB value corresponding to the grid position in the 3D LUT table, linear interpolation can be used to find the corresponding RGB value as the output result. This allows for a more accurate RGB value to be found, thus ensuring a better and more accurate compensation effect when performing color compensation on the second image.

[0161] In the technical solution provided in the above embodiments, the storage space occupied can be reduced by storing the 3D LUT table in a reduced sampling manner.

[0162] In some embodiments, applying the updated 3D LUT table to the second image to obtain a compensated second image may specifically include: in the RGB space, weightedly fusing the color information of the first image to the color information of the second image using preset weights to obtain the compensated second image. The preset weights are inversely correlated with i.

[0163] When using preset weights to weightedly fuse the color information of the first image into the color information of the second image, the color information of each pixel in the first image needs to be weighted and fused into the color information of the corresponding pixel in the second image according to the one-to-one correspondence between the pixels of the first and second images. This allows for simultaneous alignment of the brightness and color of the first and second images, adapting to a wider range of scenarios.

[0164] The smaller the value of 'i', the more prominent the corresponding second image is in the image to be displayed after switching cameras. To achieve a smooth and natural transition, during color compensation, the first image used for weighted fusion should have a higher color proportion for the second image that appears more prominently after switching cameras. Conversely, the first image used for weighted fusion should have a lower color proportion for the second image that appears more prominently after switching cameras. This ensures that the colors of the first frame displayed after switching cameras are closest to the colors of the first image, while the colors of the images after switching cameras gradually change towards the original colors of the second image. This continues until the (N+1)th frame after switching cameras, at which point the phone begins to display the original colors of the image captured by the new camera. A larger preset weight results in a higher color proportion of the first image during weighted fusion. Therefore, the smaller 'i' is, the larger the preset weight is; conversely, the larger 'i' is, the smaller the preset weight can be, allowing the preset weight to gradually decrease.

[0165] For example, when the mobile phone performs weighted fusion on the second image in the i-th frame, the preset weight can be expressed as: 1-i / N. In other embodiments, the preset weight can also be expressed by other expressions.

[0166] In an embodiment that uses a 3D LUT table to perform color compensation on the second image, the updated 3D LUT table can be applied to the second image to obtain an intermediate image. Then, the intermediate image and the second image are weighted and fused, with the weights used for fusion being the aforementioned preset weights.

[0167] In some embodiments, the color information of the first image is weighted and fused into the color information of the second image using a preset weight in the RGB space to obtain a compensated second image. This can be achieved specifically through alpha blending.

[0168] In the technical solution provided in this application embodiment, weighted fusion can be performed when color compensation is applied to the second image. Furthermore, the preset weight is determined based on the order in which the second image is captured by the new camera after switching cameras, thereby achieving a smooth and natural transition in brightness and color before and after camera switching, avoiding abrupt changes. This enhances the user's visual experience during the shooting process.

[0169] S308. Image Processing HAL transmits the compensated second image to the camera application.

[0170] Correspondingly, the camera application receives the compensated second image sent by the image processing HAL.

[0171] S309. The camera application displays the compensated second image.

[0172] As described in the above embodiments, the second image is the image to be displayed at the target display time. Therefore, in some embodiments, the camera application displays the compensated second image at the target display time.

[0173] In the technical solution provided in this application embodiment, the first image is used to perform color compensation on the first N frames of images captured by the second camera after switching cameras, and then the images are displayed. This ensures that the color changes of the images displayed before and after switching cameras on the mobile phone are smooth and natural, without any abrupt shifts.

[0174] In addition, please continue to refer to Figure 5 If the judgment result of S304 above is negative, the mobile phone can execute S310-S312.

[0175] S310. Image Processing HAL retrieves the first image from the cache.

[0176] S311. Image Processing HAL transmits the first image to the camera application.

[0177] Correspondingly, the camera application receives the first image sent by the image processing HAL.

[0178] If the phone does not switch cameras, the image processing HAL can directly transmit the first image captured by the first camera to the camera application for display.

[0179] S312. The camera application displays the first image.

[0180] In other embodiments, the phone's image processing HAL can first retrieve the first and second images from the cache, and then determine whether a camera switching event has occurred. In this embodiment, if a camera switching event is determined, color compensation is performed on the image captured by the switched camera (i.e., the second image) before it is transmitted to the camera application for display. If it is determined that no camera switching event has occurred, color compensation is not performed on the image; instead, it is directly transmitted to the camera application for display.

[0181] In the technical solution proposed in the embodiments of this application, when there is no event of switching cameras on the mobile phone, the image processing HAL may not process the image captured by the camera. Instead, it transmits the image captured by the currently used camera to the camera application for display.

[0182] Furthermore, in some abnormal situations, performing color compensation on the image after switching cameras can easily affect the image display quality. Therefore, when the phone sends a camera switching event, before performing color compensation on the second image, it's advisable to determine if the first and / or second images are abnormal to ascertain whether color compensation is necessary for the second image. Please refer to [link / reference]. Figure 9 In some embodiments, after S305 and before S306, the method may further include S401:

[0183] S401. Image Processing HAL determines whether the first image and / or the second image meet preset conditions.

[0184] If the determination result of S401 is negative, it indicates that the first image and / or the second image does not meet the preset conditions, and there is no abnormality in the first image and / or the second image. At this time, the image processing HAL can perform color compensation on the second image and then display it. In this embodiment, the above-mentioned S306-S309 can be executed if the determination result of S401 is positive.

[0185] In some embodiments, the aforementioned preset conditions can be set in conjunction with parameter information of the first image and / or the second image. Specifically, the preset conditions can be set in conjunction with parameter information related to image brightness. For example, parameter information related to image brightness may include image exposure, brightness, etc.

[0186] Image exposure reflects the amount of light received by the camera's photosensitive element during the exposure time when the image is captured. Simply put, the higher the image exposure, the brighter the image. If the first image and / or the second image have abnormal exposure, then when using the color information of the first and second images to perform color compensation on the second image in the RGB space, the compensated image may have a significant difference in exposure compared to the original second image. Therefore, in some embodiments, the aforementioned preset condition may specifically include: the exposure of the first image and / or the second image exceeds a preset exposure range. The preset exposure range can be set according to actual conditions.

[0187] Image brightness represents the brightness of an image. In some scenarios, when a mobile phone switches cameras, the brightness difference between the displayed images can be significant. Therefore, using the color information of the first and second images to perform color compensation on the second image in the RGB space may result in a large difference in brightness between the compensated image and the original second image. In other words, the compensated second image may differ significantly from the actual display effect of the image captured by the second camera, easily leading to image distortion and a poor visual experience for the user. Therefore, in some embodiments, the aforementioned preset condition may specifically include: the brightness difference between the first and second images is greater than a preset brightness difference. The preset brightness difference can be set according to actual conditions.

[0188] In some embodiments, the brightness value of an image can be determined based on the Y component of the image in the YUV space. In some embodiments, calculating the brightness difference between a first image and a second image specifically includes: calculating the histogram information of the Y component of the first image in the YUV space, and determining the average value and / or variance of the Y component of the first image based on the histogram information of the Y component of the first image. Calculating the histogram information of the Y component of the second image in the YUV space, and determining the average value and / or variance of the Y component of the second image based on the histogram information of the Y component of the second image. Then, comparing the average value and / or variance of the Y component of the first image with the average value and / or variance of the Y component of the second image respectively. If at least one parameter (the average value and variance of the Y component) exceeds a corresponding threshold, it is determined that the brightness difference between the first image and the second image is greater than a preset brightness difference. In this scheme, using the Y component of the image in the YUV space to determine the image brightness information can conveniently and quickly determine the brightness difference between the first image and the second image.

[0189] In other embodiments, when the determination result of S401 is yes, it indicates that the first image and / or the second image meet preset conditions, meaning that the first image and / or the second image may have abnormal conditions. In this case, even if the mobile phone sends a camera switching event, the image processing HAL may not perform color compensation on the image to be displayed. Instead, it directly transmits the image captured by the second camera, i.e., the second image, to the camera application. Figure 9 S402-S404 are shown.

[0190] S402. Image Processing HAL retrieves the second image from the cache.

[0191] S403. Image Processing HAL transmits the second image to the camera application.

[0192] S404. The camera application displays a second image.

[0193] This avoids the problem of poor display quality caused by color compensation of the second image when there are abnormalities in the image captured by the camera.

[0194] Next, combine Figure 10 The steps performed by the image processing HAL and camera application in the image processing method proposed in this application are described. After the mobile phone switches from the first camera to the second camera, it can perform the following steps.

[0195] S601_1. Obtain image A captured by the first camera.

[0196] In some embodiments, image A may be the last frame image captured and displayed by the first camera before the phone switches cameras.

[0197] In other embodiments, the mobile phone simultaneously captures images through a first camera and a second camera, but only displays the image captured by one of the cameras. Image A can also be an image captured in real time through the first camera after the mobile phone switches cameras. In this embodiment, image A can also be represented as image A_i. Image A_i and image B_i are captured at the same time.

[0198] S601_2. Obtain the image B_i captured by the second camera.

[0199] It should be noted that the phone can execute S601_1 first and then S601_2; or it can execute S601_2 first and then S601_1; or the phone can execute S601_1 and S601_2 simultaneously.

[0200] S602. Determine if i ≤ N.

[0201] Where N is a positive integer; N≥2.

[0202] If the judgment result of S602 is yes, then there is no need to perform color compensation on image B_i, and image B_i can be directly displayed at the corresponding display time; such as S603.

[0203] S603. Display image B_i.

[0204] If the result of S602 is yes, i.e. i≤N, then it means that image B_i is a frame from the first N frames of the switched camera (i.e., the second camera), and color compensation needs to be performed on image B_i. At this time, S604 can be executed.

[0205] S604. Calculate the brightness of image A and image B_i respectively.

[0206] S605. Determine whether image A and / or image B_i are abnormally exposed, or whether the brightness difference between image A and image B_i is greater than a preset brightness difference.

[0207] If the judgment result of S605 is yes, it means that image A and / or image B_i are abnormal. In this case, color compensation can be omitted for image B_i, and image B_i can be displayed directly, i.e., S603.

[0208] If the judgment result of S605 is negative, it means that image A and / or image B_i are not abnormal. At this time, color compensation can be performed on image B_i; for example, S606 can be executed.

[0209] S606. Determine whether i ≤ S.

[0210] Where S is a positive integer, and S < N. In this embodiment, the basic 3D LUT table is updated using the first S frames of images captured by the second camera after the phone switches cameras. Therefore, if the judgment result of S606 is yes, S607-S609 can be executed.

[0211] If the judgment result in S606 is negative, it means that image B_i does not belong to the first S frames captured by the second camera after the camera switch. In this case, it is not necessary to update the 3D LUT table using image B_i. Therefore, the 3D LUT table obtained by updating image B_S can be used directly to perform color compensation on the current frame B_i. Figure 10 As shown, if the judgment result of S606 is negative, the process can jump to S610.

[0212] S607. After registering images A and B based on preset calibration parameters, perform downsampling and then Gaussian blurring.

[0213] S608. Traverse the Gaussian blurred images A and B_i pixel by pixel to determine the color mapping relationship between each pixel in the images.

[0214] S609. Update the previous 3D LUT table based on the color mapping relationship of each pixel to obtain the updated 3D LUT table for the i-th frame.

[0215] When i=1, the previous 3D LUT table is the base 3D LUT table.

[0216] For example, updating the basic 3D LUT table based on the color mapping relationship of each pixel to obtain the updated 3D LUT table for the i-th frame can specifically include: when i=1, updating the basic 3D LUT table based on the color mapping relationship of each pixel between image A and image B_1 to obtain 3D LUT table 1. When i=2, updating 3D LUT table 1 based on the color mapping relationship of each pixel between image A and image B_2 to obtain 3D LUT table 2. And so on, until i=S, updating the 3D LUT table (S-1) to obtain 3D LUT table S. The specific implementation process of updating the previous 3D LUT table based on the color mapping relationship of each pixel can be referred to the description in related technologies.

[0217] In another example, updating the basic 3D LUT table based on the previous mapping relationship of each pixel to obtain the updated 3D LUT table for the i-th frame can also be implemented as follows: When i=1, update the basic 3D LUT table based on the color mapping relationship of each pixel between image A and image B_1 to obtain 3D LUT table 1. When i=2, update the basic 3D LUT table based on the color mapping relationship of each pixel between image A and image B_2 to obtain 3D LUT table 2. And so on, until i=S, when the basic 3D LUT table is updated based on the color mapping relationship of each pixel between image A and image B_S to obtain 3D LUT table S.

[0218] S610. Apply the updated 3D LUT table of the i-th frame to image B_i to obtain the intermediate image B_A.

[0219] It is understandable that when i > S, the 3D LUT table updated in the i-th frame is the same as the 3D LUT table updated in the S-th frame.

[0220] In some examples, when i=1, the above S610 may specifically include: applying 3D LUT table 1 to image B_1 to obtain intermediate image B_A.

[0221] When i=2, the above S610 may specifically include: applying 3D LUT table 2 to image B_2 to obtain intermediate image B_A.

[0222] When i = S, the above S610 may specifically include: applying the 3D LUT table S to the image B_S to obtain the intermediate image B_A.

[0223] When i = S+1, the above S610 may specifically include: applying the 3D LUT table S to the image B_S+1 to obtain the intermediate image B_A.

[0224] When i = N, the above S610 may specifically include: applying the 3D LUT table S to the image B_N to obtain the intermediate image B_A. And so on.

[0225] S611. Perform a weighted fusion of the intermediate image B_A and the image B_i to obtain the compensated image B_i; the weights are preset weights.

[0226] S612. Display the compensated image B_i.

[0227] The specific implementation process of S601-S612 described above can be referred to the description in the above embodiments.

[0228] In the technical solution provided in the embodiments of this application, image A captured by the first camera is used to perform color compensation on the first N frames B_i captured by the second camera after switching cameras, and then the images are displayed. This ensures that the color changes in the images caused by switching cameras on the mobile phone are smooth and natural, without abrupt jumps.

[0229] Figure 11 This diagram illustrates the images displayed by the phone before and after switching cameras. Before switching cameras, the phone is using the first camera, displaying image 0. After switching cameras, the phone switches from using the first camera to the second camera. Under certain conditions, the images displayed by the phone are, in sequence, color-compensated image 1, color-compensated image 2, ..., color-compensated image N, and image N+1, etc.

[0230] In some embodiments, the algorithm described above, which performs color compensation on the image captured after switching cameras before displaying it, can be named the MCC algorithm. This method operates after the IPE (Integrated Photograph) and before the image is sent for display.

[0231] The above embodiments are all described using image processing methods applied to electronic devices such as mobile phones. In other embodiments, the image processing methods can also be applied to cloud devices such as servers. In this embodiment, the cloud device can acquire a first image and a second image from the electronic device such as a mobile phone. After performing color compensation on the second image in the RGB space based on the color information of the first image and the color information of the second image to obtain a compensated second image, the compensated second image can be returned to the electronic device such as a mobile phone for display. The specific implementation process of each step of the above image processing method on the cloud device is similar to the description of each step in the above embodiments, and will not be repeated here.

[0232] Other embodiments of this application provide an electronic device (such as a mobile phone). The electronic device may include a display screen, a memory, and one or more processors. The display screen and memory are coupled to the processors. The display screen is used to display the interface of the electronic device, such as a preview interface for images captured by a camera in a camera application. The memory is also used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the mobile phone in the above method embodiments. The structure of the electronic device can be referred to... Figure 3 The structure of the electronic device 100 shown.

[0233] This application also provides a chip system, such as... Figure 12 As shown, the chip system 1000 includes at least one processor 1001 and at least one interface circuit 1002. The processor 1001 and the interface circuit 1002 are interconnected via lines. For example, the interface circuit 1002 can be used to receive signals from other devices (e.g., a computer's memory). As another example, the interface circuit 1002 can be used to send signals to other devices (e.g., the processor 1001). Exemplarily, the interface circuit 1002 can read instructions stored in memory and send those instructions to the processor 1001. When the instructions are executed by the processor 1001, the computer can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in this application embodiment.

[0234] This application also provides a computer-readable storage medium including computer instructions that, when executed on the aforementioned electronic device (such as a mobile phone), cause the electronic device to perform various functions or steps performed by the mobile phone in the above method embodiments.

[0235] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps performed by the mobile phone in the above method embodiments. The computer can be an electronic device, such as a mobile phone.

[0236] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0237] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0238] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0239] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0240] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0241] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method is applied to an electronic device, which includes a first camera and a second camera, wherein the first camera and the second camera have the same shooting direction; The electronic device stores a basic three-dimensional RGB mapping table, and the method includes: In response to switching from the first camera to the second camera, a first image and a second image are acquired, wherein the first image is the image captured by the first camera, and the second image is the image to be displayed captured by the second camera after the camera switch; Pixel alignment processing is performed on the first image and the second image to obtain a pixel-aligned first image and a pixel-aligned second image; color information of the pixel-aligned first image and color information of the pixel-aligned second image are obtained; Based on the color information of the first image after pixel alignment processing and the color information of the second image after pixel alignment processing, a red-green-blue (RGB) mapping relationship is generated; the RGB mapping relationship is a three-dimensional RGB mapping table; generating the red-green-blue (RGB) mapping relationship based on the color information of the first image after pixel alignment processing and the color information of the second image after pixel alignment processing includes: updating the basic three-dimensional RGB mapping table based on the color information of the first image after pixel alignment processing and the color information of the second image after pixel alignment processing to obtain an updated three-dimensional RGB mapping table; According to the RGB mapping relationship, the second image is color-compensated in the RGB space to obtain a compensated second image; the step of color-compensating the second image in the RGB space according to the RGB mapping relationship to obtain a compensated second image includes: applying the updated three-dimensional RGB mapping table to the second image to obtain an intermediate image; and performing weighted fusion of the intermediate image and the second image to obtain a compensated second image. The compensated second image is displayed.

2. The method according to claim 1, characterized in that, The second image is a frame to be displayed from N frames captured by the second camera after switching cameras, where N is a positive integer.

3. The method according to claim 1 or 2, characterized in that, The first image is the last frame captured and displayed by the first camera before the electronic device switches from the first camera to the second camera; or, Both the first camera and the second camera are in the activated state, and the first camera and the second camera operate independently; the first image is an image captured in real time by the first camera after the electronic device switches from the first camera to the second camera.

4. The method according to claim 1 or 2, characterized in that, Before switching from the first camera to the second camera and acquiring the first image and the second image, the method further includes: In response to the electronic device switching its current shooting magnification from a first shooting magnification to a second shooting magnification, the camera switches from the first camera to the second camera; wherein the first shooting magnification is less than a preset shooting magnification, and the second shooting magnification is greater than or equal to the preset shooting magnification; or, the first shooting magnification is greater than or equal to the preset shooting magnification, and the second shooting magnification is less than the preset shooting magnification; or, In response to the distance between the first camera and the subject switching from a first shooting distance to a second shooting distance, the camera switches from the first camera to the second camera; wherein the first shooting distance is less than a preset shooting distance, and the second shooting distance is greater than or equal to the preset shooting distance; or, the first shooting distance is greater than or equal to the preset shooting distance, and the second shooting distance is less than the preset shooting distance.

5. The method according to claim 1 or 2, characterized in that, The electronic device stores preset internal and external parameters of each second camera; The acquisition of color information of the first image after pixel alignment processing and color information of the second image after pixel alignment processing includes: Based on the preset calibration parameters, the first image and the second image are subjected to first pixel alignment processing to obtain the first image and the second image after first pixel alignment processing. A second pixel alignment process is performed on the first image and the second image after the first pixel alignment process to obtain a first image after the second pixel alignment process and a second image after the second pixel alignment process; the second pixel alignment process includes blurring, or the second pixel alignment process includes downsampling and blurring.

6. The method according to claim 5, characterized in that, After obtaining the first image after second pixel alignment processing and the second image after second pixel alignment processing, the method further includes: Based on the color information of the first image after the second pixel alignment process, and the color information of the second image after the second pixel alignment process, an RGB mapping relationship is generated. Based on the RGB mapping relationship, color compensation is performed on the second image to obtain the compensated second image.

7. The method according to claim 6, characterized in that, The second image is the i-th frame of N frames captured by the second camera after switching cameras, where N is a positive integer and i ≤ N; The step of generating an RGB mapping relationship based on the color information of the first image after the second pixel alignment processing and the color information of the second image after the second pixel alignment processing includes: If i does not exceed S, the color mapping relationship between each pixel of the first image and the second image is determined based on the color information of the first image after the second pixel alignment process and the color information of the second image after the second pixel alignment process; where S is a positive integer and S < N. The previous 3D RGB mapping table is updated based on the color mapping relationship between each pixel to obtain the updated 3D RGB mapping table for the i-th frame; the RGB mapping relationship includes the updated 3D RGB mapping table for the i-th frame; where i=1, the previous 3D RGB mapping table is the basic 3D RGB mapping table.

8. The method according to claim 7, characterized in that, The step of performing color compensation on the second image according to the RGB mapping relationship to obtain the compensated second image includes: The updated 3D RGB mapping table of the i-th frame is applied to the second image to obtain the intermediate image of the i-th frame; when i > S, the updated 3D RGB mapping table of the i-th frame is the updated 3D RGB mapping table of the S-th frame. Using preset weights, the intermediate image of the i-th frame and the second image are weighted and fused to obtain the compensated second image; wherein, the magnitude of the preset weights is inversely correlated with i.

9. The method according to claim 1 or 2, characterized in that, The second image is the i-th frame of N frames captured by the second camera after switching cameras, where N is a positive integer and i ≤ N; The step of performing color compensation on the second image in the RGB color space based on the color information of the first image and the color information of the second image to obtain the compensated second image includes: In the RGB space, the color information of the first image is weighted and fused into the color information of the second image using a preset weight to obtain the compensated second image; wherein, the magnitude of the preset weight is inversely correlated with i.

10. The method according to claim 1 or 2, characterized in that, After acquiring the first image and the second image, the method further includes: Determine whether the first image and / or the second image meet preset conditions; the preset conditions include: the exposure of the first image and / or the second image exceeds a preset exposure range, or the brightness difference between the first image and the second image is greater than a preset brightness difference. The method further includes: when the first image and / or the second image does not meet the preset conditions, performing the steps of obtaining the color information of the first image and the color information of the second image, and performing color compensation on the second image in the RGB space based on the color information of the first image and the color information of the second image to obtain the compensated second image.

11. An electronic device, characterized in that, The electronic device includes a display screen, a processor, and a memory; the display screen and the memory are respectively coupled to the processor. The display screen is used to display the interface of the electronic device; the memory is used to store computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer instructions; when the computer instructions are executed by a processor of an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-10.

Images