Image display method and related equipment

Abstract

This application provides a method and related apparatus for image display, relating to the field of image processing technology. The method includes: receiving a zoom-in / zoom-out control in a shooting interface for adjusting the shooting magnification; responding to the operation, dynamically zooming in or out of the displayed image according to the zoom level; when the zoom level reaches a preset magnification, the first pixel value corresponding to a first frame included in the image gradually decreases from the preset pixel value until the first frame disappears, and the second pixel value corresponding to a second frame included in the image gradually increases until it reaches the preset pixel value, wherein the first frame is a frame from N frames captured by a first camera after reaching the preset magnification, and the second frame is a frame from N frames captured by a second camera after reaching the preset magnification. This method solves the problem of image jumps during camera switching by simultaneously processing dual-channel images from two cameras and performing overlay processing on the processed dual-channel images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image display method and related equipment. Background Art

[0002] In recent years, with the continuous development of terminal devices, different types of terminal devices, especially terminal products with multiple cameras, have been widely used in users' lives. In order to enable users to obtain a better camera experience, many terminal devices are currently equipped with multiple types of cameras, such as dual-camera terminal devices with two cameras, triple-camera terminal devices with three cameras, and even terminal devices with more cameras. Since different cameras have different parameters such as focal length and field of view (FOV), when users switch between different cameras during shooting, the image effects obtained will also be different, and the shooting picture will usually jump, which has a poor visual effect from the user's perspective.

[0003] Therefore, how to make the captured images transition smoothly and eliminate the sense of picture jump when switching between different cameras has become an urgent problem to be solved. Summary of the invention

[0004] An embodiment of the present application provides a method for image display, which solves the problem of screen jumps when the camera is switched by simultaneously processing the dual-channel images corresponding to the two cameras and overlaying the processed dual-channel images.

[0005] In a first aspect, a method for displaying an image is provided, which is applied to a terminal device, wherein the terminal device includes at least a first camera and a second camera, and the method includes:

[0006] Receive the user's adjustment operation on the shooting magnification;

[0007] In response to the adjustment operation, the image displayed on the shooting interface is enlarged or reduced as the shooting magnification changes;

[0008] When the shooting magnification reaches a preset magnification, the first pixel value corresponding to the first picture included in the image currently displayed on the shooting interface gradually decreases from the preset pixel value, and the second pixel value corresponding to the second picture included in the currently displayed image gradually increases, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in the N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

[0009] In one possible implementation, the first camera may be, for example, an ultra-wide-angle camera, a wide-angle lens, or a telephoto camera; the second camera may be, for example, an ultra-wide-angle camera, a wide-angle lens, or a telephoto camera, but the first camera is different from the second camera.

[0010] In a possible implementation, the first camera is the main camera before the main camera is switched, and the second camera is the main camera after the main camera is switched. The switching here specifically refers to camera switching.

[0011] In a possible implementation, when the shooting magnification reaches a preset magnification, the main camera is switched from the first camera to the second camera. The main camera is a camera corresponding to the main road. The camera corresponding to the main road may refer to an image captured by a camera that is transmitted via the main road, and the image corresponding to the camera is processed by the main road image post-processing module.

[0012] In a possible implementation, the user's adjustment operation on the shooting magnification may include: the user inputs an adjustment operation on the magnification method or reduction control in the shooting interface of the terminal device, and the adjustment operation can be used to enlarge the shooting magnification or to reduce the shooting magnification. Exemplarily, the adjustment operation can be an operation of sliding up to enlarge or reduce the control, or can be an operation of sliding down to enlarge or reduce the control.

[0013] In one possible manner, the image displayed on the shooting interface is enlarged or reduced as the shooting magnification changes, which may specifically include: the image displayed on the shooting interface is dynamically enlarged or reduced as the shooting magnification changes. For example, when the shooting magnification increases from 1× to 2.5×, the size of the object included in the image (such as the preview image) displayed on the shooting interface will also increase from 1× to 2.5×, and the user can observe the process of the object included in the image dynamically enlarging in the preview box of the shooting interface.

[0014] In one possible manner, the preset magnification may refer to the magnification when the main camera is switched. It should be noted that different shooting parameters may correspond to different main cameras, and the main camera may also be played by different cameras at different shooting magnifications. For example, when other parameters are default parameters and the shooting magnification increases from 1× to 2.5×, the main camera may be changed from a wide-angle camera to a telephoto camera.

[0015] In one possible implementation, the picture currently displayed on the shooting interface may be an image based on the image taken by the first camera and the image taken by the second camera after image post-processing and overlaying. The first picture may be the picture corresponding to the first image taken by the first camera, and the second picture may be the picture corresponding to the second image taken by the second camera.

[0016] According to the image display method provided by the present implementation, when a preset magnification is reached and the main camera is switched, the main camera before the switching continues to transmit images as an auxiliary camera, and the image corresponding to the auxiliary camera and the image corresponding to the main camera are processed simultaneously via the auxiliary channel and the main channel, respectively, and then the auxiliary channel image and the main channel image are superimposed to obtain an image with a gradual disappearance and gradual appearance effect, so that when different cameras are switched, the images corresponding to the different cameras can be switched smoothly, avoiding the problem of screen jumps caused by hard switching.

[0017] In combination with the first aspect, in some implementations of the first aspect, the terminal device includes an auxiliary channel image post-processing module and a main channel image post-processing module, and the method further includes:

[0018] When the shooting magnification reaches a preset magnification, a first image shot by the first camera and a second image shot by the second camera are obtained, the first image is transmitted to the auxiliary image post-processing module via the auxiliary path for processing, and the second image is transmitted to the main image post-processing module via the main path for processing; wherein,

[0019] The first image is any one of the N frames of images taken by the first camera after reaching the preset magnification, and the second image is any one of the N frames of images taken by the second camera after reaching the preset magnification.

[0020] It should be noted that after the main camera is switched, the first camera can continue to shoot images as an auxiliary camera and transmit the images via the auxiliary channel. The first camera can continue to shoot and transmit N frames of images after the main camera is switched, and the N frames of images can be used for superimposition processing with the images of the second camera.

[0021] In one possible embodiment, the main path and the auxiliary path can be two parallel paths, which can be used to simultaneously process the image in the main path (i.e., the image corresponding to the second camera) and the image in the auxiliary path (i.e., the image corresponding to the first camera).

[0022] In combination with the first aspect, in some implementations of the first aspect, when the shooting magnification reaches the preset magnification, the method further includes:

[0023] Obtain whether the main camera corresponding to the main road is switched;

[0024] When the main camera corresponding to the main channel is switched from the first camera to the second camera, an overlay switch is turned on, and the overlay switch is used to turn on or off the auxiliary channel.

[0025] In one possible implementation, the overlay switch is used to turn on or off the auxiliary path, which may mean: when the overlay switch is turned on, the auxiliary path is connected, which may correspond to a dual-path (i.e., main path and auxiliary path) image processing strategy; when the overlay switch is turned off, the auxiliary path is disconnected, which may correspond to a single-path (i.e., only main path) image processing strategy.

[0026] Through the method provided by the present implementation, by dynamically controlling the opening or closing of the auxiliary path, image post-processing of two-channel or (more-channel) images can be performed when two-channel (or more-channel) images need to be processed simultaneously.

[0027] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0028] Acquire first cropping information corresponding to the first image, where the first cropping information is used to crop the size of the first image to a preset size; and,

[0029] Second cropping information corresponding to the second image is obtained, where the second cropping information is used to crop the size of the second image to a preset size.

[0030] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0031] The second image is cropped according to the second cropping information, and a cropped main road image is acquired.

[0032] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0033] Read whether the overlay switch is in an on state;

[0034] When the dissolve switch is in the on state, the first image is cropped according to the first cropping information, and the cropped auxiliary channel image is acquired.

[0035] In combination with the first aspect, in some implementations of the first aspect, a superimposition process is performed on the auxiliary image and the corresponding main image to obtain a display image after the superimposition process, wherein the superimposition process is used to gradually reduce a pixel value of the first picture corresponding to the auxiliary image and gradually increase a pixel value of the second picture corresponding to the main image in a fused image corresponding to the auxiliary image and the corresponding main image;

[0036] The sent image is displayed.

[0037] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0038] When the first picture disappears, the dissolve switch is turned off.

[0039] In combination with the first aspect, in some implementations of the first aspect, when the first picture disappears, turning off the dissolve switch specifically includes:

[0040] When the number of frames of the auxiliary channel image reaches the N frames and the first picture disappears, turning off the dissolve switch; and / or,

[0041] When the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

[0042] In a second aspect, a device for image display is provided, the device comprising a first camera, a second camera, an auxiliary channel and a main channel, the auxiliary channel comprising a multi-camera control module, a smooth zoom module, an auxiliary channel image post-processing module and an image processing engine, the main channel comprising the multi-camera control module, the smooth zoom module, the main channel image post-processing module and the image processing engine, wherein:

[0043] The multi-camera control module is used to turn on the overlay switch when the main camera is switched from the first camera to the second camera, and the overlay switch is used to turn on or off the auxiliary channel, and the main camera is the camera corresponding to the main channel;

[0044] The smooth zoom module is used to obtain first cropping information corresponding to the first image taken by the first camera and second cropping information corresponding to the second image taken by the second camera when the overlay switch is in the on state;

[0045] The auxiliary road image post-processing module is used to crop the first image according to the first cropping information and obtain the cropped auxiliary road image;

[0046] The main road image post-processing module is used to crop the second image according to the second cropping information and obtain the cropped main road image;

[0047] The image processing engine is used to perform superimposition processing on the auxiliary channel image and the corresponding main channel image to obtain a display image, the display image.

[0048] In combination with the second aspect, in some implementations of the second aspect, the multi-camera control module is further used to obtain the main camera switching from the first camera to the second camera based on the read first image and the second image.

[0049] In combination with the second aspect, in certain implementations of the second aspect, the image processing engine is specifically used to gradually reduce the first pixel value corresponding to the first picture from a preset pixel value, and gradually increase the second pixel value corresponding to the second picture when the overlay switch is in an on state, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in the N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

[0050] In a third aspect, a terminal device is provided, including:

[0051] processor;

[0052] Memory;

[0053] The memory stores a computer program, which includes instructions. When the instructions are executed by the processor, the terminal device executes the method as described in any one of the first aspects above.

[0054] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable program instructions, and when the computer-executable program instructions are executed on a computer, the computer executes the method as described in any one of the above-mentioned first aspects.

[0055] In a fourth aspect, a computer program product is provided, the computer program product comprising a computer program code, and when the computer program code is run on a computer, the computer is caused to execute the method as described in any implementation manner of the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic structural diagram of the back of a mobile phone provided in an embodiment of the present application.

[0057] Figure 2 A schematic diagram of the screen switching process corresponding to different cameras provided in an embodiment of the present application.

[0058] Figure 3 A schematic structural diagram of a terminal device 100 provided in an embodiment of the present application.

[0059] Figure 4 A software structure block diagram of a terminal device 100 provided in an embodiment of the present application.

[0060] FIG. 5A to FIG. 5F A schematic diagram of a GUI that may be involved in the implementation of an image display method provided in an embodiment of the present application.

[0061] Figure 6 A schematic diagram of the interaction of functional modules involved in the implementation of an image display method provided in an embodiment of the present application.

[0062] Figure 7 A schematic flowchart of an image display method provided in an embodiment of the present application.

[0063] Figure 8 A schematic flowchart of another image display method provided in an embodiment of the present application.

[0064] Fig.9A and Fig. 9B A schematic diagram of a situation in which the brightness of an image captured by a main camera and an auxiliary camera changes linearly provided in an embodiment of the present application.

[0065] Fig.10 A schematic flowchart of another image display method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] It should be noted that the terms used in the implementation method part of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated obstacles, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two, "at least one" and "one or more" mean one, two or more than two.

[0067] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, it is defined that the "first" and "second" features may explicitly or implicitly include one or more of the features.

[0068] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing at different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0069] Most current mobile phones have two to three, or even more cameras (or lenses, lens modules, camera modules). For example, Figure 1 As shown, it is a structural schematic diagram of the back of a mobile phone provided in an embodiment of the present application. The mobile phone includes three cameras (such as camera 1, camera 2 and camera 3) and a flash. Exemplarily, the types of the three cameras can be shown in the following Table 1.

[0070] Table 1

[0071] Camera Camera 1 Camera 2 Camera 3 type Ultra-wide-angle camera Wide-angle camera Telephoto camera

[0072] In some possible shooting scenarios, in order to obtain the best shooting effect, when the parameters such as brightness, shooting distance, shooting angle, etc. are the same, and the user uses the mobile phone to shoot videos at different magnifications, different magnifications may correspond to different cameras as the main camera. For example, when zooming in (or out) from the default shooting magnification (1×) to a certain magnification, the main camera can be switched from a wide-angle camera to a telephoto camera or an ultra-wide-angle camera. However, due to the different hardware parameters and shooting parameters of different cameras, it is easy to cause different imaging field of view (FOV) after the main camera is switched. There is a slight jump in FOV at the moment of switching cameras, resulting in a poor user experience. In addition, when the existing camera terminal performs image post-processing, it only processes the single-channel display image and cannot process dual-channel images at the same time, resulting in the final image cannot be smoothly transitioned with dual-channel delay when switching cameras.

[0073] In view of this, an embodiment of the present application provides a method for optimizing the display effect when the camera is switched. The method realizes simultaneous processing of the main image and the auxiliary image through a software architecture with multiple image processing paths, and then combines the FOV picture transition algorithm (overlay algorithm). When the main camera is switched, a smooth transition animation between the two camera images is used to replace the original hard switching process, thereby improving the visual presentation effect of the video picture and enhancing the user's viewing experience.

[0074] In order to better understand the method for optimizing the overlay effect when switching cameras provided in the embodiments of the present application, the terms or definitions that may be involved in the embodiments of the present application are first introduced below.

[0075] 1. Overlay and overlay effects

[0076] Dissolve is the gradual transition from one image to another, allowing the user to feel a continuous connection between the shots.

[0077] The overlay effect can be reflected in the video as the previous frame image gradually disappears (fades out) and the next frame image gradually appears (fades in), resulting in the two frames of the image overlapping for several seconds.

[0078] Combined with the above introduction, it can be understood that if the shooting magnification is enlarged (or the shooting distance, brightness, etc. are adjusted) during the video shooting process, when it is enlarged to a certain magnification (or increased to a certain distance, increased to a certain brightness), the main camera will switch, for example, the main camera switches from a wide-angle camera to a telephoto camera. At this time, if only the image sent for display (which can correspond to the main road image) is still processed, the image in the shooting interface will jump directly from the picture shot by the wide-angle camera to the picture shot by the telephoto camera, resulting in a picture jump that the user can intuitively perceive. Among them, the picture jump includes, for example: the brightness, color, position of the object being photographed, etc. of the picture in the two frames before and after the jump change. However, if the pictures shot by the cameras before and after the main camera is switched are processed simultaneously through multiple image processing channels, and then the pictures shot by different cameras after processing are superimposed, then the final display effect can be realized by gradually changing from the picture shot by the main camera before the switch to the image shot by the main camera after the switch. From the user's perspective, the image will gradually transition to the picture taken by the main camera after switching, without any large jumps.

[0079] 2. Main camera, auxiliary camera

[0080] In an embodiment of the present application, if the data currently being displayed is only the image data corresponding to one camera, then this camera is the main camera. At this time, other cameras may also be in a powered-on state, but the image data corresponding to the other cameras will not be processed or transmitted to the display screen for display. The other cameras can be called auxiliary cameras.

[0081] If the data currently sent for display is image data corresponding to two or more cameras, it is possible to determine which camera is the main camera and which camera (or cameras) are auxiliary cameras according to a preset shooting strategy. Among them, the shooting strategy described in the embodiment of the present application may include: the correspondence between shooting parameters such as shooting magnification, brightness, and shooting distance and the main camera. For example, if the current shooting magnification is 1×, and other shooting parameters such as brightness and shooting distance are default values, then the main camera at this time can be a wide-angle camera, while the ultra-wide-angle camera and telephoto camera can be auxiliary cameras; for another example, if the current shooting magnification is 3×, and other shooting parameters such as brightness and shooting distance are default values, then the main camera at this time can be a telephoto camera, while the ultra-wide-angle camera and wide-angle camera can be auxiliary cameras.

[0082] Alternatively, if there are two or more cameras currently performing a shooting operation and transmitting image data, the camera corresponding to the gradually appearing image in the shooting interface preview frame can be defined as the main camera, and the camera corresponding to the gradually disappearing image can be defined as the auxiliary camera.

[0083] In some embodiments, the image data corresponding to the main camera can be transmitted to the main image post-processor via the main image processing path (hereinafter referred to as the main path); the image data corresponding to the auxiliary camera can be transmitted to the auxiliary image post-processor via the auxiliary image processing path (hereinafter referred to as the auxiliary path).

[0084] In an embodiment of the present application, the image data corresponding to the main camera and the auxiliary camera can be transmitted to different image post-processors respectively via a multi-channel image path, so that different image post-processors can process multiple images at the same time. Specifically, the multi-channel image path may include a parallel (or parallel) main path and auxiliary path, wherein the main path may include, for example: main camera-multi-camera control module-smooth zoom module-main path image post-processing module (or main path image post-processor)-image processing engine-display module-display screen; the auxiliary path may include, for example: auxiliary camera-multi-camera control module-smooth zoom module-auxiliary path image post-processing module (or auxiliary path image post-processor)-image processing engine-display module-display screen.

[0085] It should be noted that the multi-camera control module, smooth zoom module, image processing engine, display module and display screen in the main path and the auxiliary path can be the same module or device. In other words, the main path and the auxiliary path have their own corresponding cameras and image post-processing modules, and the other functional modules or devices involved can be the same.

[0086] For the specific structure of the multi-channel image processing pipeline, please refer to the following Figure 6 The introduction in the embodiments will not be described in detail here.

[0087] 3. Switching point

[0088] In the embodiment of the present application, the switching point may also be described as a switching time point, which may refer to a time point when one main camera is switched to another main camera.

[0089] For example, if Figure 2 As shown, when the user starts shooting (such as corresponding to Figure 2 The shooting magnification is 1× (or 1x), and then the shooting magnification is slowly increased over time. When it increases to a certain magnification (such as 2.5×, or 2.5x), the main camera switches from camera 1 to camera 2. The switching time corresponding to the switching is also the switching point (such as corresponding to Figure 2 The switching time point from camera 1 to camera 2 in ).

[0090] In other words, before the switching point, the main camera that performs the shooting operation is camera 1, and after the switching point, the main camera that performs the shooting operation is camera 2.

[0091] It should be noted that, in the embodiment of the present application, after the main camera is switched, the main camera before switching (such as corresponding to Figure 2 Camera 1 in the figure does not stop transmitting image data immediately. It can continue to capture the preset number of frames of image data as an auxiliary camera and transmit the image data to other modules (such as the multi-camera control module). That is, after the switching point, the switched main camera (such as the one corresponding to Figure 2 Camera 2 in the image) and the main camera before switching (such as the one corresponding to Figure 2 Camera 1) in the example will simultaneously transmit the image data collected by each camera for a period of time. During this period of time, the main camera and the auxiliary camera will collect and transmit images of a preset number of frames, which can be used as input data for main image processing and input data for auxiliary image processing, respectively. Afterwards, the processed images can be overlaid to obtain a gradual overlay effect of the image after switching the camera.

[0092] 4. Field of View

[0093] The field of view FOV can also be described as the field of view. The size of the field of view determines the field of view of the camera. In general, the relationship between the field of view and the focal length is that the larger the field of view, the shorter the focal length. For example, in the embodiment of the present application, if the cameras in the mobile phone are classified by the field of view, as shown in Table 2 below, they can be divided into the following categories:

[0094] Table 2

[0095] Field of view / Less than or equal to 40° Greater than 40° and less than or equal to 60° More than 60° Camera Type Telephoto camera Wide-angle camera Ultra-wide-angle camera

[0096] For example, Figure 3As shown, it is a schematic structural diagram of a terminal device 100 provided in an embodiment of the present application.

[0097] In some embodiments, in some embodiments, the terminal device 100 can be various types of electronic devices with video shooting and image processing functions, such as: smart phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) or dedicated voice call devices, etc. The embodiments of the present application do not impose any restrictions on the specific type of the terminal device 100.

[0098] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0099] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0100] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0101] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0102] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0103] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0104] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0105] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0106] The terminal device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The display screen 194 is used to display images, videos, etc.

[0107] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0108] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy. The video codec is used to compress or decompress digital video. The NPU is a neural network (NN) computing processor that quickly processes input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0109] Exemplarily, the software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100. Figure 4 It is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0110] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime, system library, kernel layer, hardware abstraction layer (HAL), and hardware layer.

[0111] The application layer can include a series of application packages. Figure 4 As shown, the application package may include applications such as camera, calendar, map, WLAN, music, short message, gallery, call, navigation, Bluetooth, video, etc.

[0112] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 4 As shown, the application framework layer may include a window manager, a content provider, a phone manager, a resource manager, a notification manager, a view system, and the like.

[0113] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0114] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0115] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0116] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0117] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0118] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal vibrates, the indicator light flashes, etc.

[0119] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0120] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0121] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as obstacle life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0122] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0123] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0124] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0125] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0126] A 2D graphics engine is a drawing engine for 2D drawings.

[0127] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera sensor driver, audio driver, and sensor driver.

[0128] In order to better understand the image display method provided in the embodiment of the present application, the implementation of the method is introduced at the user visualization level in conjunction with the accompanying drawings.

[0129] For example, FIG. 5A to FIG. 5F FIG. 1 is a schematic diagram of a graphical user interface (GUI) that may be involved in the implementation of an image display method provided in an embodiment of the present application. For ease of description, the present application embodiment is introduced by taking a mobile phone as an example of a terminal device.

[0130] See also Figure 5A , is a schematic diagram of a shooting interface provided in an embodiment of the present application. The shooting interface may include a viewfinder, an album icon, a zoom in / out control, a shooting control, and a rotation control.

[0131] The viewfinder is used to obtain the preview image and display the preview image in real time. Figure 5A The rear view angle shown includes preview images of furniture and plants. The album icon is used to quickly enter the album. When the phone detects that the user clicks the album icon, the photos or videos that have been taken can be displayed on the touch screen. The zoom in / out control is used to enlarge or reduce the shooting magnification when shooting, wherein the image effect after zooming in or out can be presented to the user in the viewfinder. The shooting control is used to shoot or record video. When in the photo mode, the phone detects that the user clicks the shooting control, the phone performs the photo operation and saves the shot photo; or, when the phone is in the video recording mode, the user clicks the shooting control, the phone performs the video recording operation and saves the recorded video. The camera rotation control is used to control the switching of the front camera and the rear camera.

[0132] In addition, the shooting interface also includes functional controls for setting the shooting mode, such as Figure 5AThe aperture shooting mode, night scene shooting mode, portrait shooting mode, photo shooting mode, video recording mode, professional mode and more modes are shown. Among them, more modes can also include slow motion mode, panoramic mode, black and white art mode, dual view video recording mode, filter mode, high dynamic range image (HDR) mode and multi-channel video recording mode (not shown in the figure), etc., which are not limited in this application.

[0133] In some embodiments, when the electronic device detects that the user clicks the icon corresponding to the video recording mode on the camera shooting interface, the electronic device may enter the video recording mode (or video shooting mode). Figure 5B As shown, the user can input a zoom-in operation for the zoom-in / zoom-out control based on the current shooting magnification (such as the default shooting magnification 1×) according to the desired video effect. The zoom-in operation can be, for example, Figure 5B In the zoom in / out vertical bar, slide the zoom in / out slider upward. In response to the zoom in operation input by the user, the mobile phone can dynamically display the image zooming process to the user through the preview box. For example, assuming that the user zooms in the shooting magnification to 2.5 times the default magnification, Figure 5C As shown, the size of the objects in the image (i.e., furniture and plants) can be Figure 5B 2.5 times the subject shown.

[0134] For the sake of comparison, the following first introduces a scenario in which the image display method provided by the embodiment of the present application is not applied, that is, an introduction to a scenario in which when the camera is switched, the video image is also directly switched accordingly.

[0135] In some embodiments, when the shooting magnification is magnified to 2.5 times, the main camera switches from the wide-angle camera to the telephoto camera. Since the hardware parameters and shooting parameters of the telephoto camera and the wide-angle camera are different, the images taken by the two cameras will also show different effects. Figure 5D As shown, after switching to the telephoto camera, if the preview frame directly displays the picture taken by the telephoto camera, then the picture after the camera is switched will have a more obvious jump compared to the picture before the camera is switched, such as jumping directly from a dotted image to a solid image, where the dotted image represents the picture before the switch (that is, the picture taken by the wide-angle camera at 2.5 times), and the solid image represents the picture after the switch (that is, the picture taken by the telephoto camera at 2.5 times). The two frames of images include the same objects (both furniture and plants), but the FOV has undergone a large jump. From the user's perspective, the position of the photographed object has changed significantly (such as the distance between the photographed objects in the two frames of images differs by D). Figure 5C and 5DAs shown, the pixel values ​​(or brightness) of the two frames of images before and after the camera switch may also change significantly. In addition, the effects corresponding to other parameters of the two frames of images before and after the camera switch may also change significantly, which is not limited in the embodiments of the present application.

[0136] The following combination Figure 5E and Fig. 5F , introduces the scenario of applying the image display method provided by the embodiment of the present application. That is, when the camera is switched, the video image is superimposed, the picture corresponding to the main camera after the switch is gradually displayed, and the picture corresponding to the main camera before the switch is gradually hidden.

[0137] For example, Figure 5E As shown in the figure, it is a schematic diagram of the shooting interface when the user has just adjusted the magnification to 2.5 times but has not switched the camera yet. The preview image displayed in the viewfinder in the shooting interface is still the image taken by the wide-angle camera. Afterwards, the camera switches from the wide-angle camera to the telephoto camera. At this time, the wide-angle camera as the auxiliary camera will continue to shoot and transmit images with a preset number of frames (for example, 20 frames). At the same time, the telephoto camera as the main camera after switching will also shoot and transmit images with the same number of frames (for example, 20 frames) during this period. The mobile phone divides the images taken by the two cameras into two channels for simultaneous processing and sends them to the display. The display displays the final image after the two-channel image processing, that is, the superposition, according to the frame rate.

[0138] For example, Fig. 5F As shown, the mobile phone can dynamically display the processed image in the viewfinder. For example, from the user's perspective, after switching the camera, the subject at the corresponding new position can gradually appear, while the subject at the initial position can gradually disappear, where the distance between the two positions can be D.

[0139] According to the image display method provided in the embodiment of the present application, when the camera is switched, the image corresponding to the camera before the switching of a preset number of frames and the image corresponding to the camera after the switching of a preset number of frames are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed, so that when different cameras are switched, the captured pictures can be smoothly transitioned, the picture jump feeling is eliminated, and the user's viewing experience is improved.

[0140] In order to better understand the image display method provided in the embodiment of the present application, the underlying implementation logic of the method is introduced below in combination with the structure of the terminal device.

[0141] For example, Figure 6As shown, it is a schematic diagram of the interaction of functional modules involved in the implementation of an image display method provided in an embodiment of the present application. The functional modules in the embodiment of the present application may include multiple cameras (such as Figure 6 The ultra-wide-angle camera 601, wide-angle camera 602 and telephoto camera 603 shown in the figure), multi-camera control module 604, smooth zoom module 605, main image post-processing module 606, auxiliary image post-processing module 607, image processing engine 608 and display module 609, display screen 610, etc.

[0142] In some embodiments, the ultra-wide-angle camera 601 may correspond to camera 1 in Table 1 above, the wide-angle camera 602 may correspond to camera 2 in Table 1 above, and the telephoto camera 603 may correspond to camera 3 in Table 1 above. In the embodiment of the present application, the ultra-wide-angle camera 601 may also be described as an ultra-wide-angle lens, the wide-angle camera 602 may also be described as a wide-angle lens, and the telephoto camera 603 may also be described as a telephoto lens, which may be used to shoot objects in the environment under different shooting parameters (such as shooting magnification, brightness, shooting distance, etc.) and obtain images including these objects.

[0143] In some embodiments, the multi-camera control module 604 can be used to maintain the number of images transmitted by the main channel and the auxiliary channel to at least meet the needs of the superimposition algorithm, while not changing other logics of switching lenses when executing the method provided in the embodiment of the present application. Specifically, the multi-camera control module 604 can be used to control the on or off of the superimposition switch; it can also be used to determine whether superimposition processing is required based on the superimposition judgment strategy and whether the front and rear display lenses have changed; when superimposition processing is required, the corresponding superimposition frame number (corresponding to the preset frame number in the above text) is obtained according to the lens switching information, and when the dual-channel output does not meet the superimposition frame number, the dual-channel output is continuously maintained and the superimposition switch is set to the on state.

[0144] The multi-camera control module 604 can be used to determine which camera is currently the main camera and which camera or cameras are used as auxiliary cameras according to the multi-camera strategy. The multi-camera strategy may include, for example, determining which main camera corresponds to the current shooting parameters based on the corresponding relationship between the shooting parameters (such as shooting magnification, brightness, shooting distance, etc.) and the main camera.

[0145] After the multi-camera control module 604 determines the main camera according to the multi-camera strategy, it can instruct the main camera to transmit the image data it has acquired, and copy the image data of the main camera to the port corresponding to the main camera, so that the smooth zoom module can read the image data of the main camera from the port corresponding to the main camera.

[0146] When the multi-camera control module 604 determines that the main camera needs to be switched from camera 1 to camera 2 according to the multi-camera strategy, the multi-camera control module 604 can instruct camera 1 (i.e., the main camera before switching) to continue transmitting images of a preset number of frames, and instruct camera 2 (i.e., the main camera after switching) to transmit images of at least a preset number of frames; thereafter, the multi-camera control module 604 can copy the image transmitted from camera 1 to the port corresponding to the auxiliary camera, so that the smooth zoom module can read the image data of the auxiliary camera from the port corresponding to the auxiliary camera, and can copy the image transmitted from camera 2 to the port corresponding to the main camera, so that the smooth zoom module can read the image data of the main camera from the port corresponding to the main camera. The ports corresponding to the main camera and the ports corresponding to the auxiliary camera here can refer to the ports between the multi-camera control module 604 and the smooth zoom module.

[0147] In addition, the multi-camera control module 604 can also be used to determine whether the number of image frames transmitted by the main camera and the auxiliary camera reaches a preset number of frames after the main camera is switched. If the number of image frames transmitted by the main camera and the auxiliary camera reaches the preset number of frames, the multi-camera control module 604 can control the overlay switch to be turned off; if the number of image frames transmitted by the main camera and the auxiliary camera does not reach the preset number of frames, the multi-camera control module 604 can control the overlay switch to be turned on.

[0148] In some embodiments, the smooth zoom module 605 can be used to calculate the cropping information and / or mapping information corresponding to the main image and the auxiliary image of the output lens at the current focal length, so that the main image post-processing module can process the main image and the auxiliary image post-processing module can process the auxiliary image. In addition, the smooth zoom module 605 can also be used to determine which images are auxiliary images according to the image data read from which port, and then copy the auxiliary images to the auxiliary output port, and output the calculated information required for post-processing of the auxiliary images through the auxiliary output port. The smooth zoom module 605 can also be used to determine which images are main images according to the image data read from which port, and then copy the main images to the main output port, and output the calculated information required for post-processing of the main images through the main output port. The main output port and the auxiliary output port here can refer to the ports between the smooth zoom module and the main image post-processor 606 and the auxiliary image post-processor 607, respectively.

[0149] In order to ensure that no additional power consumption is added, the smooth zoom module 605 can also determine whether to copy the auxiliary channel image to the auxiliary channel output port according to the dynamic state of the superimposition switch. When the superimposition switch is on, the smooth zoom module 605 can copy the auxiliary channel image and its corresponding processing information to the auxiliary channel output port, so that the auxiliary channel image post-processing module can read the auxiliary channel image and its corresponding processing information from the auxiliary channel port; at the same time, the main channel image and its corresponding processing information can be copied to the main channel output port, so that the main channel image post-processing module can read the main channel image and its corresponding processing information from the main channel port. When the superimposition switch is off, the smooth zoom module 605 no longer copies the auxiliary channel image and its corresponding processing information to the auxiliary channel output port, but only copies the main channel image and its corresponding processing information to the main channel output port, so that the main channel image post-processing module can read the main channel image and its corresponding processing information from the main channel port.

[0150] In some embodiments, the main image post-processing module 606 can be used to read the main image and its processing information from the main output port between it and the smooth zoom module 605, and then perform image post-processing on the main image according to the main image and its processing information. The image post-processing includes, for example, image cropping, image feature mapping, etc., which is not limited in the embodiment of the present application.

[0151] In some embodiments, the auxiliary channel image post-processing module 607 can be used to read the auxiliary channel image and its processing information from the auxiliary channel output port between it and the smooth zoom module 605, and then perform image post-processing on the auxiliary channel image according to the auxiliary channel image and its processing information. The image post-processing includes, for example, image cropping, image feature mapping, etc., which is not limited in the embodiments of the present application.

[0152] In some embodiments, the image processing engine 608 can be used to perform superimposition processing on the processed images corresponding to the two cameras according to the superimposition algorithm, and can also perform other processing on the images according to other algorithms, and then transmit the processed images with superimposition effect to the display module. In a possible implementation, the image processing engine 608 can be specifically presented as a graphics processing unit (GPU).

[0153] In some embodiments, the display module 609 can also be described as a display module, which can be used to send the image data (multiple pixel point data) rendered by a processor such as a GPU to a display device. The display device here can refer to a display screen or a mobile phone screen, for example.

[0154] In some embodiments, the display screen 610 can be used to display images, and specifically can be used to display images with a superimposed effect. For example, when the camera is switched, the display screen can present the image corresponding to the main camera to the user in a gradually displayed form, and at the same time present the image corresponding to the auxiliary camera to the user in a gradually hidden form.

[0155] The following combination Figure 6 The structure shown, and Figure 7 The flowchart shown specifically introduces an image display method provided in an embodiment of the present application.

[0156] For example, Figure 7 As shown, it is a schematic flow chart of a method for image display provided by an embodiment of the present application. The process in the embodiment of the present application may be a process related to a multi-camera control module, which may be a multi-camera control module in a mobile phone (corresponding to Figure 6 The multi-camera control module 604 in FIG. 6 is used as the main body for execution. The process may specifically include the following steps:

[0157] S701, determining the camera corresponding to the current frame image based on a preset shooting strategy.

[0158] The preset shooting strategy may include: the corresponding relationship between shooting parameters such as shooting magnification, brightness, shooting distance and the main camera.

[0159] In some embodiments, determining the camera corresponding to the current frame image based on a preset shooting strategy may include: the multi-camera control module determines which camera the currently acquired image corresponds to, that is, which camera is the current main camera, based on the correspondence between the shooting parameters such as the image magnification, brightness, and shooting distance and the main camera. This shooting strategy can also be described as a multi-camera strategy.

[0160] Specifically, the multi-camera control module can combine the data of the current frame image and the data of the previous frame image, and obtain the camera corresponding to the current frame image according to the preset shooting strategy. Exemplarily, if the multi-camera control module has obtained the brightness, pixel value and other parameters corresponding to the a-th frame image, then when the multi-camera control module receives the (a+1)-th frame image transmitted by the camera, the first shooting parameters such as the brightness and pixel value of the current (a+1)-th frame image can be determined based on the a-th frame (that is, the previous frame image of the current frame image), and then the corresponding second shooting parameters such as the shooting distance and magnification are determined based on the current frame image; thereafter, the multi-camera control module can determine the camera corresponding to the current frame image based on the first shooting parameters and the second shooting parameters according to the preset shooting strategy.

[0161] It should be noted that, for each frame of image acquired, the multi-camera control module can judge it according to a preset shooting strategy, and then obtain the camera corresponding to each frame of image.

[0162] S702, judging whether to switch the camera according to the camera corresponding to the current frame image.

[0163] In some embodiments, after obtaining the camera corresponding to the current frame image, the multi-camera control module can compare the camera with the camera corresponding to the previous frame image, and determine whether the camera corresponding to the current frame image and the camera corresponding to the previous frame image are the same camera. If the current frame image and the previous frame image correspond to the same camera, it means that the main camera switch has not occurred, and the original multi-camera strategy can continue to be used to process the current frame image; if the cameras corresponding to the current frame image and the previous frame image are different, it means that the main camera switch has occurred, and the dual-channel image processing strategy may need to be used to process the current frame image.

[0164] In some embodiments, the process of a mobile phone determining whether to switch cameras based on an output status may include: a multi-camera control module of the mobile phone may determine whether the main camera has been switched based on which camera transmitted the received image; if the multi-camera control module detects that the camera corresponding to the image it acquires has changed from the first camera to the second camera, the multi-camera control module may determine that the main camera has been switched; or, if the multi-camera control module detects that the camera corresponding to the image it acquires has changed from the first camera to the first camera and the second camera, the multi-camera control module may determine that the main camera has been switched.

[0165] More specifically, when shooting, the camera can convert the optical signal into an electrical signal and transmit it to the multi-camera control module, wherein the electrical signals transmitted by different cameras are different, and the multi-camera control module can determine whether to switch the camera currently based on the characteristics of the electrical signal. For example, when the electrical signal received by the multi-camera control module has not changed, or when the electrical signal received by the multi-camera control module has always been one electrical signal, and the electrical signal is the same as the electrical signal corresponding to a certain camera, the multi-camera control module can determine that the camera is not currently switched; when the electrical signal received by the multi-camera control module changes (such as a change in the number of pulses), the multi-camera control module can determine that the current camera is switched.

[0166] In some embodiments, the multi-camera control module can store electrical signal characteristics (or pulse characteristics) corresponding to different types of cameras. When during the shooting process, after obtaining the electrical signal transmitted by the camera, the multi-camera control module can determine which camera or cameras are currently in the shooting state based on the correspondence between the electrical signal and the camera.

[0167] Among them, when the judgment result of this step is "no", that is, the camera is not switched, step S707 can be executed next, that is, the original main channel image processing strategy is output, and the original main channel image processing strategy can be, for example, an image processing strategy in which only the main channel is connected, that is, the main channel image is processed using the main channel architecture; when the judgment result of this step is "yes", that is, the main camera is switched, step S703 can be executed next.

[0168] The following is a specific description of the situation where the judgment result of this step is "yes":

[0169] S703, reading the images transmitted by the two cameras respectively before and after the switching.

[0170] The images transmitted by the two cameras before and after the switching may refer to the images transmitted by the first camera that serves as the (old) main camera before the main camera is switched and serves as the auxiliary camera after the switching, and the images transmitted by the main camera that is switched to serve as the (new) main camera. For ease of description, the embodiment of the present application may define the main camera before the switching as the first camera, and the main camera after the switching as the second camera.

[0171] In some embodiments, the multi-camera control module of the terminal device can read the images transmitted by the two cameras of the preset frame number, that is, keep reading the dual-channel images of the preset frame number. Among them, the preset frame number here can correspond to the superimposed frame number in the embodiment of the present application. The preset frame number can be pre-set as needed, for example, it can be set to N frames, N is an integer greater than 1 (such as 15 frames or 20 frames), etc., and the embodiment of the present application is not limited to this.

[0172] In some embodiments, based on the camera corresponding to the current frame image (such as the second camera) and the camera corresponding to the previous frame image (such as the first camera), the multi-camera control module can obtain the main camera before switching as the first camera, and obtain the main camera after switching as the second camera; thereafter, optionally, the multi-camera control module can instruct the first camera to continue shooting and transmitting a preset number of frames of images (i.e., auxiliary images), and can instruct the second camera to at least shoot and transmit a preset number of frames of images (i.e., main images).

[0173] S704, determining whether the current image has reached a preset frame number.

[0174] Among them, the current image may refer to the current frame image obtained by the multi-camera control module. It can be understood that when the camera is switched, the multi-camera control module will receive the images sent by the two cameras (i.e., the main camera and the auxiliary camera) before and after the switch. At this time, the current frame image may include the current frame image corresponding to the main camera and the current frame image corresponding to the auxiliary camera.

[0175] In some embodiments, when the multi-camera control module obtains camera switching, it can instruct the overlay switch to be turned on. After the overlay switch is turned on, the path for auxiliary image processing will be connected. At this time, the image corresponding to the main camera and the image corresponding to the auxiliary camera can be processed simultaneously through the main path and the auxiliary path (that is, dual paths). Afterwards, the multi-camera control module controls the overlay switch to be turned on or off according to whether the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently obtained have reached the preset frame number, wherein: if the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently obtained have both reached the preset frame number, the multi-camera control module can instruct the overlay switch to be turned off, and subsequently, the multi-camera control module can only receive the images sent by the main camera, and transmit and process the images corresponding to the main camera via the main channel, that is, restore to the original main channel image processing strategy; if the number of auxiliary channel images and / or the number of main channel images currently obtained does not reach the preset frame number, the multi-camera control module can instruct the overlay switch to be turned on, and subsequently, the multi-camera control module will continue to accept the images sent by the two cameras before and after the switching (i.e., the main camera and the auxiliary camera), respectively, and simultaneously transmit and process the images corresponding to the main camera and the images corresponding to the auxiliary camera via the main channel and the auxiliary channel, respectively, that is, continue to adopt the main channel and auxiliary channel image processing strategies.

[0176] If the judgment result of this step is "No", that is, the number of images corresponding to the auxiliary camera and / or the number of images corresponding to the main camera currently acquired by the multi-camera control module does not reach the preset frame number, then step S705 can be executed next. If the judgment result of this step is "Yes", that is, the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently acquired by the multi-camera control module reach the preset frame number, then step S706 can be executed next.

[0177] First, the situation where the judgment result of this step is "No" is introduced:

[0178] S705, output the newly added main channel and auxiliary channel image processing strategies, and set the overlay switch to the on state.

[0179] The newly added main and auxiliary image processing strategies here may refer to: in addition to transmitting and processing the image corresponding to the main camera via the main channel, the image corresponding to the auxiliary camera is also transmitted and processed via the auxiliary channel, that is, the image corresponding to the main camera and the image corresponding to the auxiliary camera are transmitted and processed via the main channel and the auxiliary channel respectively at the same time. The main and auxiliary image processing strategies can also be described as dual-channel image processing strategies.

[0180] It should be noted that if the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently obtained by the multi-camera control module do not reach the preset frame number (that is, the judgment result in step S704 is "no"), it means that the image processing engine has not obtained a sufficient number of main images and auxiliary images to perform a complete superposition process. At this time, the image processing engine may have superimposed some of the main images and the corresponding auxiliary images, but due to the insufficient number, there may still be a screen jump phenomenon when the image is displayed. Therefore, when the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently obtained by the multi-camera control module does not reach the preset frame number, in order to avoid the above-mentioned problem, the multi-camera control module needs to indicate that the current frame image is still transmitted and processed according to the main and auxiliary image processing strategies, and the superposition switch is set to the on state. Based on the main and auxiliary image processing strategies indicated by the multi-camera control module, the current frame image (including the current frame image transmitted by the main camera and the current frame image transmitted by the auxiliary camera at this time) can be subsequently divided into transmission and processing via the main path and transmission and processing via the auxiliary path, and superposition processing is performed on the basis of image post-processing.

[0181] Next, the situation where the judgment result of this step is "yes" is introduced:

[0182] S706: Output the original main channel image processing strategy to perform image processing.

[0183] Among them, the original main channel image processing strategy here is to transmit and process the main channel image through the main channel image processing channel without transmitting and processing the auxiliary channel image through the auxiliary channel, and there is no need to perform superimposition processing on the post-processed main channel image.

[0184] It is understandable that if the number of images corresponding to the auxiliary camera and the number of images corresponding to the main camera currently obtained by the multi-camera control module reach the preset frame number, it means that the image processing engine can also obtain enough images corresponding to the main camera and the auxiliary camera for superimposition processing. At this time, the mobile phone does not need to transmit and process more images corresponding to the auxiliary camera, but can perform image processing according to the original main image processing strategy. The above method can avoid consuming additional resources and avoid the superimposition processing in the embodiment of the present application causing changes to the original image processing strategy.

[0185] In order to better understand the image display method provided in the embodiment of the present application, the dual-path image processing strategy provided in the embodiment of the present application is introduced in detail below.

[0186] Exemplarily, in the dual-channel image processing strategy of the main channel and the auxiliary channel, when the multi-camera control module combines the data of the current frame image and the data of the previous frame image, and obtains that the camera corresponding to the current frame image is not the same camera as the camera corresponding to the previous frame image according to the preset shooting strategy, it can be determined that the main camera has switched. For ease of understanding, this application defines the main camera before switching (that is, the auxiliary camera after switching), that is, the camera corresponding to the previous frame image, as the first camera; and defines the main camera after switching, that is, the newly added camera corresponding to the current frame image, as the second camera. Afterwards, the multi-camera control module can store the auxiliary camera as the first camera and the main camera as the second camera in the memory after the main camera is switched. At the same time, the multi-camera control module can also turn on the overlay switch based on the switching of the main camera, and copy the image corresponding to the first camera (hereinafter referred to as the first image) to the port corresponding to the first camera, and can copy the image corresponding to the second camera (hereinafter referred to as the second image) to the port corresponding to the second camera, wherein the port corresponding to the first camera and the port corresponding to the second camera can both be ports between the multi-camera control module and the smooth zoom module.

[0187] In an embodiment of the present application, the overlay switch can be in an off state by default. When the multi-camera control module determines that the main camera is switched, the overlay switch can be instructed to switch from the off state to the on state, that is, the overlay switch is turned on; or, optionally, the overlay switch can also be in an on state by default. When the multi-camera control module determines that the main camera is switched, the auxiliary camera is instructed to capture and transmit images of a preset number of frames. After the image corresponding to the auxiliary camera of the preset number of frames is read, the auxiliary camera is instructed to no longer transmit its corresponding image. At this time, the multi-camera control module does not need to control the switch of the overlay switch.

[0188] In some embodiments, the smooth zoom module can read from the memory that the current auxiliary camera is the first camera, and read the current main camera, the second camera. After that, the smooth zoom module reads the first image corresponding to the first camera from the port corresponding to the first camera, and uses it as the image corresponding to the auxiliary camera, and reads the second image corresponding to the second camera from the port corresponding to the second camera, and uses it as the image corresponding to the main camera. After that, the smooth zoom module can respectively calculate the processing information corresponding to the first image, and calculate the processing information corresponding to the second image. The processing information here may include, for example, cropping information and mapping information, wherein the cropping information is used to unify the images corresponding to the two cameras into the same size, which may be a preset size; the mapping information is used to match according to the feature points in the corresponding images, and obtain the images corresponding to the two cameras at the same or similar viewing angles, so that the image content captured by the two cameras on the mobile phone is as consistent as possible, that is, the corresponding images captured by the two cameras can contain the same or similar photographed objects. Among them, the way in which the smooth zoom module calculates and obtains the image processing information can refer to the existing method, and the embodiment of the present application will not be described in detail.

[0189] In addition, an auxiliary output port may be added between the smooth zoom module and the auxiliary image post-processing, and a main output port may be provided between the smooth zoom module and the main image post-processing. In some embodiments, after the smooth zoom module obtains the processing information corresponding to the first image and the processing information corresponding to the second image, the first image and its corresponding processing information may be copied to the auxiliary output port, and the second image and its corresponding image processing information may be copied to the main output port.

[0190] In some embodiments, the auxiliary image postprocessor can read the first image and its corresponding processing information from the auxiliary output port, and the main image postprocessor can read the second image and its corresponding processing information from the main output port. Afterwards, the auxiliary image postprocessor can process the first image according to the processing information corresponding to the first image, obtain the processed auxiliary image, and then transmit the auxiliary image to the image processing engine; at the same time, the main image postprocessor can also process the second image according to the processing information corresponding to the second image, obtain the processed main image, and then transmit the main image to the image processing engine.

[0191] In some embodiments, the image processing engine can query from the memory whether the overlay switch is in the on state. Based on the overlay switch being in the on state, the image processing engine can perform overlay processing on the acquired main image and auxiliary image according to the overlay algorithm. The overlay processing can be performed in real time, that is, each time the image processing engine acquires a frame of processed main image and a corresponding processed auxiliary image, it will perform overlay processing on the two frames of images in real time. The main image and the corresponding auxiliary image that are simultaneously overlaid can be images that are simultaneously captured, transmitted, and post-processed by the main camera and the auxiliary camera, respectively.

[0192] In a possible implementation, the on state of the superimposition switch can be set as a preset information, and the preset information is stored in the memory. When the superimposition switch is turned on, the multi-camera control module can set the preset information to indicate that the superimposition switch is in the on state; when the superimposition switch is turned off, the multi-camera control module can set the preset information to indicate that the superimposition switch is in the off state. Optionally, the on state of the superimposition switch can also be represented by an indicator bit, and the information in the indicator bit can be used to indicate whether the image processing engine uses the superimposition algorithm to perform superimposition processing on the image; or, optionally, the superimposition switch can also be regarded as an algorithm calling function, and the algorithm calling function is used by the image processing engine to call the superimposition algorithm to perform superimposition processing on the image. Optionally, when the superimposition switch is an indicator bit or an algorithm calling function, the multi-camera control module can send it to the image processing engine when the first camera is switched to the second camera, instructing the image processing engine to process the image according to the superimposition algorithm.

[0193] It is worth noting that, compared with the original software architecture of the main image processing, in the software architecture provided in the embodiment of the present application, another image processing path including functional modules such as a multi-camera control module, a smooth zoom module, an auxiliary image post-processor, and an image processing engine is newly added, forming a dual-channel image processing architecture capable of simultaneously transmitting and processing two-channel images. Specifically, in the architecture provided in the embodiment of the present application, an auxiliary output port is newly added between the smooth zoom module and the auxiliary image post-processor, and the auxiliary output port enables the auxiliary image post-processing module to read the auxiliary image and its processing data; and a corresponding port is newly added between the auxiliary image post-processor and the image processing engine, and the port enables the auxiliary image post-processing module to transmit the auxiliary post-processed image to the image processing engine. In addition, the software architecture provided in the embodiment of the present application also adds an overlay algorithm to the image processing engine, so that in the case of dual-channel image processing, the image processing engine can perform overlay processing on the main post-processed image and the auxiliary post-processed image according to the overlay algorithm.

[0194] It should be noted that the above text only takes the dual-channel image processing strategy of the main channel and the auxiliary channel as an example to introduce the image display method provided by the embodiment of the present application, but in actual application, the image display method can also include processing more channels of images simultaneously through more channels. For example, the image display method can provide a software architecture with three image processing channels, and the software architecture includes a main channel and two auxiliary channels, which can simultaneously transmit and process images corresponding to three cameras. The embodiment of the present application does not limit the number of image processing channels in the software architecture.

[0195] According to the image display method provided in the embodiment of the present application, when the camera is switched, the image corresponding to the camera before the switching of a preset number of frames and the image corresponding to the camera after the switching of a preset number of frames are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed, so that when different cameras are switched, the captured pictures can be smoothly transitioned, the picture jump feeling is eliminated, and the user's viewing experience is improved.

[0196] For example, Figure 8 As shown, it is a schematic flow chart of another method for image display provided by an embodiment of the present application. The process in the embodiment of the present application may be a process related to the smooth zoom module, which may be implemented by the smooth zoom module in the mobile phone (corresponding to Figure 6 The multi-camera control module 605 in the embodiment is used as the main body for execution. The process may specifically include the following steps:

[0197] S801, determining whether the overlay switch is in an on state.

[0198] In some embodiments, the smooth zoom module can determine whether the superimposition switch is in the on state according to the preset information in the memory for indicating the on state of the superimposition switch. Alternatively, the smooth zoom module can also determine whether the superimposition switch is in the on state according to the indication bit for indicating the on state of the superimposition switch; or, alternatively, the smooth zoom module can also determine whether the superimposition switch is in the on state according to whether it reads a function that calls the superimposition switch.

[0199] Specifically, the process of the smooth zoom module determining whether the superimposition switch is in an on state may include: the smooth zoom module reading preset information from the memory, wherein the preset information may be used to indicate whether the superimposition switch is in an on state; when the preset information indicates that the superimposition switch is in an on state, the smooth zoom module may obtain that the superimposition switch is in an on state; and when the preset information indicates that the superimposition switch is in an off state, the smooth zoom module may obtain that the superimposition switch is in an off state.

[0200] Alternatively, optionally, the process by which the smooth zoom module determines whether the overlay switch is in the on state may also include: the smooth zoom module may determine whether the overlay switch is in the on state based on the message transmitted by the multi-camera control module, for example, the smooth zoom module may determine that the overlay switch is in the on state based on a specific indication bit in the message transmitted by the multi-camera control module carrying information indicating that the overlay switch is in the on state; or, optionally, the smooth zoom module may determine that the overlay switch is in the on state based on an algorithm calling function carried in the message transmitted by the multi-camera control module.

[0201] When the judgment result in this step is "no", that is, the superimposition switch is not in the on state, then step S802 can be executed next, that is, no operation is performed, and the auxiliary path is closed. When the judgment result in this step is "yes", that is, the superimposition switch is in the on state, then steps S803 to S807 can be executed next.

[0202] The following mainly introduces the situation where the judgment result in this step is "yes".

[0203] S803, obtaining the main input port information and the auxiliary input port information.

[0204] Among them, the auxiliary input port can correspond to the port corresponding to the first camera mentioned above, which can refer to the port between the multi-camera control module and the smooth zoom module in the auxiliary channel; the main input port can correspond to the port corresponding to the second camera mentioned above, which can refer to the port between the multi-camera control module and the smooth zoom module in the main channel.

[0205] In some embodiments, the smooth zoom module can receive the first image transmitted by the multi-camera control module through the auxiliary input port, and can read the second image transmitted by the multi-camera control module through the main input port.

[0206] In some embodiments, the smooth zoom module of the mobile phone can query the main input port information and the auxiliary input port information from the mobile phone memory. Exemplarily, the auxiliary input port information here can refer to the address information of the auxiliary input port, and the main input information can refer to the address information of the main input port.

[0207] In some embodiments, the smooth zoom module can obtain the first image transmitted by the multi-camera control module based on the auxiliary input port information, and can also obtain the second image transmitted by the multi-camera control module based on the main input port information.

[0208] S804, obtaining processing information corresponding to a first image, where the first image is an image corresponding to the auxiliary camera.

[0209] In some embodiments, the auxiliary image processing information here may correspond to the processing information corresponding to the first image, and the processing information may include cropping and / or mapping information for the auxiliary image. The cropping information may include, for example, a preset image size; the mapping information may include, for example, a mapping rule, a sparse dot matrix for mapping, etc. The smooth zoom module may perform a one-time processing on the acquired first image based on the auxiliary image processing information, wherein the one-time processing may include cropping and / or mapping.

[0210] In some embodiments, the smooth zoom module may query the memory for processing information corresponding to images under different shooting parameters.

[0211] It should be noted that the clipping information and mapping information in the embodiments of the present application can be flexibly set as needed, and the embodiments of the present application are not limited to this.

[0212] S805, copy the first image to the auxiliary output port, and output corresponding processing information.

[0213] The auxiliary channel output port may refer to a port between the smooth zoom module and the auxiliary channel image post-processing module in the auxiliary channel. The smooth zoom module may transmit the acquired auxiliary channel image to the auxiliary channel image post-processing module through the port; in addition, the smooth zoom module may also output cropping information for the auxiliary channel image through the auxiliary channel output port. Optionally, the smooth zoom module may also output mapping information for the auxiliary channel image through the auxiliary channel output port.

[0214] In some embodiments, the software architecture in the embodiments of the present application may further include a main output port, which may refer to a port between the smooth zoom module and the main image post-processing module in the main path. The smooth zoom module may transmit the main image to the main image post-processing module through the port; in addition, the smooth zoom module may also output cropping information for the main image through the main output port. Optionally, the smooth zoom module may also output mapping information for the main image through the main output port.

[0215] S806: The auxiliary road image post-processing module processes the first image according to the processing information.

[0216] In some embodiments, the auxiliary image post-processing module may process the first image based on the first image sent by the smooth zoom module and the processing information for the first image. Specifically, the auxiliary image post-processing module may crop the auxiliary image based on the cropping information corresponding to the auxiliary first image; optionally, the auxiliary image post-processing module may also map the auxiliary image based on the mapping information corresponding to the auxiliary image.

[0217] In some embodiments, the main image post-processing module may also process the second image based on the second image sent by the smooth zoom module and the processing information for the second image. Specifically, the main image post-processing module may crop the second image based on the cropping information corresponding to the second image; optionally, the main image post-processing module may also map the second image based on the mapping information corresponding to the second image.

[0218] It should be noted that the purpose of mapping is to align the corresponding images taken by the main camera and the auxiliary camera at the same time, and the alignment includes pixel alignment, alignment of the photographed object based on the feature points of the photographed object included in the image, etc. The specific implementation process of the mapping can refer to the existing process, and the embodiments of the present application are not limited to this.

[0219] In some embodiments, after the auxiliary image post-processor processes the first image once, the auxiliary image obtained after the processing can be transmitted to the image processing engine; similarly, after the main image post-processor processes the second image once, the main image obtained after the processing can be transmitted to the image processing engine. The process of the auxiliary image post-processing module processing the first image and the process of the main image post-processing module processing the second image can be similar to the process in the existing process, and will not be described in detail here.

[0220] In some embodiments, the image processing engine can perform superimposition processing on the main image after one processing and the corresponding auxiliary image after one processing according to a preset superimposition algorithm, obtain the display image, and transmit the display image to the display module; thereafter, the display module sends the acquired superimposed image to the display screen, thereby presenting a smooth image transition process with a superimposition effect to the user.

[0221] The following introduces the overlay processing involved in the image display method provided in the embodiment of the present application.

[0222] In some embodiments, the overlay processing may specifically include: when the overlay switch is turned on, the image processing engine reduces the pixel value of the auxiliary image corresponding to the auxiliary camera frame by frame from the current pixel value according to a preset first weight, until the pixel value of the auxiliary image is reduced to 0 when the overlay frame number is reached; at the same time, according to a preset second weight, the pixel value of the main image corresponding to the main camera is increased frame by frame from 0 until the pixel value of the main image is increased to a preset pixel value when the overlay frame number is reached, wherein the preset pixel value may be the same as the pixel value of the main image displayed on the display screen when the camera is not switched.

[0223] For example, assume that the frame corresponding to the camera switching is frame 0, the number of superimposed frames is N, and N is an integer greater than 1. Then, when the pixel value of the auxiliary image is reduced in a linear manner, the change of the pixel value of the auxiliary image over time (or frame number) can be seen from Fig.9A Specifically, when the camera is switched, from the 1st frame to the Nth frame after the switching point, the mobile phone can reduce the pixel value of the main image frame by frame according to the first weight, wherein the first weight can be linearly reduced with time (or number of frames), in which case, the pixel value of the auxiliary image also changes linearly with time (or number of frames).

[0224] When the pixel value of the main image is increased linearly, the change of the pixel value of the main image over time (or frame number) can be seen from Fig. 9B Specifically, when the camera is switched, from the 1st frame to the Nth frame after the switching point, the mobile phone can increase the pixel value of the main image frame by frame according to the second weight, wherein the second weight can increase linearly with time (or number of frames), in which case, the pixel value of the main image also changes linearly with time (or number of frames).

[0225] It should be noted that Fig.9A and Fig. 9B The embodiment is only introduced by taking the example that the pixel values ​​of the auxiliary road image and the main road image change linearly with time. In actual applications, the pixel values ​​of the auxiliary road image and the main road image may also change nonlinearly with time, and the embodiment of the present application does not limit this.

[0226] It should also be noted that the embodiment of the present application only uses the pixel value as an example to illustrate the parameter adjusted in the overlay process, but in actual applications, the parameters adjusted in the overlay process can also include other parameters such as color, brightness, etc., and the embodiment of the present application does not limit this.

[0227] According to the image display method provided in the embodiment of the present application, when the camera is switched, the image corresponding to the camera before the switching of a preset number of frames and the image corresponding to the camera after the switching of a preset number of frames are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed, so that when different cameras are switched, the captured pictures can be smoothly transitioned, the picture jump feeling is eliminated, and the user's viewing experience is improved.

[0228] For example, Fig.10 As shown, it is a schematic flow chart of another method for image display provided in an embodiment of the present application. The process can be applied to a terminal device including at least a first camera and a second camera, and the process can specifically include the following steps:

[0229] S1001, receiving a user's adjustment operation on the shooting magnification.

[0230] S1002: In response to the adjustment operation, the image displayed on the shooting interface is enlarged or reduced as the shooting magnification changes.

[0231] S1003, when the shooting magnification reaches a preset magnification, the first pixel value corresponding to the first picture included in the image currently displayed on the shooting interface gradually decreases from the preset pixel value, and the second pixel value corresponding to the second picture included in the image currently displayed gradually increases, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in the N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

[0232] In some embodiments, when the shooting magnification reaches a preset magnification, the main camera of the terminal device can be switched from the first camera to the second camera. Exemplarily, the first camera can be, for example, one of an ultra-wide-angle camera, a wide-angle lens, and a telephoto camera; the second camera can be, for example, one of an ultra-wide-angle camera, a wide-angle lens, and a telephoto camera, but the first camera is not the same as the second camera. The first camera is the main camera before the main camera is switched. After the main camera is switched, the first camera is no longer the main camera, but continues to shoot and transmit images as an auxiliary camera; the second camera is the main camera after the main camera is switched. Before the main camera is switched, the second camera can be in a state of being powered on but not transmitting images.

[0233] It can be understood that in the embodiment of the present application, after the main camera is switched, the path for transmitting the image corresponding to the first camera can be called the auxiliary path, and the path for transmitting the image corresponding to the second camera can be called the main path. The main path can be a path that is connected when only one image is transmitted, and the auxiliary path can be a path that is connected only when the overlay switch is turned on. In other embodiments, the main path and the auxiliary path can be pre-set, and the camera corresponding to the main path can be called the main camera, and the camera corresponding to the auxiliary path can be called the auxiliary camera. The camera corresponding to the main path can refer to a camera whose image is transmitted via the main path, and the image corresponding to the camera is processed by the main path image post-processing module.

[0234] In some embodiments, the user's adjustment operation for the shooting magnification may include: the user inputs an adjustment operation for the magnification method or reduction control in the terminal device shooting interface, and the adjustment operation can be used to enlarge the shooting magnification or to reduce the shooting magnification. Exemplarily, the adjustment operation can be an operation of sliding up to enlarge or reduce the control, or can be an operation of sliding down to enlarge or reduce the control.

[0235] In some embodiments, the image displayed on the shooting interface is enlarged or reduced as the shooting magnification changes, which may specifically include: the image displayed on the shooting interface is dynamically enlarged or dynamically reduced as the shooting magnification changes. For example, when the shooting magnification increases from 1× to 2.5×, the size of the object included in the image (such as the preview image) displayed on the shooting interface will also increase from 1× to 2.5×, and the user can observe the process of the object included in the image dynamically enlarging in the preview box of the shooting interface.

[0236] In some embodiments, the preset magnification may refer to the magnification when the main camera is switched. It should be noted that different shooting parameters may correspond to different main cameras, and the main camera may also be played by different cameras at different shooting magnifications. For example, when other parameters are default parameters and the shooting magnification increases from 1× to 2.5×, the main camera may be changed from a wide-angle camera to a telephoto camera.

[0237] In some embodiments, the screen currently displayed on the shooting interface may be an image based on the image taken by the first camera and the image taken by the second camera after image post-processing and overlaying. The first screen may be the screen corresponding to the first image taken by the first camera, and the second screen may be the screen corresponding to the second image taken by the second camera.

[0238] It can be understood that when the preset magnification is reached and the main camera is switched, the main camera before the switch continues to transmit images as an auxiliary camera, and the image corresponding to the auxiliary camera and the image corresponding to the main camera are processed simultaneously through the auxiliary channel and the main channel respectively, and then the auxiliary channel image and the main channel image are superimposed to obtain an image with a gradual disappearance and appearance effect, so that when different cameras are switched, the images corresponding to the different cameras can switch smoothly, avoiding the problem of screen jumps caused by hard switching.

[0239] In some embodiments, the terminal device includes an auxiliary channel image post-processing module and a main channel image post-processing module, and the method also includes: when the shooting magnification reaches a preset magnification, obtaining a first image shot by the first camera and a second image shot by the second camera, the first image is transmitted to the auxiliary channel image post-processing module via the auxiliary channel for processing, and the second image is transmitted to the main channel image post-processing module via the main channel for processing; wherein, the first image is any one of the N frames of images taken by the first camera after reaching the preset magnification, and the second image is any one of the N frames of images taken by the second camera after reaching the preset magnification.

[0240] It should be noted that after the main camera is switched, the first camera can continue to shoot images as an auxiliary camera and transmit the images via the auxiliary channel. The first camera can continue to shoot and transmit N frames of images after the main camera is switched, and the N frames of images can be used for superimposition processing with the images of the second camera.

[0241] In one possible embodiment, the main path and the auxiliary path can be two parallel paths, which can be used to simultaneously process the image in the main path (i.e., the image corresponding to the second camera) and the image in the auxiliary path (i.e., the image corresponding to the first camera).

[0242] In some embodiments, when the shooting magnification reaches the preset magnification, the method further includes: obtaining whether the main camera corresponding to the main channel is switched; when the main camera corresponding to the main channel is switched from the first camera to the second camera, turning on the overlay switch, and the overlay switch is used to turn on or off the auxiliary channel.

[0243] It can be understood that by dynamically controlling the opening or closing of the auxiliary path, image post-processing of dual-path or (more-path) images can be performed when dual-path (or more-path) images need to be processed simultaneously.

[0244] In some embodiments, the method further includes: obtaining first cropping information corresponding to the first image, the first cropping information being used to crop the size of the first image to a preset size; and obtaining second cropping information corresponding to the second image, the second cropping information being used to crop the size of the second image to a preset size.

[0245] In some embodiments, the method further includes: cropping the second image according to the second cropping information, and acquiring the cropped main road image.

[0246] In some embodiments, the method further includes: reading whether the superimposition switch is in an on state; when the superimposition switch is in an on state, cropping the first image according to the first cropping information, and acquiring the cropped auxiliary path image.

[0247] In some embodiments, the auxiliary image and the corresponding main image are overlaid to obtain a display image after the overlay processing, and the overlay processing is used to gradually reduce the pixel value of the first picture corresponding to the auxiliary image and gradually increase the pixel value of the second picture corresponding to the main image in the fused image corresponding to the auxiliary image and the corresponding main image; and the display image is displayed.

[0248] In some embodiments, the method further includes: when the first picture disappears, turning off the dissolve switch.

[0249] In some embodiments, when the first picture disappears, the overlay switch is turned off, specifically including: when the number of frames of the auxiliary image reaches the N frames and the first picture disappears, the overlay switch is turned off; and / or, when the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

[0250] According to the image display method provided in the embodiment of the present application, when the camera is switched, the image corresponding to the camera before the switching of a preset number of frames and the image corresponding to the camera after the switching of a preset number of frames are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed, so that when different cameras are switched, the captured pictures can be smoothly transitioned, the picture jump feeling is eliminated, and the user's viewing experience is improved.

[0251] Based on the same technical concept, an embodiment of the present application further provides an image display device, the device comprising a first camera, a second camera, an auxiliary channel and a main channel, the auxiliary channel comprising a multi-camera control module, a smooth zoom module, an auxiliary channel image post-processing module and an image processing engine, the main channel comprising the multi-camera control module, the smooth zoom module, the main channel image post-processing module and the image processing engine, wherein: the multi-camera control module is used to turn on an overlay switch when the main camera is switched from the first camera to the second camera, the overlay switch is used to turn on or off the auxiliary channel, and the main camera is the camera corresponding to the main channel; The smooth zoom module is used to obtain the first cropping information corresponding to the first image taken by the first camera and the second cropping information corresponding to the second image taken by the second camera when the overlay switch is in the on state; the auxiliary image post-processing module is used to crop the first image according to the first cropping information and obtain the cropped auxiliary image; the main image post-processing module is used to crop the second image according to the second cropping information and obtain the cropped main image; the image processing engine is used to perform overlay processing on the auxiliary image and the corresponding main image to obtain the image for display.

[0252] In some embodiments, the multi-camera control module is further used to obtain the main camera switching from the first camera to the second camera according to the read first image and the second image.

[0253] In some embodiments, the image processing engine is specifically used to gradually reduce the first pixel value corresponding to the first picture from a preset pixel value, and gradually increase the second pixel value corresponding to the second picture when the overlay switch is in an on state, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in the N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

[0254] Based on the same technical concept, an embodiment of the present application also provides a terminal device, including a processor; a memory; the memory stores a computer program, and the computer program includes instructions. When the instructions are executed by the processor, the terminal device executes one or more steps of any of the above methods.

[0255] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable program instructions. When the computer-executable program instructions are executed on a computer, the computer or processor executes one or more steps in any of the above methods.

[0256] Based on the same technical concept, an embodiment of the present application also provides a computer program product comprising instructions, wherein the computer program product includes computer program code. When the computer program code runs on a computer, the computer or processor executes one or more steps in any of the above methods.

[0257] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0258] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0259] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A method for image display, characterized in that: Applied to a terminal device, the terminal device includes at least a first camera and a second camera, and the method includes: Receive the user's adjustment operation on the shooting magnification; In response to the adjustment operation, the image displayed on the shooting interface of the terminal device is enlarged or reduced as the shooting magnification changes; When the shooting magnification reaches a preset magnification, first cropping information and second cropping information are obtained, and the first image is cropped to a preset size according to the first cropping information, and the second image is cropped to the preset size according to the second cropping information, wherein the first image is an image captured by the first camera, and the second image is an image captured by the second camera; Acquire mapping information, and match feature points in the first image and the second image according to the mapping information; A first pixel value corresponding to a first picture included in the image currently displayed on the shooting interface gradually decreases from a preset pixel value, and a second pixel value corresponding to a second picture included in the image currently displayed gradually increases, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

2. The method according to claim 1, characterized in that The terminal device includes an auxiliary channel image post-processing module and a main channel image post-processing module, and the method further includes: When the shooting magnification reaches a preset magnification, a first image shot by the first camera and a second image shot by the second camera are obtained, the first image is transmitted to the auxiliary image post-processing module via the auxiliary path for processing, and the second image is transmitted to the main image post-processing module via the main path for processing; wherein, The first image is any one of the N frames of images taken by the first camera after reaching the preset magnification, and the second image is any one of the N frames of images taken by the second camera after reaching the preset magnification.

3. The method according to claim 2, characterized in that When the shooting magnification reaches the preset magnification, the method further includes: Obtain whether the main camera corresponding to the main road is switched; When the main camera corresponding to the main channel is switched from the first camera to the second camera, an overlay switch is turned on, and the overlay switch is used to turn on or off the auxiliary channel.

4. The method according to claim 3, characterized in that The method further comprises: The second image is cropped according to the second cropping information, and a cropped main road image is acquired.

5. The method according to claim 4, characterized in that The method further comprises: Read whether the overlay switch is in an on state; When the dissolve switch is in the on state, the first image is cropped according to the first cropping information, and the cropped auxiliary channel image is acquired.

6. The method according to claim 5, characterized in that The method further comprises: Performing a superimposition process on the auxiliary image and the corresponding main image to obtain a display image after the superimposition process, wherein the superimposition process is used to gradually reduce the pixel value of the first picture corresponding to the auxiliary image and gradually increase the pixel value of the second picture corresponding to the main image in a fused image corresponding to the auxiliary image and the corresponding main image; The sent image is displayed.

7. The method according to any one of claims 4 to 6, characterized in that: The method further comprises: When the first picture disappears, the dissolve switch is turned off.

8. The method according to claim 7, characterized in that When the first picture disappears, turning off the dissolve switch specifically includes: When the number of frames of the auxiliary channel image reaches the N frames and the first picture disappears, turning off the dissolve switch; and / or, When the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

9. An image display device, characterized in that: The device includes a first camera, a second camera, an auxiliary path and a main path, the auxiliary path includes a multi-camera control module, a smooth zoom module, an auxiliary path image post-processing module and an image processing engine, and the main path includes the multi-camera control module, the smooth zoom module, the main path image post-processing module and the image processing engine, wherein: The multi-camera control module is used to turn on the overlay switch when the main camera is switched from the first camera to the second camera, and the overlay switch is used to turn on or off the auxiliary channel, and the main camera is the camera corresponding to the main channel; The smooth zoom module is used to obtain first cropping information corresponding to the first image taken by the first camera and second cropping information corresponding to the second image taken by the second camera when the overlay switch is in the on state; The auxiliary road image post-processing module is used to crop the first image according to the first cropping information and obtain the cropped auxiliary road image; The main road image post-processing module is used to crop the second image according to the second cropping information and obtain the cropped main road image; The image processing engine is used to perform superimposition processing on the auxiliary channel image and the corresponding main channel image to obtain an image for display.

10. The device according to claim 9, characterized in that The multi-camera control module is further used to obtain the main camera switching from the first camera to the second camera according to the read first image and the second image.

11. The device according to claim 9 or 10, characterized in that The image processing engine is specifically used to, when the overlay switch is in an on state, gradually reduce the first pixel value corresponding to the first picture from a preset pixel value, and gradually increase the second pixel value corresponding to the second picture, and when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera after reaching the preset magnification, and the second picture is a picture in the N frames of images taken by the second camera after reaching the preset magnification, and N is an integer greater than 1.

12. A terminal device, characterized in that: include: processor; Memory; The memory stores a computer program, wherein the computer program includes instructions, and when the instructions are executed by the processor, the terminal device executes the method according to any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 8.

Images