Image display method and related equipment

Abstract

The invention provides an image display method and related equipment, and relates to the technical field of image processing. The method comprises the following steps: receiving an adjustment operation of a user for a zoom ratio; in response to the adjustment operation, an image displayed on a shooting interface of the terminal equipment is zoomed in or out along with the change of the zoom ratio; before a main camera is switched from a first camera to a second camera, a first pixel value corresponding to a first picture included in an image currently displayed on a shooting interface is gradually reduced, and a second pixel value corresponding to a second picture included in the image currently displayed is gradually increased. According to the method, two paths of images corresponding to the main camera before and after switching are subjected to superposition processing in advance before the main camera is actually switched, so that the problem of slow response to smooth picture switching caused by superposition processing after the main camera is switched is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image display method and related equipment. Background Art

[0002] In recent years, with the continuous development of terminal devices, different types of terminal devices, especially terminal products with multiple cameras, have been widely used in users' lives. In order to enable users to obtain a better camera experience, many terminal devices are currently equipped with multiple types of cameras, such as dual-camera terminal devices with two cameras, triple-camera terminal devices with three cameras, and even terminal devices with more cameras. Since different cameras have different parameters such as focal length and field of view (FOV), when users switch between different cameras during shooting, the image effects obtained will also be different, and the shooting picture will usually jump, which has a poor visual effect from the user's perspective.

[0003] Therefore, how to make the captured images transition smoothly and eliminate the sense of picture jump when switching between different cameras has become an urgent problem to be solved. Summary of the invention

[0004] An embodiment of the present application provides a method for image display, which solves the problem of slow response of smooth image switching caused by performing the overlay processing after switching the main camera by overlaying the two images corresponding to the main camera before and after the actual switching of the main camera.

[0005] In a first aspect, a method for displaying an image is provided, which is applied to a terminal device, wherein the terminal device includes at least a first camera and a second camera, and the method includes:

[0006] Receive the user's adjustment operation on the zoom ratio;

[0007] In response to the adjustment operation, the image displayed on the shooting interface of the terminal device is enlarged or reduced as the zoom ratio changes;

[0008] Before the main camera is switched from the first camera to the second camera, the first pixel value corresponding to the first picture included in the image currently displayed on the shooting interface gradually decreases, and the second pixel value corresponding to the second picture included in the currently displayed image gradually increases.

[0009] In one possible implementation, the first camera may be, for example, an ultra-wide-angle camera, a wide-angle lens, or a telephoto camera; the second camera may be, for example, an ultra-wide-angle camera, a wide-angle lens, or a telephoto camera, but the first camera is different from the second camera.

[0010] In a possible implementation, the first camera is used as the main camera before the main camera is switched, and the second camera is used as the main camera after the main camera is switched. The switching here specifically refers to the switching of the main camera.

[0011] In a possible implementation, the user's adjustment operation on the zoom ratio may include: the user inputs a click operation on the zoom ratio adjustment control in the terminal device shooting interface, and the adjustment operation may be used to increase the zoom ratio or to reduce the zoom ratio.

[0012] It should be noted that different shooting parameters can correspond to different main cameras, and different cameras can also serve as the main camera under different zoom ratios. For example, when the shooting distance and shooting brightness are the same, and the zoom ratio increases from 1× to 2.5×, the main camera can be changed from a wide-angle camera to a telephoto camera.

[0013] In one possible implementation, the picture currently displayed on the shooting interface may be an image based on the image taken by the first camera and the image taken by the second camera after image post-processing and overlaying. The first picture may be the picture corresponding to the first image taken by the first camera, and the second picture may be the picture corresponding to the second image taken by the second camera.

[0014] According to the image display method provided by the embodiment of the present application, the main camera after switching is predicted according to the shooting distance, brightness and zoom ratio during the period from when the user performs the point-to-point operation to when the main camera actually switches, and the images respectively collected by the two main cameras before and after the switching are superimposed before the switching time point, thereby shortening the time between the point-to-point time point and the presentation of the final image. In other words, the method completes the superimposition processing of the two-way images of the new and old main cameras before the switching time point, rather than starting the superimposition processing after the switching time point, thereby achieving a smooth transition of the FOV while not affecting the image response speed to the user adjusting the zoom ratio.

[0015] In combination with the first aspect, in some implementations of the first aspect, the first picture is an image picture corresponding to the first camera, and the second picture is an image picture corresponding to the second camera.

[0016] In combination with the first aspect, in some implementations of the first aspect, the method further includes: in response to the adjustment operation, adjusting the first zoom ratio to a second zoom ratio, wherein the first zoom ratio corresponds to a first shooting distance and a first shooting brightness, the second zoom ratio corresponds to a second shooting distance and a second shooting brightness, the first shooting distance is the same as or different from the second shooting distance, the first shooting brightness is the same as or different from the second shooting brightness, the main camera corresponding to the first zoom ratio, the first shooting distance and the first shooting brightness is the first camera, and the main camera corresponding to the second zoom ratio, the second shooting distance and the second shooting brightness is the first camera;

[0017] A new main camera is acquired as the second camera according to the second zoom magnification, the second shooting distance, and the second shooting brightness.

[0018] It should be noted that, after determining that the new main camera is the second camera, the second camera can capture images and transmit them via the main channel, and the first camera can continue to capture images and transmit them via the auxiliary channel. Alternatively, the second camera can capture images and transmit them via the auxiliary channel, and the first camera can continue to capture images and transmit them via the main channel.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: in response to the second camera being used as the new main camera, starting from N frames of images closest to a switching point, superimposing the first image corresponding to the first camera and the second image corresponding to the second camera, wherein the switching point is the time point when the main camera is switched from the first camera to the second camera, and N is an integer greater than 1.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera, and the second picture is a picture in the N frames of images taken by the second camera, and the preset pixel value is the pixel value of the picture displayed before the adjustment operation.

[0021] In combination with the first aspect, in some implementations of the first aspect, the overlay process ends before the switching point.

[0022] In combination with the first aspect, in some implementations of the first aspect, the method further includes: before switching the main camera from the first camera to the second camera, detecting that the second shooting distance changes to a third shooting distance, and / or the second shooting brightness changes to the third shooting brightness;

[0023] According to the third shooting distance and / or the third shooting brightness, acquiring a new main camera as the third camera;

[0024] If the third camera is different from the second camera, terminating the superimposition process for the first image and the second image;

[0025] If the third camera is the same as the second camera, the overlay processing process for the first image and the second image continues to be executed.

[0026] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0027] Before switching the main camera from the first camera to the second camera, detecting that the second zoom ratio is adjusted to a third zoom ratio;

[0028] According to the third zoom ratio, acquiring a new main camera as a fourth camera;

[0029] If the fourth camera is different from the second camera, terminating the superimposition process for the first image and the second image;

[0030] If the fourth camera is the same as the second camera, the overlay processing process for the first image and the second image continues to be performed.

[0031] In combination with the first aspect, in some implementations of the first aspect, the adjustment operation is a click operation on the zoom ratio adjustment control.

[0032] In combination with the first aspect, in certain implementations of the first aspect, when the N frames of images closest to the switching point are acquired, an overlay switch is turned on, and the overlay switch is used to turn on or off the auxiliary path.

[0033] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0034] Acquire first cropping information of the first image, where the first cropping information is used to crop the size of the first image to a preset size; and,

[0035] Second cropping information of the second image is acquired, where the second cropping information is used to crop the size of the second image to a preset size.

[0036] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0037] The second image is cropped according to the second cropping information, and a cropped main road image is acquired.

[0038] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0039] Read whether the overlay switch is in an on state;

[0040] When the dissolve switch is in the on state, the first image is cropped according to the first cropping information, and the cropped auxiliary channel image is acquired.

[0041] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0042] Performing a superimposition process on the auxiliary image and the corresponding main image to obtain a display image after the superimposition process, wherein the superimposition process is used to gradually reduce the pixel value of the picture corresponding to the auxiliary image and gradually increase the pixel value of the picture corresponding to the main image in a fused image corresponding to the auxiliary image and the corresponding main image;

[0043] The sent image is displayed.

[0044] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0045] When the first picture disappears, the dissolve switch is turned off.

[0046] In combination with the first aspect, in some implementations of the first aspect, when the first picture disappears, turning off the dissolve switch specifically includes:

[0047] When the number of frames of the auxiliary channel image reaches the N frames and the first picture disappears, turning off the dissolve switch; and / or,

[0048] When the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

[0049] In a second aspect, a terminal device is provided, including:

[0050] processor;

[0051] Memory;

[0052] The memory stores a computer program, which includes instructions. When the instructions are executed by the processor, the terminal device executes the method as described in any one of the first aspects above.

[0053] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable program instructions, and when the computer-executable program instructions are executed on a computer, the computer executes the method as described in any one of the above-mentioned first aspects.

[0054] In a fourth aspect, a computer program product is provided, the computer program product comprising a computer program code, and when the computer program code is run on a computer, the computer is caused to execute the method as described in any implementation manner of the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a graphical user interface (GUI) for adjusting a zoom ratio through a point-to-point operation provided in an embodiment of the present application.

[0056] Figure 2A A schematic diagram of the screen switching process corresponding to different cameras provided in an embodiment of the present application.

[0057] Figure 2B A schematic diagram of another screen switching process corresponding to different cameras provided in an embodiment of the present application.

[0058] Figure 3 A schematic structural diagram of the back of a mobile phone provided in an embodiment of the present application.

[0059] Figure 4 A schematic structural diagram of a terminal device 100 provided in an embodiment of the present application.

[0060] Figure 5 A software structure block diagram of a terminal device 100 provided in an embodiment of the present application.

[0061] 6A to 6F A schematic diagram of a GUI that may be involved in the implementation of an image display method provided in an embodiment of the present application.

[0062] Figure 7 A schematic diagram of the interaction of functional modules involved in the implementation of an image display method provided in an embodiment of the present application.

[0063] Figure 8 A schematic flowchart of an image display method provided in an embodiment of the present application.

[0064] Fig. 9 A schematic flowchart of another image display method provided in an embodiment of the present application.

[0065] Fig. 10A and Fig. 10BA schematic diagram of a situation in which the brightness of an image captured by a main camera and an auxiliary camera changes linearly provided in an embodiment of the present application.

[0066] Fig.11 A schematic flowchart of another image display method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] It should be noted that the terms used in the implementation method part of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated obstacles, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two, "at least one" and "one or more" mean one, two or more than two.

[0068] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, it is defined that the "first" and "second" features may explicitly or implicitly include one or more of the features.

[0069] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing at different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0070] In order to better understand the method for optimizing the overlay effect when switching cameras provided in the embodiments of the present application, the terms or definitions that may be involved in the embodiments of the present application are first introduced below.

[0071] 1. New main camera, original main camera

[0072] In the shooting scene, the main camera (or display camera) is related to shooting parameters such as zoom ratio, shooting distance, and brightness. Different shooting parameters will correspond to different main cameras.

[0073] After the terminal device receives the user input to adjust the zoom ratio, if the main camera corresponding to the new shooting parameters is different from the main camera corresponding to the original shooting parameters before the adjustment, the terminal device needs to switch the main camera later. For the sake of distinction, in this case, the main camera corresponding to the original shooting parameters before the zoom ratio adjustment is described as the original main camera, and the main camera corresponding to the new shooting parameters after the zoom ratio adjustment is described as the new main camera.

[0074] In an embodiment of the present application, if it is necessary to perform overlay processing on the two images respectively captured by the new main camera and the original main camera, the image captured by the new main camera can be transmitted to the main image post-processor through the main image processing path (hereinafter collectively referred to as the main path), and the image captured by the original main camera can be transmitted to the auxiliary image post-processor via the auxiliary image processing path (hereinafter collectively referred to as the auxiliary path). The image corresponding to the new main camera can be described as a main image, and the image captured by the original main camera can be described as an auxiliary image. The two image processing paths can process the main image and the auxiliary image at the same time. Among them, the main path may include, for example: main camera-multi-camera control module-smooth zoom module-main image post-processing module (or main image post-processor); the auxiliary path may include, for example: auxiliary camera-multi-camera control module-smooth zoom module-auxiliary image post-processing module (or auxiliary image post-processor). Optionally, the main path and the auxiliary path may also include an image processing engine, a display module, and a display screen.

[0075] It should be noted that the multi-camera control module, smooth zoom module, image processing engine, display module and display screen in the main path and the auxiliary path can be the same module or device. In other words, the main path and the auxiliary path have their own corresponding cameras and image post-processing modules, and the other functional modules or devices involved can be the same.

[0076] For the specific structure of the multi-channel image processing pipeline, please refer to the following Figure 7 The introduction in the embodiments will not be described in detail here.

[0077] It should also be noted that when the main camera is working, other cameras can also be powered on.

[0078] 2. Point cutting operation

[0079] In the embodiment of the present application, the point-to-point operation may refer to an operation of switching the zoom ratio by clicking, and the click operation may also be described as point-to-point. Figure 1 As shown, in a video shooting scene with an initial zoom ratio of 1×, the user can enlarge the zoom ratio from 1× to 2× by clicking the zoom ratio adjustment button 01 in the preview interface. At this time, the user's operation of clicking button 01 can be called a point-to-point operation.

[0080] It should be noted that the point-to-point operation is only an example of a method for adjusting the zoom ratio, and in particular, an example of a method for quickly adjusting the zoom ratio. The method of quickly adjusting the zoom ratio here can be understood as a method in which the time for adjusting the zoom ratio is extremely short, or even negligible. In some embodiments, the method of quickly adjusting the zoom ratio may also include many other methods. For example, a user inputs the command "2x zoom ratio" to the terminal device through voice. In response to the command, the terminal device automatically adjusts the zoom ratio to 2×. Since the zoom ratio adjustment process after the terminal device receives the command is very short, it can be regarded as a method for quickly adjusting the zoom ratio. The embodiments of this application below only introduce the point-to-point operation as an example of quickly adjusting the zoom ratio, but it is not limited to this in actual applications.

[0081] 3. Cut time point

[0082] In the embodiment of the present application, the point-cut time point can also be described as a point-cut point. Figure 2A As shown, the point-to-point time point refers to the time when the user inputs a point-to-point operation to the electronic device, or refers to the time when the electronic device receives the point-to-point operation input by the user.

[0083] 4. Switching time point

[0084] In the embodiment of the present application, the switching time point may also be described as a switching point or a main camera switching point. Figure 2A As shown, the switching time point may refer to the time when the main camera is switched, that is, the time point when one main camera is switched to another main camera.

[0085] For example, when the user starts shooting (such as corresponding to Figure 2A The zoom ratio of the start shooting time point in the image is 1× (or 1x), and then the user Figure 2A In response to the point-cut operation, the terminal device switches the main camera from camera 1 to camera 2 at a second time, and the second time corresponding to the switching of the main camera is the switching point (e.g., Figure 2A The main camera shown switches from camera 1 to camera 2).

[0086] It should be noted that in the embodiment of the present application, the switching point is not the same as the point-cut time point. Generally speaking, after receiving the point-cut operation, the terminal device needs to spend a certain amount of time to determine whether the main camera needs to be switched based on the adjusted endpoint zoom ratio, shooting distance, brightness, etc., and execute the operation of switching the main camera after determining that the main camera needs to be switched. Therefore, the switching point is usually later than the point-cut point. The time difference between the point-cut point and the switching point is related to the amplitude of the change in the zoom ratio. For example, when the zoom ratio is adjusted from 0.5× to 1×, the time difference between the point-cut point and the switching point can be the duration corresponding to 6-10 frames of images; when the zoom ratio is adjusted from 1× to 2.5×, the time difference between the point-cut point and the switching point can be the duration corresponding to 10-15 frames of images, etc.

[0087] 5. Overlay and overlay effects

[0088] Dissolve is the gradual transition from one image to another, allowing the user to feel a continuous connection between the shots.

[0089] The overlay effect can be reflected in the video as the previous frame image gradually disappears (fades out) and the next frame image gradually appears (fades in), resulting in the two frames of the image overlapping for several seconds.

[0090] Combined with the above introduction, it can be understood that if the zoom ratio is enlarged (or the shooting distance, brightness, etc. are adjusted) during the video shooting process, when it is enlarged to a certain ratio (or increased to a certain distance, increased to a certain brightness), the main camera will switch, for example, the main camera switches from a wide-angle camera to a telephoto camera. At this time, if only one channel of image display (which can correspond to the main channel image) is still processed, the image in the shooting interface will jump directly from the picture taken by the wide-angle camera to the picture taken by the telephoto camera, resulting in a picture jump that the user can intuitively perceive. Among them, the picture jump includes, for example: the brightness, color, position of the object being photographed, etc. of the picture in the two frames before and after the jump change. However, if the pictures taken by the cameras before and after the main camera is switched are processed simultaneously through multiple image processing channels, and then the pictures taken by different cameras after processing are superimposed, then the final display effect can be realized by gradually changing from the picture taken by the main camera before the switch to the image taken by the main camera after the switch. From the user's perspective, the image will gradually transition to the picture taken by the main camera after switching, without any large jumps.

[0091] 6. Field of View

[0092] The field of view FOV can also be described as the field of view. The size of the field of view determines the field of view of the camera. In general, the relationship between the field of view and the focal length is that the larger the field of view, the shorter the focal length. For example, in the embodiment of the present application, if the cameras in the mobile phone are classified by the field of view, as shown in Table 1 below, they can be divided into the following categories:

[0093] Table 1

[0094] Field of view / Less than or equal to 40° Greater than 40° and less than or equal to 60° More than 60° Camera Type Telephoto camera Wide-angle camera Ultra-wide-angle camera

[0095] Combined with the introduction in the background technology, most current mobile phones have two to three, or even more cameras (or lenses, lens modules, camera modules). For example, Figure 3 As shown, it is a structural schematic diagram of the back of a mobile phone provided in an embodiment of the present application. The mobile phone includes three cameras (such as camera 1, camera 2 and camera 3) and a flash. Exemplarily, the types of the three cameras can be shown in Table 2 below.

[0096] Table 2

[0097] Camera Camera 1 Camera 2 Camera 3 type Ultra-wide-angle camera Wide-angle camera Telephoto camera

[0098] In some possible shooting scenarios, in order to obtain the best shooting effect, different shooting distances, brightness, and zoom ratios may correspond to different cameras as the main camera. For example, when zooming in (or out) from the default zoom ratio (1×) to a certain ratio, the shooting distance and brightness change accordingly, and the main camera can be switched from a wide-angle camera to a telephoto camera or an ultra-wide-angle camera. However, due to the different hardware parameters and shooting parameters of different cameras, it is easy to cause different imaging field of view (FOV) after switching the main camera. There is a slight jump in FOV at the moment of switching cameras, resulting in a poor user experience.

[0099] In order to solve the problem of FOV jump caused by switching cameras, in some embodiments, the image corresponding to the main camera before switching and the image corresponding to the main camera after switching can be processed simultaneously through a software architecture with multiple image processing channels, and then combined with the FOV picture transition algorithm (overlay algorithm), the two images are overlaid, so that when the main camera is switched, the original hard switching process is replaced with a smooth transition animation between the two camera images. The overlay processing of the image can make the image collected by the main camera before the switch gradually disappear, and make the image collected by the main camera after the switch gradually appear. This effect of fading and appearing can improve the presentation effect of the video picture at the user's visual level and enhance the user's viewing experience.

[0100] However, the process of superimposing the main image and the auxiliary image is usually performed after the main camera is actually switched (e.g., Figure 2A Considering that the point-cutting point at which the user performs the point-cutting operation (used to adjust the zoom ratio) is before the switching point, the overlay processing performed after the switching point will objectively increase the delay of the zoom field of view (zoom) switching, and prolong the response time of the terminal device to the point-cutting operation, resulting in a poor user experience.

[0101] In view of this, an embodiment of the present application provides a method for displaying an image, which predicts the main camera to be switched according to the shooting distance, brightness and zoom factor during the period from when the user performs a point-to-cut operation to when the main camera is actually switched, and performs a superimposition process on the images respectively captured by the two main cameras before and after the switching time point in advance (such as Figure 2B In other words, this method completes the superimposition processing of the two-way images of the new and old main cameras before the switching time point, rather than starting the superimposition processing after the switching time point, thereby achieving a smooth transition of the FOV while not affecting the picture response speed for the user to adjust the zoom ratio.

[0102] For example, Figure 4 As shown, it is a schematic structural diagram of a terminal device 100 provided in an embodiment of the present application.

[0103] In some embodiments, in some embodiments, the terminal device 100 can be various types of electronic devices with video shooting and image processing functions, such as: smart phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) or dedicated voice call devices, etc. The embodiments of the present application do not impose any restrictions on the specific type of the terminal device 100.

[0104] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0105] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0106] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0107] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0108] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0109] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0110] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0111] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0112] The terminal device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The display screen 194 is used to display images, videos, etc.

[0113] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0114] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy. The video codec is used to compress or decompress digital video. The NPU is a neural network (NN) computing processor that quickly processes input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0115] Exemplarily, the software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100. Figure 5 It is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0116] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime, system library, kernel layer, hardware abstraction layer (HAL), and hardware layer.

[0117] The application layer can include a series of application packages. Figure 5 As shown, the application package may include applications such as camera, calendar, map, WLAN, music, short message, gallery, call, navigation, Bluetooth, video, etc.

[0118] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 5 As shown, the application framework layer may include a window manager, a content provider, a phone manager, a resource manager, a notification manager, a view system, and the like.

[0119] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0120] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0121] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0122] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0123] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0124] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal vibrates, the indicator light flashes, etc.

[0125] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0126] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0127] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as obstacle life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0128] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0129] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0130] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0131] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0132] A 2D graphics engine is a drawing engine for 2D drawings.

[0133] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera sensor driver, audio driver, and sensor driver.

[0134] In order to better understand the image display method provided in the embodiment of the present application, the implementation of the method is introduced at the user visualization level in conjunction with the accompanying drawings.

[0135] For example, 6A to 6F FIG. 1 is a schematic diagram of a graphical user interface (GUI) that may be involved in the implementation of an image display method provided in an embodiment of the present application. For ease of description, the present application embodiment is introduced by taking a mobile phone as an example of a terminal device.

[0136] See also Fig. 6A , is a schematic diagram of a shooting interface provided in an embodiment of the present application. The shooting interface may include a preview box, an album icon, a shooting control, a camera rotation control, a zoom ratio adjustment control, and the like.

[0137] The preview box is used to obtain the preview image and display the preview image in real time. Fig. 6AThe rear view angle shown includes preview images of furniture and plants. The album icon is used to quickly enter the album. When the phone detects that the user clicks the album icon, the photos or videos that have been taken can be displayed on the display. The shooting control is used to perform shooting operations. When the phone detects that the user clicks the shooting control, it can shoot in the selected shooting mode. The shooting control is used to shoot or record. When in the shooting mode, the phone detects that the user clicks the shooting control, the phone performs the shooting operation and saves the shot photos; or when the phone is in the recording mode, the user clicks the shooting control, the phone performs the recording operation and saves the recorded video. The camera rotation control is used to control the switching of the front camera and the rear camera. The zoom ratio adjustment control is used to enlarge or reduce the zoom ratio when shooting. For example, when the phone detects that the user clicks another zoom ratio different from the current zoom ratio, the image can be enlarged or reduced according to the adjusted zoom ratio, wherein the image effect after enlargement or reduction can be presented to the user in the preview box.

[0138] In addition, the shooting interface also includes functional controls for setting the shooting mode, such as Fig. 6A The aperture shooting mode, night scene shooting mode, portrait shooting mode, photo shooting mode, video recording mode, professional mode and more modes are shown. Among them, more modes can also include slow motion mode, panoramic mode, black and white art mode, dual view video recording mode, filter mode, high dynamic range image (HDR) mode and multi-channel video recording mode (not shown in the figure), etc., which are not limited in this application.

[0139] In some embodiments, when the terminal device detects that the user clicks the icon corresponding to the video recording mode in the camera's shooting interface, the video recording mode (or video shooting mode) can be entered, and the default zoom ratio in the video recording mode can be 1×.

[0140] During video recording, Figure 6B As shown, the user can click the control corresponding to the 2× zoom ratio according to the desired video effect, that is, input the point-to-point operation 601. In response to the point-to-point operation 601 input by the user, the mobile phone can dynamically display the image magnification process to the user through the preview box. Figure 6C As shown in FIG. 1 , the image is shown after the zoom ratio is adjusted to 2×. The size of the photographed object (i.e., furniture and plants) in the image can be Fig. 6A or Figure 6B Twice the subject shown.

[0141] In order to facilitate understanding of the image superimposition effect involved in the embodiments of the present application, the following first introduces the scenario in which when the camera is switched, the images corresponding to the main camera before and after the switch are not superimposed, but the video image is directly switched after the main camera is switched.

[0142] In some embodiments, when the zoom ratio is enlarged to 2×, the main camera switches from the wide-angle camera to the telephoto camera. Since the hardware parameters and shooting parameters of the telephoto camera and the wide-angle camera are different, the images taken by the two cameras will also show different effects. Fig.6D As shown, after switching to the telephoto camera, if the preview frame directly displays the picture taken by the telephoto camera, the picture after the camera is switched will have a more obvious jump compared to the picture before the camera is switched, such as jumping directly from a dotted image to a solid image, wherein the dotted image represents the picture before the switch (that is, the picture taken by the wide-angle camera at 1 times), and the solid image represents the picture after the switch (that is, the picture taken by the telephoto camera at 2.5 times). The two frames of images include the same objects (both furniture and plants), but the FOV has a large jump. From the user's perspective, the position of the photographed object has changed significantly (such as the distance between the photographed objects in the two frames of images is different by D). In addition to the large change in the position of the photographed object, the pixel value (or brightness) of the two frames of images before and after the main camera is switched may also have a significant jump. In addition, the effects corresponding to other parameters of the two frames of images before and after the camera is switched may also have a significant jump, which is not limited in the embodiments of the present application.

[0143] The following combination Fig. 6E and Fig. 6F Then, when the camera is switched, a scene is introduced in which the images corresponding to the main camera before and after the switching are superimposed so that the image finally displayed after the superimposition processing has a gradual disappearance and appearance effect.

[0144] For example, Fig. 6E As shown, it is a schematic diagram of the shooting interface when the user has just adjusted the zoom ratio to 2x by inputting the point-cut operation 601, but the mobile phone has not yet switched to the main camera. The preview image displayed in the viewfinder in the shooting interface is still the image taken by the wide-angle camera. Afterwards, the camera switches from the wide-angle camera to the telephoto camera. At this time, the wide-angle camera as the auxiliary camera will continue to shoot and transmit images. At the same time, the telephoto camera as the switched main camera will also shoot and transmit images during this period. The mobile phone divides the images taken by the two cameras into two channels for simultaneous processing and superimposition processing, and then transmits the superimposed image to the display, and the display displays the superimposed final image according to the frame rate.

[0145] For example, Fig. 6FAs shown, the mobile phone can dynamically display the processed image in the preview box. For example, from the user's perspective, the photographed object at the new position (corresponding to the image screen corresponding to the new main camera) can be gradually revealed, and the photographed object at the initial position (corresponding to the image screen corresponding to the old main camera) can be gradually disappeared, wherein the photographed object can be gradually switched from the initial position to the new position, and the final distance between the new position and the initial position can be D.

[0146] It should be noted that in the embodiment of the present application, the superimposition process is performed before the mobile phone actually switches the main camera. Specifically, after the mobile phone receives the point-to-cut operation input by the user, it can predict which new main camera will be after the switch based on the end point zoom ratio (that is, the zoom ratio after the user adjusts it), the shooting distance and the brightness. If the predicted new main camera is inconsistent with the current main camera (which may correspond to the old main camera mentioned above), the mobile phone will need to perform the operation of switching the main camera later. Before switching the main camera, the mobile phone can obtain the image captured by the current main camera (old main camera) (recorded as the first image), and obtain the image captured by the predicted new main camera (recorded as the second image), and then select the N frames of the first image closest to the switching time point and the N frames of the second image closest to the switching time point for image post-processing at the same time, and perform superimposition processing on the first image and the second image after post-processing, and then transmit the superimposed image to the display screen for display.

[0147] According to the method for image display provided by the embodiment of the present application, before the main camera is actually switched, the image corresponding to the main camera before the switching, which is the preset number of frames closest to the switching point, and the image corresponding to the main camera after the switching, which is the preset number of frames closest to the switching point, are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed again, so that when different cameras are switched, the image picture with a smooth transition effect can be presented to the user in a timely manner by gradually disappearing the image picture of the main camera before the switching and gradually showing the image picture of the main camera after the switching. On the basis of eliminating the feeling of picture jump, the response point cutting operation time is shortened, thereby improving the user's viewing experience.

[0148] In order to better understand the image display method provided in the embodiment of the present application, the underlying implementation logic of the method is introduced below in combination with the structure of the terminal device.

[0149] For example, Figure 7As shown, it is a schematic diagram of the interaction of functional modules involved in the implementation of an image display method provided in an embodiment of the present application. The functional modules in the embodiment of the present application may include an Android application package (APK) module 702, a multi-camera control module 706, a smooth zoom module 707, a main image post-processing module 708, an auxiliary image post-processing module 709, an image processing engine 710, and a display module 711. In addition, the image display method provided in an embodiment of the present application may also involve a display screen 701 in a terminal device and multiple cameras (such as Figure 7 The ultra-wide-angle camera 703, wide-angle camera 704, telephoto camera 705, etc. are shown.

[0150] In some embodiments, the display screen 701 may be a touch-type display screen for receiving a point-switching operation input by a user on a terminal device (eg, corresponding to Figure 6B In addition, the display screen 701 can also be used to display an image, specifically, it can be used to display an image with a superimposed effect. For example, when receiving a point-cut operation, the display screen can present the image corresponding to the new main camera to the user in a gradually displayed form, and at the same time present the image corresponding to the original camera to the user in a gradually hidden form.

[0151] In some embodiments, the APK module can be used to obtain user operation information transmitted by the display screen, transmit the user operation information to the multi-camera control module 706, and send a request to the multi-camera control module 706 to determine whether it is a point-to-point operation.

[0152] In some embodiments, the ultra-wide-angle camera 703 may correspond to camera 1 in Table 1 above, the wide-angle camera 704 may correspond to camera 2 in Table 1 above, and the telephoto camera 705 may correspond to camera 3 in Table 1 above. In the embodiment of the present application, the ultra-wide-angle camera 703 may also be described as an ultra-wide-angle lens, the wide-angle camera 704 may also be described as a wide-angle lens, and the telephoto camera 705 may also be described as a telephoto lens, which may be used to shoot objects in the environment under different shooting parameters (such as zoom ratio, brightness, shooting distance, etc.) and obtain images including these objects.

[0153] In some embodiments, the multi-camera control module 706 can be used to receive user operation information transmitted by the APK module and a request message for requesting to determine whether it is a point-to-point operation; and can respond to the request message to determine whether the current user operation is a point-to-point operation based on the user operation information.

[0154] The multi-camera control module 706 can also be used to predict whether the main camera needs to be switched according to the multi-camera strategy. The multi-camera strategy may, for example, include determining the main camera based on the corresponding relationship between the shooting parameters (including zoom ratio, brightness, and shooting distance) and the main camera. Specifically, when it is determined that the user operation is a point-to-point operation (the main camera has not been switched at this time), the multi-camera control module 706 can obtain a new main camera corresponding to these shooting parameters according to the end point zoom ratio adjusted by the user, the brightness (or shooting brightness) perceived by the current main camera (corresponding to the original main camera), and the current shooting distance; if the new main camera is consistent with the original main camera, then image processing and image display are performed according to the original logic; if the new main camera is inconsistent with the original main camera, then the new main camera is the main camera after the shooting parameters are adjusted predicted by the multi-camera control module 706.

[0155] The multi-camera control module 706 can also be used to keep the number of images transmitted by the main channel and the auxiliary channel at least to meet the needs of the overlay algorithm. Specifically, the multi-camera control module 706 can be used to control the on or off of the overlay switch. The multi-camera control module 706 can also determine whether overlay processing is required based on whether the predicted new main camera is consistent with the original main camera. If the new main camera is inconsistent with the original main camera, it means that the main camera needs to be switched later, which also means that before switching the main camera, the two-channel image needs to be overlaid. When overlay processing is required, the multi-camera control module 706 can also be used to obtain which frames of images need to be overlaid according to the main camera switching information. When the dual-channel output does not meet the number of overlay frames, the dual-channel output is continuously maintained and the overlay switch is set to the on state.

[0156] Specifically, after the multi-camera control module 706 acquires a new main camera according to the multi-camera strategy prediction, it can instruct the new main camera to transmit the image data it has collected through the main line, and copy the image data of the main camera to the port corresponding to the new main camera, so that the smooth zoom module 707 can read the corresponding image data from the port corresponding to the new main camera. At the same time, the original main camera can continue to transmit the image data it has collected, and copy the image data of the original main camera to the port corresponding to the original main camera through the auxiliary line, so that the smooth zoom module 707 can read the corresponding image data from the port corresponding to the original main camera. Here, the port corresponding to the new main camera and the port corresponding to the original main camera can refer to the port between the multi-camera control module 706 and the smooth zoom module 707.

[0157] The multi-camera control module 706 can also be used to determine whether the number of image frames transmitted by the original main camera and the new main camera reaches the preset number of frames after the point-cut operation. If the number of image frames transmitted by the main camera and the auxiliary camera reaches the preset number of frames, the multi-camera control module 704 can control the overlay switch to be turned off; if the number of image frames transmitted by the main camera and the auxiliary camera does not reach the preset number of frames, the multi-camera control module 706 can control the overlay switch to be turned on. The preset number of frames here can refer to the corresponding number of image frames during the period from the point-cut point to the switching point. For example, the point-cut operation input by the user is used to adjust the zoom ratio from 1× to 2×. In this scenario, the duration from the point-cut point to the switching point corresponds to the duration of 10 frames of images, so the preset number of frames can be 10.

[0158] In addition, the multi-camera control module 706 can also determine which frame in the preset number of frames to start the superimposition process. Specifically, the multi-camera control module 706 can control the superimposition process to be performed on the N frames of images closest to the switching time point. For example, assuming that the preset number of frames is 10 frames and N is 3, the multi-camera control module 706 can determine to start the superimposition process from the 8th frame in the preset number of frames, that is, to perform the superimposition process on the dual-channel images of the 8th frame, the 9th frame, and the 10th frame.

[0159] In some embodiments, the smooth zoom module 707 can be used to calculate the cropping information and / or mapping information corresponding to the main image and the auxiliary image of the output lens at the current focal length, so that the main image post-processing module can process the main image and the auxiliary image post-processing module can process the auxiliary image. In addition, the smooth zoom module 707 can also be used to determine which images are auxiliary images according to the image data read from which port, and then copy the auxiliary images to the auxiliary output port, and output the calculated information required for post-processing of the auxiliary images through the auxiliary output port. The smooth zoom module 707 can also be used to determine which images are main images according to the image data read from which port, and then copy the main images to the main output port, and output the calculated information required for post-processing of the main images through the main output port. The main output port and the auxiliary output port here can refer to the ports between the smooth zoom module and the main image post-processor 706 and the auxiliary image post-processor 707, respectively.

[0160] In order to ensure that no additional power consumption is added, the smooth zoom module 707 can also determine whether to copy the auxiliary channel image to the auxiliary channel output port according to the dynamic state of the overlay switch. When the overlay switch is on, the smooth zoom module 707 can copy the auxiliary channel image and its corresponding processing information to the auxiliary channel output port, so that the auxiliary channel image post-processing module can read the auxiliary channel image and its corresponding processing information from the auxiliary channel port; at the same time, the main channel image and its corresponding processing information can be copied to the main channel output port, so that the main channel image post-processing module can read the main channel image and its corresponding processing information from the main channel port. When the overlay switch is off, the smooth zoom module 707 no longer copies the auxiliary channel image and its corresponding processing information to the auxiliary channel output port, but only copies the main channel image and its corresponding processing information to the main channel output port, so that the main channel image post-processing module can read the main channel image and its corresponding processing information from the main channel port.

[0161] In some embodiments, the main image post-processing module 708 can be used to read the main image and its processing information from the main output port between it and the smooth zoom module 707, and then perform image post-processing on the main image according to the main image and its processing information. The image post-processing includes, for example, image cropping, image feature mapping, etc., which is not limited in the embodiment of the present application.

[0162] In some embodiments, the auxiliary channel image post-processing module 709 can be used to read the auxiliary channel image and its processing information from the auxiliary channel output port between it and the smooth zoom module 707, and then perform image post-processing on the auxiliary channel image according to the auxiliary channel image and its processing information. The image post-processing includes, for example, image cropping, image feature mapping, etc., which is not limited in the embodiments of the present application.

[0163] In some embodiments, the image processing engine 710 may be used to perform superimposition processing on the processed images corresponding to the two cameras according to the superimposition algorithm, and may also perform other processing on the images according to other algorithms, and then transmit the processed images with superimposition effect to the display module 711. In a possible implementation, the image processing engine 710 may be specifically presented as a graphics processing unit (GPU).

[0164] In some embodiments, the display module 711, which can also be described as a display module, can be used to send the image data (multiple pixel point data) rendered by a processor such as a GPU to a display device. The display device here can refer to the display screen 701 or a mobile phone screen, for example.

[0165] For example, Figure 8 FIG. 1 is a schematic flow chart of an image display method provided in an embodiment of the present application. The flow chart may specifically include the following steps:

[0166] S801, obtaining user operation.

[0167] The user operation may be an operation of adjusting a zoom ratio, and specifically may be a point-to-point operation.

[0168] S802: Determine whether the user operation is a point-to-cut operation.

[0169] In some embodiments, the method of determining whether the user operation is a point-to-point operation may include, for example, obtaining the characteristics of the current user operation; matching the characteristics of the user operation with the characteristics of the preset point-to-point operation; if the characteristics of the user operation are consistent with the characteristics of the preset point-to-point operation, determining that the user operation is a point-to-point operation. The characteristics of the user operation may include, for example, the input method of the user operation, such as click, long press, voice input, etc.; the button targeted by the user operation, such as whether it is a zoom ratio adjustment button, etc.

[0170] If the user operation is a point-to-point operation, that is, the judgment result of this step is "yes", then step S803 can be executed next; if the user operation is not a point-to-point operation, that is, the judgment result of this step is "no", then step S807 can be executed next, that is, image processing and image display are performed according to the original logic. The original logic here refers to the image processing logic and image display logic before the point-to-point operation, that is, the logic of image processing and image display for the image data corresponding to the original main camera.

[0171] S803, obtaining the end point zoom ratio, brightness and shooting distance at which the point-to-point operation ends.

[0172] In some embodiments, when it is determined that the user operation is a point-to-point operation, the terminal device can obtain the end point zoom ratio where the point-to-point operation ends. For example, if the user adjusts the zoom ratio from 1× to 2× through the point-to-point operation, the end point zoom ratio is 2×.

[0173] In some embodiments, the terminal device may also obtain the brightness and shooting distance after the zoom ratio is adjusted based on the image captured by the current main camera (corresponding to the original main camera) after the point-cut operation. The brightness here may refer to the brightness perceived by the current main camera after the zoom ratio is adjusted.

[0174] In some embodiments, after the terminal device receives the point-cut operation, it can perform brightness and shooting distance detection on the first frame image after the point-cut point (the image currently displayed by the main camera), and store the detected brightness result L0 and shooting distance D0.

[0175] S804, determining whether the main camera corresponding to the new shooting parameters is consistent with the original main camera.

[0176] In some embodiments, the terminal device may store shooting parameters before the point of intersection, including: zoom ratio, brightness, and shooting distance.

[0177] In some embodiments, the terminal device determines whether the main camera corresponding to the new shooting parameters is consistent with the original main camera in the following manner: the terminal device determines the new main camera corresponding to the new shooting parameters according to the acquired new shooting parameters; and compares whether the new main camera is consistent with the current main camera. There is a corresponding relationship between the main camera and the shooting parameters (including zoom ratio, brightness, and shooting distance), and different shooting parameters (or different shooting parameter ranges) can correspond to different main cameras.

[0178] In some embodiments, if the new main camera is inconsistent with the original main camera, the original main camera and the new main camera are instructed to simultaneously transmit their respective captured images at least before the switching point, that is, to obtain two channels of images.

[0179] Among them, if the main camera corresponding to the new shooting parameters is consistent with the original main camera, that is, the judgment result of this step is "yes", then step S805 can be executed next; if the main camera corresponding to the new shooting parameters is inconsistent with the original main camera, that is, the judgment result of this step is "no", then step S806 can be executed next.

[0180] S805, start the overlay process, obtain two channels of image data, and synchronize image parameters.

[0181] In some embodiments, the terminal device may perform an overlay process on the N frames of two-way images closest to the switching point. For example, during the period from the point cut point to the switching point, the corresponding number of frames of the transmitted image is 10 frames, then the images corresponding to the new main camera and the images corresponding to the original main camera in the last 3 frames may be overlaid. The specific implementation process of the overlay process will be described in detail below and will not be repeated here.

[0182] The two channels of image data here may include image data corresponding to the new main camera and image data corresponding to the original main camera. The image data corresponding to the new main camera may be transmitted through the main channel of the dual-channel image processing architecture, and the image data corresponding to the original main camera may be transmitted through the auxiliary channel of the dual-channel image processing architecture, but the embodiments of the present application are not limited to this. Alternatively, the image data corresponding to the new main camera may be transmitted through the auxiliary channel of the dual-channel image processing architecture, and the image data corresponding to the original main camera may be transmitted through the main channel of the dual-channel image processing architecture, and after the main camera is actually switched, the image data of the new main camera is transmitted through the main channel.

[0183] The synchronous image parameters here may refer to 3A synchronization of the image, that is, the color, brightness and size of the synchronous image.

[0184] S806, performing image processing and image display according to the original logic.

[0185] The original logic here refers to the image processing logic and image display logic before the point cutting operation, that is, the logic of performing image processing and image display on the image data corresponding to the original main camera.

[0186] According to the image display method provided by the embodiment of the present application, the main camera after switching is predicted according to the shooting distance, brightness and zoom ratio during the period from when the user performs the point-to-point operation to when the main camera actually switches, and the images respectively collected by the two main cameras before and after the switching are superimposed before the switching time point, thereby shortening the time between the point-to-point time point and the presentation of the final image. In other words, the method completes the superimposition processing of the two-way images of the new and old main cameras before the switching time point, rather than starting the superimposition processing after the switching time point, thereby achieving a smooth transition of the FOV while not affecting the image response speed to the user adjusting the zoom ratio.

[0187] It should be noted that the overlay processing in the image display method provided in the embodiment of the present application is performed before the switching point. If the shooting parameters do not change from the point-cut point to the switching point, the overlay processing can be performed according to the above method. Figure 7 However, in actual applications, sometimes the user continuously inputs zoom ratio adjustment operations in a short period of time, or the shooting distance and / or brightness changes significantly in a short period of time, then the corresponding main camera may change for the second time during the period from the point cut point to the switching point. At this time, if the two-way image corresponding to the original main camera and the first predicted new main camera is continuously superimposed, the picture change cannot be accurately presented, so an interrupt mechanism needs to be executed.

[0188] The interruption mechanism specifically refers to: if the predicted new main camera changes during the period from the point cut point to the switching point, the current overlay processing process for the previously predicted new main camera image will be terminated. For example, if the user adjusts the zoom ratio from 1× to 2× at the point cut point, and then quickly adjusts the zoom ratio from 2× to 1× before the switching point, the main camera may change as follows: the original main camera is a wide-angle camera, the new main camera predicted for the first time is a telephoto camera, and the new main camera predicted for the second time is a wide-angle camera. Among them, the telephoto camera predicted for the first time is the new main camera obtained based on the zoom ratio of 2× and its corresponding brightness and shooting distance; the wide-angle camera predicted for the second time is the new main camera obtained based on the zoom ratio of 1× and its corresponding brightness and shooting distance.

[0189] If the second predicted time of acquiring the new main camera occurs during the overlay process that has already started (i.e., the overlay process of the two images corresponding to the wide-angle camera and the first predicted telephoto camera), then the overlay process can be interrupted. Afterwards, the image processing logic can be re-determined based on the new main camera predicted for the second time. For example, in this example, since the second predicted main camera is the same as the original main camera, both are wide-angle cameras, then after interrupting the overlay process, image processing can be performed according to the original image processing logic, that is, image processing is still performed only on one image corresponding to the wide-angle camera as the main camera.

[0190] The following describes the specific process of executing the interrupt mechanism in the image display method provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0191] For example, Fig. 9 , which is a schematic flow chart of another image display method provided in an embodiment of the present application.

[0192] It should be noted that Fig. 9 The process in the embodiment can be performed after the point-cut operation and before the main camera is switched, that is, it is performed during the point-cut point and the switching point. And the scenario corresponding to the process is: after the terminal device receives the point-cut operation input by the user at the point-cut point, it has performed the first main camera prediction based on the adjusted zoom ratio and its corresponding shooting distance and brightness, and the new main camera predicted for the first time is different from the original main camera; then, the terminal device performs the above Figure 7 The process shown in the embodiment performs the overlay processing on the images of the new main camera and the original main camera.

[0193] S901, detecting whether the shooting distance and brightness have changed.

[0194] In some embodiments, after the point-cut operation, the terminal device can detect and store the shooting distance D0 and brightness L0 corresponding to the first frame image after the point-cut point, and detect the shooting distance and brightness corresponding to each frame image thereafter, and compare the newly acquired shooting distance and brightness with the stored shooting distance D0 and brightness L0; if the newly acquired shooting distance and brightness have not changed compared to the shooting distance D0 and brightness L0, there is no need to store the newly acquired shooting distance and brightness; if the newly acquired shooting distance and / or brightness have changed compared to the shooting distance D0 and brightness L0, the stored initial shooting distance and / or initial brightness is updated. Among them, the shooting distance D0 and brightness L0 are the shooting distance and brightness corresponding to the zoom ratio adjusted by the terminal device in response to the point-cut operation.

[0195] If the shooting distance and brightness have changed, that is, the judgment result of this step is "yes", then step S902 can be executed next; if the shooting distance and brightness have not changed, that is, the judgment result of this step is "no", then step S906 can be executed next, that is, image processing and image display are performed according to the original logic. The original logic here refers to the image processing logic and image display logic before the point-cut operation, that is, the logic of image processing and image display for the image data corresponding to the original main camera.

[0196] S902, match the new main camera according to the current shooting distance and brightness.

[0197] Here, matching the new main camera according to the current shooting distance and brightness can also be understood as the second prediction of the new main camera. In order to distinguish it from the new main camera obtained by the second prediction, the new main camera obtained by the first prediction can be described as the original main camera.

[0198] In some embodiments, if the currently acquired shooting distance and / or brightness changes, a new main camera can be matched according to the newly acquired current shooting distance and brightness. Specifically, the terminal device can obtain the main camera corresponding to the current zoom ratio, the current shooting distance and the current brightness as the new main camera according to the correspondence between the zoom ratio, the shooting distance and the brightness and the main camera.

[0199] S903, determining whether the main camera corresponding to the new shooting parameters is consistent with the original main camera.

[0200] The new shooting parameters here may include the current zoom ratio, the current shooting distance and the current brightness. The main camera corresponding to the new shooting parameters here may correspond to the new main camera.

[0201] If the new main camera is consistent with the original main camera, that is, the judgment result of this step is "yes", then step S904 can be executed next; if the new main camera is inconsistent with the original main camera, that is, the judgment result of this step is "no", then step S905 can be executed next.

[0202] S904, continue the overlay processing process.

[0203] It can be understood that if the new main camera is consistent with the original main camera, that is, the main camera predicted for the second time is consistent with the main camera predicted for the first time, then it means that the subsequent main camera is still the main camera obtained by the first prediction, and the main camera does not need to be switched, so the existing overlay processing process can be continued.

[0204] In other words, after the new main camera is predicted and obtained for the first time, and the superimposition process is performed on the image of the new main camera predicted and obtained for the first time, although the shooting parameters such as shooting distance and brightness have changed, the main camera corresponding to the changed new shooting parameters is still the new main camera predicted and obtained for the first time, and the main camera does not need to be switched subsequently. Therefore, in this case, the existing superimposition process can be continued.

[0205] S905, interrupting the current overlay processing process.

[0206] It is understandable that if the new main camera is inconsistent with the original main camera, that is, the main camera predicted for the second time is inconsistent with the main camera predicted for the first time, then it means that the subsequent main camera needs to be switched from the main camera obtained by the first prediction to the main camera obtained by the second prediction. Therefore, the current overlay processing process for the image of the main camera obtained by the first prediction does not need to be performed again.

[0207] For example, suppose that after starting to shoot, the initial zoom ratio is 1×, and the main camera is a wide-angle camera. During the shooting process, the terminal device receives a point-to-cut operation input by the user and switches the zoom ratio from 1× to 2×. As the zoom ratio changes, the shooting distance and brightness may change accordingly. The shooting distance after this change is recorded as D0, and the brightness after this change is recorded as L0. Afterwards, the terminal device needs to predict the new main camera for the first time based on the changed shooting parameters (zoom ratio 2×, shooting distance D0 and brightness L0). If the new main camera predicted by the terminal device for the first time is a telephoto camera, then the terminal device can then overlay the image captured by the wide-angle camera and the image captured by the telephoto camera. During the superimposition process, if the shooting distance and brightness change again (assuming that the zoom ratio does not change at this time, but not limited to this), and the shooting distance after the change is D1, and the brightness after the change is L1, then after detecting the shooting distance and brightness changes, the terminal device can predict the new main camera for the second time according to the latest shooting parameters (zoom ratio 2×, shooting distance D1 and brightness L1). If the new main camera predicted for the second time is a wide-angle camera, since it is different from the telephoto camera predicted for the first time, the main camera may not be actually switched to the telephoto camera predicted for the first time. Therefore, the superimposition process for the images captured by the wide-angle camera and the images captured by the telephoto camera can be terminated. If the new main camera predicted for the second time is a telephoto camera, since it is the same as the telephoto camera predicted for the first time, the main camera may continue to be actually switched to the telephoto camera predicted for the first time. Therefore, the superimposition process for the images captured by the wide-angle camera and the images captured by the telephoto camera can continue according to the original logic.

[0208] After the first prediction obtains the matching expected main camera, there are generally several frames or dozens of frames (such as 6-10 frames) before the end of the superposition. During this period, if the ambient brightness changes or the shooting distance changes, the expected main camera predicted for the first time may change. Therefore, the embodiment of the present application uses an interrupt protection mechanism. Before the superposition is performed (generally the first few frames of the termination magnification, the termination magnification can correspond to the zoom magnification at the switching point), if the shooting distance and / or brightness change is detected, the new main camera is re-matched. If the re-predicted new main camera changes compared to the main camera predicted for the first time, the superposition process is interrupted. In addition, in a fast point-to-point scenario, such as quickly switching from 1x to 2.5x and then from 2.5x to 1x, the camera may not reach the termination magnification before starting the next point-to-point process. Therefore, if it is determined that the zoom magnification changes rapidly from the point-to-point to the switching point, and the changed zoom magnification corresponds to a different main camera, then this situation does not meet the requirements. Figure 7 The superimposition logic shown in the embodiment can also interrupt the superimposition process that has been carried out.

[0209] According to the method for image display provided by the embodiment of the present application, by terminating the overlay processing process when the predicted main camera changes again during the overlay processing of the image captured by the predicted new main camera and the changed main camera is different from the previously predicted main camera, the image processing logic can be determined in time according to the main camera obtained by the latest prediction when the predicted main camera changes, thereby improving the accuracy of the overlay processing and the accuracy of the image presented to the user.

[0210] In order to better understand the image display method provided in the embodiment of the present application, the principle of overlay processing is introduced below.

[0211] In some embodiments, when the original main camera is the first camera, after the cut-point operation, the predicted new main camera is the second camera, and the first camera is different from the second camera, the multi-camera control module can determine the frame number after the cut-point to start the superimposition process according to the adjustment range of the zoom ratio, or determine the frame number before the switching point to start the superimposition process. Taking the M-th frame image after the cut-point as an example, before reaching the M-th frame image, the multi-camera control module can instruct the superimposition switch to be turned on. After the superimposition switch is turned on, the auxiliary image processing path will be connected. At this time, the image corresponding to the new main camera and the image corresponding to the original main camera can be processed simultaneously through the main path and the auxiliary path (that is, the dual path). Afterwards, the multi-camera control module controls the overlay switch to be turned on or off according to whether the number of images corresponding to the original main camera and the number of images corresponding to the new main camera currently obtained have reached the preset N frames, wherein: if the number of images corresponding to the original main camera and the number of images corresponding to the new main camera currently obtained have both reached N frames, the multi-camera control module can instruct the overlay switch to be turned off, and subsequently, the multi-camera control module can only receive the images sent by the new main camera, and transmit and process the images corresponding to the new main camera via the main channel, that is, restore to the main channel image processing strategy; if the number of auxiliary channel images and / or the number of main channel images currently obtained does not reach N frames, the multi-camera control module can instruct the overlay switch to be turned on, and subsequently, the multi-camera control module will continue to receive the images sent by the two main cameras before and after the switch (that is, the new main camera and the original main camera), respectively, and simultaneously transmit and process the images corresponding to the new main camera and the images corresponding to the original main camera via the main channel and the auxiliary channel, respectively, that is, continue to adopt the main channel and auxiliary channel image processing strategies.

[0212] If the number of images corresponding to the original main camera and the number of images corresponding to the new main camera currently acquired by the multi-camera control module do not reach N frames, it means that the image processing engine has not acquired enough main images and auxiliary images to perform a complete superposition process. At this time, the image processing engine may have performed superposition processing on some main images and corresponding auxiliary images, but due to insufficient number, there may still be a screen jump phenomenon when the image is displayed. Therefore, when the number of images corresponding to the original main camera and the number of images corresponding to the new main camera currently acquired by the multi-camera control module do not reach the preset number of frames, in order to avoid the above-mentioned problem, the multi-camera control module needs to indicate that the current frame image is still transmitted and processed according to the main and auxiliary image processing strategies, and the superposition switch is set to the on state. Based on the main and auxiliary image processing strategies indicated by the multi-camera control module, the current frame image (including the current frame image transmitted by the new main camera and the current frame image transmitted by the original main camera at this time) can be subsequently divided into transmission and processing via the main path and transmission and processing via the auxiliary path, and superposition processing is performed on the basis of image post-processing.

[0213] If the number of images corresponding to the original main camera and the number of images corresponding to the new main camera currently acquired by the multi-camera control module reaches N frames, it means that the image processing engine can also obtain enough images corresponding to the new main camera and the images corresponding to the original main camera for overlay processing. At this time, the terminal device does not need to transmit and process more images corresponding to the original main camera, but can perform image processing according to the original main channel image processing strategy. The above method can avoid consuming additional resources and avoid the overlay processing in the embodiment of the present application causing changes to the original image processing strategy.

[0214] In some embodiments, the smooth zoom module can read the image transmitted by the current original main camera through the auxiliary channel from the memory, and read the image transmitted by the current new main camera through the main channel. After that, the smooth zoom module reads the first image corresponding to the original main camera from the port corresponding to the original main camera, and reads the second image corresponding to the new main camera from the port corresponding to the new main camera. After that, the smooth zoom module can respectively calculate the processing information corresponding to the first image, and calculate the processing information corresponding to the second image, and perform 3A synchronization on the first image and the second image. The processing information here may include, for example, cropping information, color information, brightness information, and mapping information, wherein the cropping information is used to unify the images corresponding to the two cameras into the same size, which may be a preset size; the mapping information is used to match according to the feature points in the corresponding images, and obtain the images corresponding to the two cameras at the same or similar viewing angles, so that the image content captured by the two cameras on the mobile phone is as consistent as possible, that is, the corresponding images captured by the two cameras can contain the same or similar photographed objects. Among them, the way in which the smooth zoom module calculates and obtains the image processing information can refer to the existing method, and the embodiment of the present application will not be described in detail.

[0215] In addition, an auxiliary output port may be provided between the smooth zoom module and the auxiliary image post-processing, and a main output port may be provided between the smooth zoom module and the main image post-processing. In some embodiments, after the smooth zoom module obtains the processing information corresponding to the first image and the processing information corresponding to the second image, the first image and its corresponding processing information may be copied to the auxiliary output port, and the second image and its corresponding image processing information may be copied to the main output port.

[0216] In some embodiments, the auxiliary image postprocessor can read the first image and its corresponding processing information from the auxiliary output port, and the main image postprocessor can read the second image and its corresponding processing information from the main output port. Afterwards, the auxiliary image postprocessor can process the first image according to the processing information corresponding to the first image, obtain the processed auxiliary image, and then transmit the auxiliary image to the image processing engine; at the same time, the main image postprocessor can also process the second image according to the processing information corresponding to the second image, obtain the processed main image, and then transmit the main image to the image processing engine.

[0217] In some embodiments, the image processing engine can query from the memory whether the overlay switch is in the on state. Based on the overlay switch being in the on state, the image processing engine can perform overlay processing on the acquired main image and auxiliary image according to the overlay algorithm. The overlay processing can be performed in real time, that is, each time the image processing engine acquires a frame of processed main image and a corresponding processed auxiliary image, it will perform overlay processing on the two frames of images in real time. The main image and the corresponding auxiliary image that are simultaneously overlaid can be images that are simultaneously captured, transmitted, and post-processed by the main camera and the auxiliary camera, respectively.

[0218] In a possible implementation, the on state of the superimposition switch can be set as a preset information, and the preset information is stored in the memory. When the superimposition switch is turned on, the multi-camera control module can set the preset information to indicate that the superimposition switch is in the on state; when the superimposition switch is turned off, the multi-camera control module can set the preset information to indicate that the superimposition switch is in the off state. Optionally, the on state of the superimposition switch can also be represented by an indicator bit, and the information in the indicator bit can be used to indicate whether the image processing engine uses the superimposition algorithm to perform superimposition processing on the image; or, optionally, the superimposition switch can also be regarded as an algorithm calling function, and the algorithm calling function is used by the image processing engine to call the superimposition algorithm to perform superimposition processing on the image. Optionally, when the superimposition switch is an indicator bit or an algorithm calling function, the multi-camera control module can send it to the image processing engine when the first camera is switched to the second camera, instructing the image processing engine to process the image according to the superimposition algorithm.

[0219] In some embodiments, the overlay processing may specifically include: when the overlay switch is turned on, the image processing engine reduces the pixel value of the auxiliary image corresponding to the auxiliary camera frame by frame from the current pixel value according to a preset first weight, until the pixel value of the auxiliary image is reduced to 0 when the overlay frame number is reached; at the same time, according to the preset second weight, the pixel value of the main image corresponding to the main camera is increased frame by frame from 0 until the pixel value of the main image is increased to a preset pixel value when the overlay frame number is reached, wherein the preset pixel value may be the same as the pixel value of the main image displayed on the display screen when no point cutting operation is performed.

[0220] For example, it is assumed that the frame corresponding to the start of the superposition is the first frame, the number of superposition frames is Q, and Q is an integer greater than 1. Then, when the pixel value of the auxiliary image is reduced in a linear manner, the change of the pixel value of the auxiliary image with time (or frame number) can be seen from Fig. 10A Specifically, when the overlay process starts, from the 1st frame to the Qth frame, the terminal device can reduce the pixel value of the auxiliary image frame by frame according to the first weight, wherein the first weight can be linearly reduced with time (or the number of frames), in which case the pixel value of the auxiliary image also changes linearly with time (or the number of frames).

[0221] When the pixel value of the main image is increased linearly, the change of the pixel value of the main image over time (or frame number) can be seen from Fig. 10B Specifically, when the overlay process starts, from the 1st frame to the Qth frame, the terminal device can increase the pixel value of the main image frame by frame according to the second weight, wherein the second weight can increase linearly with time (or the number of frames), in which case the pixel value of the main image also changes linearly with time (or the number of frames).

[0222] It should be noted that Fig. 10A and Fig. 10B The embodiment is only introduced by taking the example that the pixel values ​​of the auxiliary road image and the main road image change linearly with time. In actual applications, the pixel values ​​of the auxiliary road image and the main road image may also change nonlinearly with time, and the embodiment of the present application does not limit this.

[0223] It should also be noted that the embodiment of the present application only uses the pixel value as an example to illustrate the parameter adjusted in the overlay process, but in actual applications, the parameters adjusted in the overlay process can also include other parameters such as color, brightness, etc., and the embodiment of the present application does not limit this.

[0224] According to the image display method provided in the embodiment of the present application, when the camera is switched, the image corresponding to the camera before the switching of a preset number of frames and the image corresponding to the camera after the switching of a preset number of frames are obtained, and the dual-channel images corresponding to the two cameras are respectively post-processed through a dual-channel processing path, and finally the processed images are superimposed, so that when different cameras are switched, the captured pictures can be smoothly transitioned, the picture jump feeling is eliminated, and the user's viewing experience is improved.

[0225] For example, Fig.11 FIG. 1 is a schematic flow chart of another method for displaying an image provided by an embodiment of the present application. Specifically, the method may include the following steps:

[0226] S1101, receiving a user's adjustment operation for a zoom ratio.

[0227] S1102: In response to the adjustment operation, the image displayed on the shooting interface of the terminal device is enlarged or reduced as the zoom ratio changes.

[0228] S1103, before switching the main camera from the first camera to the second camera, the first pixel value corresponding to the first picture included in the image currently displayed on the shooting interface gradually decreases, and the second pixel value corresponding to the second picture included in the image currently displayed gradually increases.

[0229] In some embodiments, the first picture is an image picture corresponding to the first camera, and the second picture is an image picture corresponding to the second camera.

[0230] In some embodiments, the method also includes: in response to the adjustment operation, adjusting the first zoom ratio to a second zoom ratio, wherein the first zoom ratio corresponds to a first shooting distance and a first shooting brightness, the second zoom ratio corresponds to a second shooting distance and a second shooting brightness, the first shooting distance is the same as or different from the second shooting distance, the first shooting brightness is the same as or different from the second shooting brightness, the main camera corresponding to the first zoom ratio, the first shooting distance and the first shooting brightness is the first camera, and the main camera corresponding to the second zoom ratio, the second shooting distance and the second shooting brightness is the first camera; according to the second zoom ratio, the second shooting distance and the second shooting brightness, the new main camera is obtained as the second camera.

[0231] In some embodiments, the method also includes: in response to the second camera being used as the new main camera, starting from the N frames of images closest to the switching point, superimposing the first image corresponding to the first camera and the second image corresponding to the second camera, wherein the switching point is the time point when the main camera is switched from the first camera to the second camera, and N is an integer greater than 1.

[0232] In some embodiments, the method also includes: when the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera, and the second picture is a picture in the N frames of images taken by the second camera, and the preset pixel value is the pixel value of the picture displayed before the adjustment operation.

[0233] In some embodiments, the method further includes: the dissolve process ends before the switching point.

[0234] In some embodiments, the method also includes: before switching the main camera from the first camera to the second camera, detecting that the second shooting distance becomes a third shooting distance, and / or the second shooting brightness becomes the third shooting brightness; according to the third shooting distance and / or the third shooting brightness, obtaining the new main camera as the third camera; if the third camera is different from the second camera, terminating the overlay processing process for the first image and the second image; if the third camera is the same as the second camera, continuing the overlay processing process for the first image and the second image.

[0235] In some embodiments, the method also includes: before switching the main camera from the first camera to the second camera, detecting that the second zoom ratio is adjusted to a third zoom ratio; according to the third zoom ratio, acquiring the new main camera as a fourth camera; if the fourth camera is different from the second camera, terminating the overlay processing process for the first image and the second image; if the fourth camera is the same as the second camera, continuing the overlay processing process for the first image and the second image.

[0236] In some embodiments, the adjustment operation is a click operation on a zoom ratio adjustment control.

[0237] In some embodiments, when the N frames of images closest to the switching point are acquired, an overlay switch is turned on, and the overlay switch is used to turn on or off the auxiliary path.

[0238] In some embodiments, the method further includes: obtaining first cropping information of the first image, the first cropping information being used to crop the size of the first image to a preset size; and obtaining second cropping information of the second image, the second cropping information being used to crop the size of the second image to a preset size.

[0239] In some embodiments, the method further includes: cropping the second image according to the second cropping information, and acquiring the cropped main road image.

[0240] In some embodiments, the method further includes: reading whether the superimposition switch is in an on state; when the superimposition switch is in an on state, cropping the first image according to the first cropping information, and acquiring the cropped auxiliary path image.

[0241] In some embodiments, the method further includes: performing overlay processing on the auxiliary image and the corresponding main image to obtain a display image after the overlay processing, wherein the overlay processing is used to gradually reduce the pixel value of the picture corresponding to the auxiliary image and gradually increase the pixel value of the picture corresponding to the main image in the fused image corresponding to the auxiliary image and the corresponding main image; and displaying the display image.

[0242] In some embodiments, the method further includes: when the first picture disappears, turning off the dissolve switch.

[0243] In some embodiments, when the first picture disappears, the overlay switch is turned off, specifically including: when the number of frames of the auxiliary image reaches the N frames and the first picture disappears, the overlay switch is turned off; and / or, when the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

[0244] According to the image display method provided by the embodiment of the present application, the main camera after switching is predicted according to the shooting distance, brightness and zoom ratio during the period from when the user performs the point-to-point operation to when the main camera actually switches, and the images respectively collected by the two main cameras before and after the switching are superimposed before the switching time point, thereby shortening the time between the point-to-point time point and the presentation of the final image. In other words, the method completes the superimposition processing of the two-way images of the new and old main cameras before the switching time point, rather than starting the superimposition processing after the switching time point, thereby achieving a smooth transition of the FOV while not affecting the image response speed to the user adjusting the zoom ratio.

[0245] Based on the same technical concept, an embodiment of the present application also provides a terminal device, including a processor; a memory; the memory stores a computer program, and the computer program includes instructions. When the instructions are executed by the processor, the terminal device executes one or more steps of any of the above methods.

[0246] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable program instructions. When the computer-executable program instructions are executed on a computer, the computer or processor executes one or more steps in any of the above methods.

[0247] Based on the same technical concept, an embodiment of the present application also provides a computer program product comprising instructions, wherein the computer program product includes computer program code. When the computer program code runs on a computer, the computer or processor executes one or more steps in any of the above methods.

[0248] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0249] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0250] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A method for image display, characterized in that: Applied to a terminal device, the terminal device includes at least a first camera and a second camera, and the method includes: Receive the user's adjustment operation on the zoom ratio; In response to the adjustment operation, the image displayed on the shooting interface of the terminal device is enlarged or reduced as the zoom ratio changes; Before the main camera is switched from the first camera to the second camera, the first pixel value corresponding to the first picture included in the image currently displayed on the shooting interface gradually decreases, and the second pixel value corresponding to the second picture included in the currently displayed image gradually increases.

2. The method according to claim 1, characterized in that The first picture is an image picture corresponding to the first camera, and the second picture is an image picture corresponding to the second camera.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to the adjustment operation, the first zoom ratio is adjusted to a second zoom ratio, wherein the first zoom ratio corresponds to a first shooting distance and a first shooting brightness, the second zoom ratio corresponds to a second shooting distance and a second shooting brightness, the first shooting distance is the same as or different from the second shooting distance, the first shooting brightness is the same as or different from the second shooting brightness, the main camera corresponding to the first zoom ratio, the first shooting distance and the first shooting brightness is the first camera, and the main camera corresponding to the second zoom ratio, the second shooting distance and the second shooting brightness is the first camera; A new main camera is acquired as the second camera according to the second zoom magnification, the second shooting distance, and the second shooting brightness.

4. The method according to claim 3, characterized in that The method further comprises: In response to the second camera being used as the new main camera, starting from N frames of images closest to a switching point, a first image corresponding to the first camera and a second image corresponding to the second camera are overlaid, wherein the switching point is the time point when the main camera is switched from the first camera to the second camera, and N is an integer greater than 1.

5. The method according to claim 4, characterized in that The method further comprises: When the first picture disappears, the second pixel value corresponding to the second picture reaches the preset pixel value, wherein the first picture is a picture in the N frames of images taken by the first camera, and the second picture is a picture in the N frames of images taken by the second camera, and the preset pixel value is the pixel value of the picture displayed before the adjustment operation.

6. The method according to claim 4 or 5, characterized in that: The method further comprises: The dissolve process ends before the switching point.

7. The method according to any one of claims 4 to 6, characterized in that: The method further comprises: Before switching the main camera from the first camera to the second camera, detecting that the second shooting distance changes to a third shooting distance, and / or the second shooting brightness changes to the third shooting brightness; According to the third shooting distance and / or the third shooting brightness, acquiring a new main camera as the third camera; If the third camera is different from the second camera, terminating the superimposition process for the first image and the second image; If the third camera is the same as the second camera, the overlay processing process for the first image and the second image continues to be executed.

8. The method according to any one of claims 4 to 6, characterized in that: The method further comprises: Before switching the main camera from the first camera to the second camera, detecting that the second zoom ratio is adjusted to a third zoom ratio; According to the third zoom ratio, acquiring a new main camera as a fourth camera; If the fourth camera is different from the second camera, terminating the superimposition process for the first image and the second image; If the fourth camera is the same as the second camera, the overlay processing process for the first image and the second image continues to be performed.

9. The method according to any one of claims 1 to 8, characterized in that The adjustment operation is a click operation on the zoom ratio adjustment control.

10. The method according to any one of claims 4 to 9, characterized in that: When the N frames of images closest to the switching point are acquired, an overlay switch is turned on, and the overlay switch is used to turn on or off the auxiliary path.

11. The method according to any one of claims 4 to 10, characterized in that: The method further comprises: Acquire first cropping information of the first image, where the first cropping information is used to crop the size of the first image to a preset size; and, Second cropping information of the second image is acquired, where the second cropping information is used to crop the size of the second image to a preset size.

12. The method according to claim 11, characterized in that The method further comprises: The second image is cropped according to the second cropping information, and a cropped main road image is acquired.

13. The method according to claim 11 or 12, characterized in that: The method further comprises: Read whether the overlay switch is in an on state; When the dissolve switch is in the on state, the first image is cropped according to the first cropping information, and the cropped auxiliary channel image is acquired.

14. The method according to claim 13, characterized in that The method further comprises: Performing a superimposition process on the auxiliary image and the corresponding main image to obtain a display image after the superimposition process, wherein the superimposition process is used to gradually reduce the pixel value of the picture corresponding to the auxiliary image and gradually increase the pixel value of the picture corresponding to the main image in a fused image corresponding to the auxiliary image and the corresponding main image; The sent image is displayed.

15. The method according to any one of claims 9 to 14, characterized in that: The method further comprises: When the first picture disappears, the dissolve switch is turned off.

16. The method according to claim 15, characterized in that When the first picture disappears, turning off the dissolve switch specifically includes: When the number of frames of the auxiliary channel image reaches the N frames and the first picture disappears, turning off the dissolve switch; and / or, When the number of frames of the main image reaches the N frames and the second pixel value corresponding to the second picture reaches the preset pixel value, the overlay switch is turned off.

17. A terminal device, characterized in that: include: processor; Memory; The memory stores a computer program, wherein the computer program includes instructions, and when the instructions are executed by the processor, the terminal device executes the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 16.

Images