An image display method, a terminal, and a storage medium

Abstract

Embodiments of the present application provide an image display method, a terminal and a storage medium, and relate to the technical field of terminals. The method comprises: in response to a shooting mode switching operation, determining, based on a first shooting mode and a second shooting mode, a first constraint strategy for an image frame for display during shooting mode switching, a second constraint strategy for calling a physical camera, a third constraint strategy for an image processing mode, and a fourth constraint strategy for an intermediate focal length during shooting; determining, according to the fourth constraint strategy, a plurality of intermediate focal lengths in the process of shooting mode switching in a first correspondence between a preset relative time and a focal length; adjusting the focal length to the corresponding intermediate focal length when reaching each relative time according to the second constraint strategy, and calling a target physical camera to shoot; and determining a target image frame from the image frame shot by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, so as to improve user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image display method, a terminal, and a storage medium. Background Technology

[0002] With the development of electronic technology, mobile devices offer increasingly more functions. For example, users can use the camera application on their devices to capture images, recording wonderful moments, touching scenes, and other beautiful scenes. See also Figure 1a , Figure 1a The terminal's display interface includes multiple application icons. After detecting that the user clicks the camera application icon, the terminal determines to launch the camera application and displays the following: Figure 1b The image shows the user interface of the camera application. The interface includes a preset preview area 100 and a shooting mode switching area 110. The preset preview area 100 is used to display images captured by the camera. For example, Figure 1c The preset preview area shows an image of the person R1 being photographed. The shooting mode switching area 110 includes multiple mode labels. Mode labels 111, 112, 113, and 114 represent different shooting modes.

[0003] Users can Figure 1b Select different shooting modes on the displayed interface. For example, Figure 1b The default shooting mode of the camera app is the shooting mode represented by mode identifier 112. When the app detects that the user clicks on mode identifier 111, it indicates that the user needs to switch to the shooting mode represented by mode identifier 111. However, when switching shooting modes, the image displayed in the preset preview area has low clarity, which reduces the user experience. Summary of the Invention

[0004] The purpose of this application is to provide an image display method, terminal, and storage medium to improve the clarity of the image displayed in a preset preview area and enhance the user experience. The specific technical solution is as follows:

[0005] Firstly, in order to achieve the above objectives, embodiments of this application provide an image display method, the method comprising:

[0006] In response to the shooting mode switching operation, based on the current first shooting mode and the second shooting mode to be switched to, a first constraint strategy for the image frame to be displayed, a second constraint strategy for calling the physical camera, a third constraint strategy for the image processing method, and a fourth constraint strategy for the intermediate focal length to be shot are determined when switching shooting modes.

[0007] According to the fourth constraint strategy, in the first correspondence between the preset relative time and focal length, multiple intermediate focal lengths for shooting during the shooting mode switching process are determined; wherein, the relative time is the time of shooting relative to the time of starting the shooting mode switching; the multiple intermediate focal lengths are different.

[0008] According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take pictures at that intermediate focal length; wherein, the target physical camera is the physical camera in the first physical camera used in the first shooting mode and the second physical camera used in the second shooting mode.

[0009] According to the third constraint strategy and the first constraint strategy, a target image frame that meets the preset conditions is determined from the image frames captured by the target physical camera, and the target image frame is displayed in a preset preview area; wherein, the clarity of the target image frame is higher than that of other image frames.

[0010] As can be seen from the above, the solution provided in this embodiment constrains the intermediate focal length, the image frames sent for display, and the physical camera called during the shooting mode switching process from multiple aspects according to different constraint strategies, thereby improving the clarity of the captured images. The target image frames that meet the preset conditions are displayed in the preset preview area. The clarity of the target image frames is higher than that of other image frames, which can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0011] In one embodiment of this application, the step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and calling the target physical camera to take pictures at the intermediate focal length, includes:

[0012] At each first relative moment, the focal length is adjusted to the intermediate focal length corresponding to that first relative moment, and the first physical camera is called to take a picture at that intermediate focal length.

[0013] After the number of captured image frames reaches a first number, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the first physical camera and the second physical camera are called to take pictures according to that intermediate focal length.

[0014] The method further includes:

[0015] When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting by the first physical camera is stopped.

[0016] As can be seen from the above, in this embodiment of the application, during the zooming process of switching shooting modes, the second physical camera is activated in advance as an auxiliary path for shooting, which can speed up the switching of shooting modes and achieve seamless switching of physical cameras for the user. Furthermore, image frames not yet stably captured by the second physical camera are not displayed; instead, image frames captured by the first physical camera are used for display, ensuring that the preset preview area displays high-resolution target image frames, thus improving the user experience.

[0017] Furthermore, when the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the second physical camera can capture images stably. If the image frame captured by the second physical camera is sent for display, the preset preview area can still display a target image frame with high clarity. In this case, the first physical camera is stopped from being used for shooting, which can reduce the power consumption of the terminal.

[0018] In one embodiment of this application, the first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames delayed before the shooting mode switching is completed;

[0019] The step of determining a target image frame that meets preset conditions from image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes:

[0020] According to the 3A consistency constraint strategy, the number of image frames delayed in display during the shooting mode switching process is determined as the second number.

[0021] For each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, the image frame captured at the first relative moment is displayed as the target image frame in the preset preview area.

[0022] If the image frame captured at the first relative moment does not meet the preset conditions, a second number of target image frames captured before the first relative moment that meet the preset conditions are determined, and the determined target image frames are displayed in the preset preview area.

[0023] As can be seen from the above, in the embodiments of this application, during the process of switching shooting modes, for each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, it is directly sent for display; if the image frame captured at the first relative moment does not meet the preset conditions, it is delayed for display, that is, the previously captured target image frame is used for display, so that the displayed target image frame is of higher clarity, thereby making the preset preview area display the target image frame of higher clarity, improving the user experience.

[0024] In one embodiment of this application, the third constraint strategy includes: a SAT constraint strategy; the SAT constraint strategy includes: pixel offset between the center points of images captured by different physical cameras;

[0025] The step of determining a target image frame that meets preset conditions from image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes:

[0026] According to the SAT strategy, calculate the pixel offset between the center point of the image frame captured by the first physical camera and the center point of the image frame captured by the second physical camera;

[0027] Based on the pixel offset, the image frames captured by the second physical camera are processed;

[0028] For each second relative time, if the image frame captured at that second relative time meets the preset conditions, the image frame captured at that second relative time is displayed as the target image frame in the preset preview area.

[0029] If the image frame captured at the second relative moment does not meet the preset conditions, determine the second number of target image frames that meet the preset conditions and were captured by the first physical camera before the second relative moment, and display the determined target image frames in the preset preview area.

[0030] As can be seen from the above, in the embodiments of this application, during the process of switching shooting modes, for each second relative moment, if the image frame captured at the second relative moment meets the preset conditions, it is directly sent for display; if the image frame captured at the second relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame captured by the first physical camera is used for display, so that the displayed image is a target image frame with higher clarity, thereby making the preset preview area display a target image frame with higher clarity, improving the user experience.

[0031] Furthermore, a second number of frames (e.g., 3) need to be inserted between the currently invoked first physical camera and the switch to the second physical camera. These second number of frames use image frames captured by the first physical camera for display. In other words, the second physical camera is started two frames ahead of time. On the one hand, this allows the second physical camera to perform 3A convergence, avoiding the problem of excessive scene differences before and after 3A synchronous switching. On the basis of 3A synchronization, several more frames are converged to achieve the optimal effect of the image frames to be displayed. On the other hand, SAT calculation is triggered within the time of the second number of frames to achieve the optimal effect of the center point offset between physical camera switching, so that the user can complete the shooting mode switch without being aware of it, thus improving the user experience.

[0032] In one embodiment of this application, the following steps are used to detect whether a captured image frame meets a preset condition:

[0033] For each relative moment, the image metadata acquired at that relative moment according to the corresponding intermediate focal length is cached in the first storage area;

[0034] According to the order in which the image metadata is cached, the image metadata is retrieved from the first storage area in sequence, processed to obtain the corresponding image frame, and the obtained image frame is stored in the second storage area.

[0035] When the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it is determined that the image frame captured at that relative moment meets the preset condition.

[0036] As can be seen from the above, the solution provided in this embodiment determines whether the physical camera is stably capturing images by detecting whether the image frame meets the preset conditions. Subsequently, the target image frame that meets the preset conditions is displayed in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0037] In one embodiment of this application, the first storage area is the storage area corresponding to the Result object called by the Pipeline image processing method; the second storage area is the storage area corresponding to the Request object called by the Pipeline image processing method.

[0038] In one embodiment of this application, the second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between the physical camera and the shooting mode in the terminal;

[0039] Before invoking the second physical camera to take a picture, the method further includes:

[0040] According to the focal length constraint strategy, the physical camera corresponding to the second shooting mode is determined from the physical cameras in the terminal and used as the second physical camera.

[0041] As can be seen from the above, the solution provided in this embodiment improves the user experience by calling different physical cameras to provide different shooting modes.

[0042] In one embodiment of this application, the fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used by different shooting modes during the shooting mode switching process;

[0043] According to the fourth constraint strategy, in the preset first correspondence between relative time and focal length, multiple intermediate focal lengths for shooting during the shooting mode switching process are determined, including:

[0044] According to the center point constraint strategy, the number of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode is determined as the third number.

[0045] Based on the first correspondence between the preset relative time and focal length, the third number of intermediate focal lengths are determined.

[0046] As can be seen from the above, in this embodiment, a fourth number of image frames are captured at each intermediate focal length. Subsequently, the target image frames from the fourth number of image frames captured at each intermediate focal length can be displayed in a preset preview area, so that the preset preview area displays the target image frames with higher clarity, thereby improving the user experience.

[0047] In one embodiment of this application, after determining a third number of intermediate focal lengths in the first correspondence between a preset relative time and focal length, the method further includes:

[0048] Based on the third number of multiple intermediate focal lengths and the number of image frames required to be captured during the shooting mode switching process, the number of image frames required to be captured at each intermediate focal length is calculated as the fourth number.

[0049] The step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and then calling the target physical camera to take a picture at that intermediate focal length includes:

[0050] At each relative moment, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take the fourth number of image frames at that intermediate focal length.

[0051] As can be seen from the above, in this embodiment, a fourth number of image frames are captured at each intermediate focal length. Subsequently, the target image frames from the fourth number of image frames captured at each intermediate focal length can be displayed in a preset preview area, so that the preset preview area displays the target image frames with higher clarity, thereby improving the user experience.

[0052] In one embodiment of this application, the relationship between relative time and focal length in the first correspondence is a quadratic curve, or the relationship between relative time and focal length in the first correspondence is a Bézier curve.

[0053] As can be seen from the above, in this embodiment, the change trend of relative time and focal length in the first correspondence relationship is a curve that is slow at first and then fast. The initial change trend of the relationship between relative time and focal length in the first correspondence relationship is relatively slow, and the corresponding change in the field of view of the physical camera is also relatively slow. This increases the preparation time before switching physical cameras, allowing the physical camera to be switched to to start first as a secondary camera. The subsequent change trend of the relationship between relative time and focal length in the first correspondence relationship is faster, and the change in the field of view is relatively large. Switching physical cameras at this time, even if there is a slight offset between physical cameras, will not be strongly perceived by the user, improving the user experience.

[0054] In one embodiment of this application, the determined constraint strategy further includes a stability constraint strategy; the stability constraint strategy includes: prohibiting the user from using the shooting function during the shooting mode switching process;

[0055] The method further includes:

[0056] During the transition from the first shooting mode to the second shooting mode, shooting operations are prohibited.

[0057] As can be seen from the above, in this embodiment, during the switch from the first shooting mode to the second shooting mode, the shooting operation is prohibited, thus improving shooting stability. The shooting operation is only responded to after the shooting mode switch is complete, resulting in higher quality images and improved user experience.

[0058] In one embodiment of this application, the determined constraint strategy further includes a power consumption constraint strategy; the power consumption constraint strategy includes: when no shooting mode switching is performed, the number of physical cameras invoked is 1;

[0059] The method further includes:

[0060] When not switching shooting modes, it is prohibited to call other physical cameras besides the physical camera called by the current shooting mode.

[0061] As can be seen from the above, in the embodiments of this application, when the shooting mode is not switched, the terminal maintains a single stream, that is, only one physical camera is called, thereby reducing the power consumption of the terminal.

[0062] In one embodiment of this application, the method further includes:

[0063] After successfully switching to the second shooting mode, the target image frame captured by the second physical camera will be displayed in the preset preview area.

[0064] As can be seen from the above, in this embodiment of the application, after successfully switching from the first shooting mode to the second shooting mode, the physical camera has been switched and the focal length has been adjusted. The camera can stably capture image frames. At this time, the image frames captured have high clarity. Displaying the target image frames captured in the second shooting mode in the preset preview area can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0065] In one embodiment of this application, a third shooting mode is spaced between the first shooting mode and the second shooting mode;

[0066] The step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and then calling the target physical camera to take a picture at that intermediate focal length includes:

[0067] At each first relative moment, the focal length is adjusted to the intermediate focal length corresponding to that first relative moment, and the first physical camera is called to take a picture at that intermediate focal length.

[0068] After the number of captured image frames reaches the first number, at each third relative moment, the focal length is adjusted to the intermediate focal length corresponding to that third relative moment, and the first physical camera and the third physical camera used by the third shooting mode are called to shoot according to the intermediate focal length.

[0069] After the number of captured image frames reaches the fifth number, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the third physical camera and the second physical camera are called to take pictures according to the intermediate focal length.

[0070] The method further includes:

[0071] When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting by the third physical camera is stopped.

[0072] As can be seen from the above, in this embodiment of the application, during the zooming process of switching shooting modes, the physical camera used in the next shooting mode is activated in advance as an auxiliary path for shooting, which can speed up the switching of shooting modes and achieve seamless switching of physical cameras for the user. Furthermore, image frames not yet stably captured by the physical camera are not displayed; instead, image frames captured by the physical camera used in the previous shooting mode are used for display, ensuring that the preset preview area displays high-resolution target image frames, thus improving the user experience.

[0073] Furthermore, when the focal length is adjusted to the focal length used in the next shooting mode, and the image frame captured by the physical camera used in the next shooting mode meets the preset conditions, the physical camera used in the next shooting mode can capture images stably. If the image frame captured by the physical camera used in the next shooting mode is sent for display, the preset preview area can still display a target image frame with high clarity. In this case, the use of the physical camera used in the previous shooting mode will be stopped, which can reduce the power consumption of the terminal.

[0074] In one embodiment of this application, the first shooting mode is: ultra-wide-angle mode; the second shooting mode is: super telephoto mode.

[0075] In one embodiment of this application, the first shooting mode is: super telephoto mode; the second shooting mode is: ultra-wide-angle mode.

[0076] Secondly, embodiments of this application also provide a terminal, including:

[0077] One or more processors and memory;

[0078] The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, which the one or more processors call to cause the terminal to execute any of the above-described image display methods.

[0079] Thirdly, embodiments of this application also provide a computer-readable storage medium, including a computer program, which, when run on a terminal, causes the terminal to execute any of the image display methods described above.

[0080] Fourthly, embodiments of this application also provide a computer program product, the computer program product comprising executable instructions, which, when executed on a terminal, cause the terminal to perform any of the image display methods described above.

[0081] Fifthly, embodiments of this application also provide a chip system applied to a terminal. The chip system includes one or more processors, which are used to invoke computer instructions to cause the terminal to input data into the chip system and execute any of the image display methods described above to process the data and output the processing result.

[0082] The beneficial effects of the solutions provided in the embodiments of the second, third, fourth and fifth aspects above can be found in the beneficial effects of the solutions provided in the embodiments of the first aspect above. Attached Figure Description

[0083] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1a A schematic diagram of a user interface for a terminal provided in an embodiment of this application;

[0085] Figure 1b A schematic diagram of a user interface for a camera application provided in an embodiment of this application;

[0086] Figure 1c A schematic diagram of a first shooting scenario provided in an embodiment of this application;

[0087] Figure 2 A structural diagram of a terminal provided in an embodiment of this application;

[0088] Figure 3 A software structure block diagram of a terminal provided in an embodiment of this application;

[0089] Figure 4 A flowchart illustrating an image display method provided in an embodiment of this application;

[0090] Figure 5a A schematic diagram of a second shooting scenario provided in an embodiment of this application;

[0091] Figure 5b A schematic diagram illustrating a third shooting scenario provided in an embodiment of this application;

[0092] Figure 5c A schematic diagram illustrating the fourth shooting scenario provided in the embodiments of this application;

[0093] Figure 5d A schematic diagram illustrating the fifth shooting scenario provided in the embodiments of this application;

[0094] Figure 6a A graph of relative time versus focal length provided for an embodiment of this application;

[0095] Figure 6b A second graph of relative time versus focal length provided for embodiments of this application;

[0096] Figure 7 This is a schematic diagram of the first type of shooting mode switching data provided in the embodiments of this application;

[0097] Figure 8 This is a schematic diagram of the second shooting mode switching data provided in the embodiments of this application;

[0098] Figure 9 This is a structural diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0099] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0100] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, "first instruction" and "second instruction" are used to distinguish different user instructions and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0101] It should be noted that, in this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0102] The image display method provided in this application is applied to a terminal. The terminal can be a mobile phone, tablet computer, laptop computer, personal digital assistant (PDA), smartwatch, netbook, wearable electronic device, augmented reality (AR) device, virtual reality (VR) device, in-vehicle device, smart car, robot, smart glasses, smart TV, or other terminal equipped with a camera. Thus, the terminal can display the captured image in a preset preview area when switching shooting modes, according to the method provided in this application.

[0103] For example, Figure 2A structural diagram of terminal 200 is shown. Terminal 200 may include a processor 210, a display screen 220, a camera 230, internal memory 240, a Subscriber Identification Module (SIM) card interface 250, a Universal Serial Bus (USB) interface 260, a charging management module 270, a battery management module 271, a battery 272 with battery cells and battery protection devices, a sensor module 280, a mobile communication module 290, a wireless communication module 300, antenna 1, and antenna 2, etc. The sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, a touch sensor 280C, an ambient light sensor 280D, etc.

[0104] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal 200. In other embodiments of this application, the terminal 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] Processor 210 may include one or more processing units, such as a Central Processing Unit (CPU), an Application Processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors. In some embodiments, terminal 200 may also include one or more processors 210. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. In other embodiments, processor 210 may also include a memory for storing instructions and data. For example, the memory in processor 210 may be a cache memory. This memory can store instructions or data that processor 210 has just used or is repeatedly used. If processor 210 needs to reuse the instruction or data, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the terminal 200 in processing data or executing instructions.

[0106] In some embodiments, the processor 210 may include one or more interfaces. These interfaces may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. The USB interface 260 is a USB standard-compliant interface, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 260 can be used to connect a charger to charge the terminal 200, and can also be used for data transfer between the terminal 200 and peripheral devices. The USB interface 260 can also be used to connect headphones for audio playback.

[0107] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are for illustrative purposes only and do not constitute a structural limitation on the terminal 200. In other embodiments of this application, the terminal 200 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0108] The wireless communication function of terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 290, wireless communication module 300, modem processor and baseband processor.

[0109] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0110] Terminal 200 implements display functions through a GPU, display screen 220, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 220 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0111] The display screen 220 is used to display images, videos, etc. The display screen 220 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 200 may include one or more display screens 220.

[0112] In some embodiments of this application, when the display panel uses materials such as OLED, AMOLED, and FLED, the above-mentioned Figure 2 The display screen 220 can be bent. Here, "the display screen 220 can be bent" means that the display screen can be bent to any angle at any point and held at that angle. For example, the display screen 220 can be folded in half horizontally from the middle, or vertically from the middle.

[0113] The display screen 220 of terminal 200 can be a flexible screen. Currently, flexible screens are attracting much attention due to their unique characteristics and enormous potential. Compared to traditional screens, flexible screens are highly flexible and bendable, providing users with new interaction methods based on their bendability and meeting more user needs for terminals. For terminals equipped with foldable displays, the foldable display can switch between a small screen in folded mode and a large screen in unfolded mode at any time. Therefore, users are increasingly using split-screen functionality on terminals equipped with foldable displays.

[0114] Terminal 200 can perform shooting functions through ISP, camera 230, video codec, GPU, display 220 and application processor, etc., wherein camera 230 includes front camera and rear camera.

[0115] The ISP is used to process data fed back from the camera 230. For example, during shooting, when the shutter is opened, light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 230.

[0116] Camera 230 is used to capture photos or videos. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into standard red-green-blue (RGB), YUV, or other image signals. In some embodiments, terminal 200 may include one or N cameras 230, where N is a positive integer greater than 1.

[0117] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal 200 selects a frequency point, the DSP can perform Fourier transforms on the frequency energy.

[0118] Video codecs are used to compress or decompress digital video. Terminal 200 may support one or more video codecs. Thus, terminal 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0119] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in terminals, such as image recognition, facial recognition, speech recognition, and text understanding.

[0120] The internal memory 240 can be used to store one or more computer programs, which include instructions. The processor 210 can execute the instructions stored in the internal memory 240, thereby causing the terminal 200 to perform the image display method, various applications, and data processing provided in some embodiments of this application. The internal memory 240 may include a program storage area and a data storage area. The program storage area may store the operating system; it may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data created by the terminal 200 during use (such as photos, contacts, etc.). Furthermore, the internal memory 240 may include high-speed random access memory and non-volatile memory, such as one or more disk storage components, flash memory components, Universal Flash Storage (UFS), etc. In some embodiments, the processor 210 can execute instructions stored in the internal memory 240 and / or instructions stored in memory disposed within the processor 210, thereby causing the terminal 200 to perform the image display method, other applications, and data processing provided in the embodiments of this application.

[0121] The internal memory 240 can be used to store related programs of the image display method provided in the embodiments of this application. The processor 210 can be used to call the related programs of the image display method stored in the internal memory 240 when displaying information, and execute the image display method of the embodiments of this application.

[0122] The sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, a touch sensor 280C, an ambient light sensor 280D, etc.

[0123] Pressure sensor 280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 280A can be disposed on display screen 220. Pressure sensor 280A can be of many types, such as resistive pressure sensor, inductive pressure sensor, or capacitive pressure sensor. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 280A, the capacitance between the electrodes changes, and terminal 200 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 220, terminal 200 detects the touch operation based on pressure sensor 280A. Terminal 200 can also calculate the touch position based on the detection signal from pressure sensor 280A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed; when a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0124] The fingerprint sensor 280B is used to collect fingerprints. The terminal 200 can use the collected fingerprint characteristics to perform functions such as unlocking, accessing app locks, taking photos, and answering calls.

[0125] Touch sensor 280C, also known as a touch device, can be disposed on display screen 220. The touch sensor 280C and display screen 220 together form a touchscreen, also known as a touch display. Touch sensor 280C is used to detect touch operations applied to or near it. Touch sensor 280C can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 220. In other embodiments, touch sensor 280C may also be disposed on the surface of terminal 200, and in a different location from display screen 220.

[0126] The ambient light sensor 280D is used to sense the ambient light intensity. The terminal 200 can adaptively adjust the brightness of the display screen 220 based on the sensed ambient light intensity. The ambient light sensor 280D can also be used to automatically adjust the white balance during shooting. The ambient light sensor 280D can also transmit environmental information about the device's location to the GPU.

[0127] The ambient light sensor 280D is also used to acquire the brightness, light ratio, color temperature, etc. of the environment in which the camera 230 captures images.

[0128] Figure 3This is a software architecture block diagram for a terminal to which this application's embodiments apply. The terminal's software system can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture divides the terminal's software system into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into three layers: the application layer, the application framework layer, and the hardware abstract layer (HAL).

[0129] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. For example... Figure 3 As shown, the application package may include multiple applications, such as camera, gallery, browser, and music applications. Understandably, the port of each of these applications can be used to receive data.

[0130] The application framework layer provides APIs and a programming framework for applications within the application layer. The application framework layer includes predefined functions. For example... Figure 3 As shown, the application framework layer may include a window manager, content provider, view system, resource manager, notification manager, and Dynamic Host Configuration Protocol (DHCP) module, etc.

[0131] The driver layer is the layer between hardware and software, used to drive the hardware and make it work. Multiple drivers can be installed in the driver layer to operate the hardware. Examples include camera drivers, display drivers, audio drivers, and sensor drivers.

[0132] In addition, the terminal also includes a hardware layer, which may include a camera, speaker, CPU and NPU, etc. The hardware layer is connected to the driver layer.

[0133] In this embodiment of the application, after the camera application in the application layer is launched, the camera driver in the driver layer calls the camera (i.e., the physical camera) in the hardware layer to capture images, and the images captured by the camera are displayed in a preset preview area on the display screen.

[0134] Next, the image display scheme provided in this application will be described in detail through specific embodiments.

[0135] In one embodiment of this application, see [link to embodiment]. Figure 4, Figure 4 A flowchart of an image display method provided in this application embodiment, the method including the following steps:

[0136] S401: In response to the shooting mode switching operation, based on the current first shooting mode and the second shooting mode to which the shooting mode needs to be switched, determine a first constraint strategy for the image frame to be displayed, a second constraint strategy for calling the physical camera, a third constraint strategy for the image processing method, and a fourth constraint strategy for the intermediate focal length to be captured.

[0137] S402: According to the fourth constraint strategy, in the first correspondence between the preset relative time and focal length, determine multiple intermediate focal lengths for shooting during the shooting mode switching process.

[0138] The relative time refers to the moment when the shot is taken relative to the moment when the shooting mode is switched; the intermediate focal lengths are different.

[0139] S403: According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take a picture at that intermediate focal length.

[0140] The target physical camera is the physical camera used in the first shooting mode and the second shooting mode.

[0141] S404: According to the third constraint strategy and the first constraint strategy, determine the target image frame that meets the preset conditions from the image frames captured by the target physical camera, and display the target image frame in the preset preview area.

[0142] The target image frame has higher resolution than the other image frames.

[0143] As can be seen from the above, the solution provided in this embodiment constrains the intermediate focal length, the image frames sent for display, and the physical camera called during the shooting mode switching process from multiple aspects according to different constraint strategies, thereby improving the clarity of the captured images. The target image frames that meet the preset conditions are displayed in the preset preview area. The clarity of the target image frames is higher than that of other image frames, which can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0144] In step S401, a camera application for taking pictures is installed in the terminal, and the camera application provides multiple shooting modes.

[0145] Shooting modes include: Ultra Wide (UW) mode, Wide (W) mode, Tele (T) mode, and Super Tele (ST) mode. The Wide mode can also be called the main camera mode. In Wide mode, the physical camera's focal length is the equivalent focal length. The equivalent focal length is a first value. For example, the first value is 27mm. Figure 5a The medium mode identifier 112 indicates wide-angle mode. The physical camera focal length used for other shooting modes can be determined based on the equivalent focal length.

[0146] In ultra-wide-angle mode, the physical camera's focal length is a multiple of the equivalent focal length. For example, Figure 5a The medium mode identifier 111 indicates the ultra-wide-angle mode, in which the physical camera's focal length is 0.6 times the equivalent focal length. Figure 5a The example shown is for illustrative purposes only; the first multiplier can be set to other values ​​as needed. For instance, in ultra-wide-angle mode, the physical camera's focal length can also be 0.5 times the equivalent focal length.

[0147] In telephoto mode, the physical camera's focal length is a multiple of the equivalent focal length. For example, Figure 5a The medium mode identifier 113 indicates telephoto mode, in which the physical camera's focal length is 3 times the equivalent focal length. Figure 5a The example shown is for illustrative purposes only; the second multiplier can be set to other values ​​as needed. For instance, in telephoto mode, the physical camera's focal length can also be 2.5 times the equivalent focal length.

[0148] In super telephoto mode, the physical camera's focal length is a third multiple of the equivalent focal length. For example, Figure 5a The medium mode identifier 114 indicates super telephoto mode, in which the physical camera's focal length is 6 times the equivalent focal length. Figure 5a The example shown is for illustrative purposes only; the third multiplier can be set to other values ​​as needed. For instance, in super telephoto mode, the physical camera's focal length can also be 5 times the equivalent focal length.

[0149] The camera app defaults to main camera mode (i.e., wide-angle mode) upon startup. For example... Figure 5a As shown, when the camera application is in wide-angle mode, due to the limited field of view, it can only capture the complete image of person R1, but cannot capture the complete images of people R2 and R3.

[0150] When capturing a complete image including subjects R1, R2, and R3, the physical camera's focal length needs to be reduced. For example, switch to ultra-wide-angle mode. The user can tap mode icon 111. In response to the user's shooting mode switching action, the device will switch the camera application's shooting mode to [mode name missing]. Figure 5bThe ultra-wide-angle mode shown can capture complete images of people R1, R2, and R3.

[0151] When capturing a complete image containing only the person R1, the physical camera's focal length needs to be increased. For example, switch to telephoto mode. The user can tap mode icon 113. In response to the user's shooting mode switching operation, the terminal switches the camera application's shooting mode to... Figure 5c The telephoto mode shown allows you to capture a complete image containing only the person R1.

[0152] When capturing detailed images of a portion of the subject (R1), it's necessary to increase the physical camera's focal length again. For example, switch to super telephoto mode. The user can tap mode icon 114. In response to the user's shooting mode switching operation, the terminal switches the camera application's shooting mode to... Figure 5d The super telephoto mode shown allows you to capture detailed images containing only the upper body of person R1.

[0153] Accordingly, upon detecting a shooting mode switching operation, in response to the operation, the terminal can determine multiple constraint strategies for switching shooting modes based on the current first shooting mode and the desired second shooting mode. These constraint strategies include: a first constraint strategy for the displayed image frame, a second constraint strategy for invoking the physical camera, a third constraint strategy for the image processing method, and a fourth constraint strategy for the intermediate focal length used for shooting.

[0154] The following describes, with reference to Table 1, several constraint strategies in the embodiments of this application by way of example.

[0155] Table 1

[0156]

[0157]

[0158] Focal length constraint strategies include the correspondence between the physical camera in the terminal and the shooting mode.

[0159] The terminal includes multiple physical cameras, each configured for a different shooting mode. This allows for a focal length that is shorter in ultra-wide-angle mode than in main camera mode; shorter in main camera mode than in telephoto mode; and shorter in telephoto mode than in super-telephoto mode. Furthermore, a focal length constraint strategy is implemented between each shooting mode to avoid discontinuous field of view (Fov), thus improving the user experience.

[0160] In addition, shooting mode switching cannot skip focal lengths. For example, when a user instructs to switch from ultra-wide-angle mode to telephoto mode, the device needs to first switch the camera application's shooting mode from ultra-wide-angle mode to wide-angle mode, and then switch from wide-angle mode to telephoto mode. It cannot switch directly from ultra-wide-angle mode to telephoto mode.

[0161] The 3A constraint strategy includes delaying the display of image frames before completing the shooting mode switch. 3A refers to the physical camera's Auto Focus (AF), Automatic Exposure (AE), and Auto White Balance (AWB).

[0162] When the second physical camera is invoked, the images captured before 3A convergence are of low quality. Therefore, the image frames captured by the second physical camera need to be displayed with a delay. That is, the first few image frames (e.g., the first 3 frames) captured immediately after the second physical camera is activated are not displayed as auxiliary frames. Images captured after the second physical camera achieves 3A convergence are then displayed. Displaying means showing the captured image frames in a preset preview area.

[0163] The SAT constraint strategy includes pixel offsets between the center points of images taken by different physical cameras.

[0164] The SAT algorithm requires dual-stream operation, meaning it uses two physical cameras (using the first and second physical cameras as an example) to capture images. Based on the image frames captured by the first and second physical cameras, the pixel offset (offset) between the center points of the images captured by the different physical cameras is calculated. The pixel offset calculation process takes approximately three frames. Therefore, the image warp cannot be superimposed on the zoom process until after three frames; that is, spatial alignment transformation of the captured image frames can only be performed according to the calculated pixel offset after three frames.

[0165] The center point constraint strategy includes the number of intermediate focal lengths between the focal lengths used in different shooting modes during the shooting mode switching process.

[0166] When the focal length remains constant, switching between physical cameras results in a significant shift in the center point of the image frames taken by different physical cameras, which is noticeably perceptible to the user. Therefore, it is necessary to switch physical cameras during zooming. Furthermore, simultaneously increasing the zoom speed and the speed of physical camera switching allows for a virtually imperceptible transition between physical cameras. Thus, reducing the number of intermediate focal lengths used between different shooting modes can accelerate both zooming and physical camera switching.

[0167] The smoothness constraint strategy includes: prohibiting modification of the physical camera configuration during shooting mode switching, and limiting the number of image frames captured by two physical cameras simultaneously during shooting mode switching.

[0168] Switching between physical cameras results in dropped frames or longer frame intervals in the captured images. Modifying the physical camera configuration during shooting mode switching affects the smoothness of the transition. Therefore, modifying the physical camera configuration during shooting mode switching is prohibited to improve zoom preview smoothness. Furthermore, simultaneously using two physical cameras during shooting mode switching consumes significant power. Therefore, the dual-stream time should be reduced by setting a smaller number of image frames captured by both physical cameras simultaneously during shooting mode switching, and using three physical cameras should be prohibited to lower terminal power consumption.

[0169] Stability constraint strategies include: prohibiting users from using the shooting function during shooting mode switching.

[0170] Taking photos during zooming results in poor stability. This is because the image metadata captured by the physical camera undergoes ISP processing via a pipeline approach. Before processing, the image metadata is stored in a Result object, while the processed image frames are stored in a Request object. When a user uses the shooting function, data needs to be retrieved from both the Request and Result objects, and the image to be saved at the user's instruction is generated based on this data. However, there can be a difference of several frames between the data in the Request and Result objects, leading to poor shooting stability and low image quality. Therefore, the shooting function should be disabled during zooming to improve shooting stability.

[0171] The power consumption constraint strategy includes: when no shooting mode switching is performed, the number of physical cameras invoked is 1.

[0172] The more physical cameras a terminal activates, the longer it takes to activate multiple physical cameras, and the higher the terminal's power consumption. Therefore, without changing the zoom level, keeping the terminal in single-stream mode, i.e., using only one physical camera, reduces the terminal's power consumption.

[0173] In some embodiments, the first shooting mode and the second shooting mode are two adjacent shooting modes. For example, the first shooting mode is an ultra-wide-angle mode; the second shooting mode is a wide-angle mode. Or, the first shooting mode is a wide-angle mode; the second shooting mode is a telephoto mode, etc.

[0174] In some embodiments, a third shooting mode is spaced between the first shooting mode and the second shooting mode. There can be multiple third shooting modes. For example, the first shooting mode is an ultra-wide-angle mode; the second shooting mode is a super-telephoto mode. The third shooting mode spaced between the ultra-wide-angle mode and the super-telephoto mode includes: wide-angle mode and telephoto mode. That is, from... Figure 5b The shooting mode shown has been switched to Figure 5d The shooting mode shown.

[0175] Alternatively, the first shooting mode can be super telephoto mode; the second shooting mode can be ultra-wide-angle mode. That is, from... Figure 5d The shooting mode shown has been switched to Figure 5b The shooting mode shown.

[0176] There can be one third shooting mode. For example, the first shooting mode is ultra-wide-angle mode; the second shooting mode is telephoto mode. The ultra-wide-angle mode and the telephoto mode are separated by the wide-angle mode.

[0177] The above are just examples; the first shooting mode and the second shooting mode can also be other shooting modes. For example, the first shooting mode can be telephoto mode, and the second shooting mode can be wide-angle mode. Or, the first shooting mode can be wide-angle mode, and the second shooting mode can be super telephoto mode, etc.

[0178] For step S402, the relative time is the time when the shooting begins relative to the time when the shooting mode is switched.

[0179] Relative time can be represented by captured image frames. For example, relative time can be expressed as: the time when the first image frame is captured after the start of the shooting mode switch, the time when the second image frame is captured after the start of the shooting mode switch, the time when the third image frame is captured after the start of the shooting mode switch, ..., the time when the nth image frame is captured after the start of the shooting mode switch. n represents the number of image frames required during the shooting mode switch process.

[0180] Correspondingly, the first correspondence can be expressed as follows: the focal length of the physical camera at the moment of taking the first image frame after the moment of starting the shooting mode switch is f1, the focal length of the physical camera at the moment of taking the second image frame after the moment of starting the shooting mode switch is f2, the focal length of the physical camera at the moment of taking the third image frame after the moment of starting the shooting mode switch is f3, ..., the focal length of the physical camera at the moment of taking the nth image frame after the moment of starting the shooting mode switch is fn.

[0181] Relative time can also be expressed as time. For example, relative time can be expressed as: the 30th millisecond after the start of the shooting mode switching, the 60th millisecond after the start of the shooting mode switching, the 90th millisecond after the start of the shooting mode switching, ..., the mth millisecond after the start of the shooting mode switching. m represents the duration required for the shooting mode switching.

[0182] Accordingly, the first correspondence can be expressed as follows: the physical camera's focal length is f1 at the 30th millisecond after the start of the shooting mode switch, f2 at the 60th millisecond after the start of the shooting mode switch, f3 at the 90th millisecond after the start of the shooting mode switch, ..., and fn at the mth millisecond after the start of the shooting mode switch. n represents the number of image frames required during the shooting mode switch process.

[0183] The first correspondence can be the Zoom Ratio function, which can be any of the following: a linear curve, a quadratic curve, or a Bézier curve. The Zoom Ratio function represents the relationship between relative time and focal length.

[0184] In some embodiments, the Zoom Ratio function can be selected to have a trend of gradual change followed by rapid change. For example, a quadratic curve or a Bézier curve can be selected. That is, the relationship between relative time and focal length in the first correspondence is a quadratic curve, or the relationship between relative time and focal length in the first correspondence is a Bézier curve.

[0185] Switching physical cameras when the field of view changes little or not at all can result in significant differences between captured image frames, which is strongly perceived by the user and negatively impacts the user experience. Therefore, the Zoom Ratio function uses a curve with a gradual initial change followed by a faster change. The initial slow change in the Zoom Ratio function corresponds to a slower change in the field of view of the physical camera, allowing more preparation time before switching to a physical camera, enabling the camera to be switched to to start first as a secondary camera. The subsequent rapid change in the Zoom Ratio function, with a larger change in the field of view, means that even slight shifts between physical cameras are less noticeable to the user, improving the user experience.

[0186] The following combination Figure 6a and Figure 6b The first correspondence is illustrated by an example. Figure 6a and Figure 6bIn the diagram, T indicates telephoto mode; ST indicates super telephoto mode; W indicates wide-angle mode; and UW indicates ultra-wide-angle mode. Furthermore, in... Figure 6a and Figure 6b The curves shown exhibit a trend of initial slowdown followed by a gradual decrease. ST points to the focal length used in super telephoto mode; T points to the focal length used in telephoto mode; W points to the focal length used in wide-angle mode; and UW points to the focal length used in ultra-wide-angle mode. Figure 6a In this context, UW+W represents the intermediate focal length when switching from ultra-wide-angle mode to wide-angle mode. W+T represents the intermediate focal length when switching from wide-angle mode to telephoto mode. T+ST represents the intermediate focal length when switching from telephoto mode to super telephoto mode. Figure 6b In this context, ST+T represents the intermediate focal length during the transition from super-telephoto mode to telephoto mode. T+W represents the intermediate focal length during the transition from telephoto mode to wide-angle mode. W+UW represents the intermediate focal length during the transition from wide-angle mode to ultra-wide-angle mode.

[0187] In some embodiments, the first shooting mode is an ultra-wide-angle mode; the second shooting mode is a super-telephoto mode. In the first correspondence, the focal length increases with increasing relative time. For example, in the first correspondence, the relative time and focal length are related according to… Figure 6a The curve changes shown. Figure 6a In this mode, the focal length gradually increases from the focal length used in the ultra-wide-angle mode, through several intermediate focal lengths, to the focal length used in the super telephoto mode.

[0188] In some embodiments, the first shooting mode is a super telephoto mode; the second shooting mode is an ultra-wide-angle mode. In the first correspondence, the focal length decreases as the relative time increases. The relative time and focal length in the first correspondence are based on… Figure 6b The curve changes shown. Figure 6b In the process, the focal length gradually decreases from the focal length used in the super telephoto mode, through multiple intermediate focal lengths, to the focal length used in the ultra-wide-angle mode.

[0189] In some embodiments, the fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used by different shooting modes during the shooting mode switching process.

[0190] Accordingly, step S402 may include the following steps: determining the number of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode according to the center point constraint strategy, as a third number; determining the third number of intermediate focal lengths in a preset first correspondence between relative time and focal length.

[0191] The terminal determines a third number of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode according to the center point constraint strategy, and determines the third number of intermediate focal lengths according to the first correspondence.

[0192] The third number can be set according to needs. Setting the third number to a smaller value reduces the number of intermediate focal lengths used during shooting mode switching, thereby reducing the time required for shooting mode switching. This speeds up the switching process and reduces the time the image frames are displayed in the preset preview area, allowing users to switch shooting modes seamlessly and improving the user experience.

[0193] Regarding step S403, after determining each intermediate focal length, at each relative time, the terminal adjusts the focal length to the focal length corresponding to that relative time and calls the target physical camera to take a picture at that intermediate focal length. When the number of captured image frames has not reached a first number, the target physical camera is the first physical camera; after the number of captured image frames reaches the first number, the target physical camera includes both the first physical camera and the second physical camera.

[0194] For example, at 30 milliseconds after the start of the shooting mode switching, the focal length is adjusted to f1, and the target physical camera is used to take a picture at this intermediate focal length; at 60 milliseconds after the start of the shooting mode switching, the focal length is adjusted to f2, and the target physical camera is used to take a picture at this intermediate focal length, and so on, until the shooting mode switching is completed.

[0195] In some embodiments, after determining a third number of intermediate focal lengths, the method may further include the following steps: based on the third number of intermediate focal lengths and the number of image frames to be captured during the shooting mode switching process, calculating the number of image frames to be captured at each intermediate focal length as a fourth number.

[0196] Accordingly, step S403 includes the following steps: when each relative moment is reached, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take a fourth number of image frames at that intermediate focal length.

[0197] Having determined the third number of intermediate focal lengths, the fourth number of image frames required at each intermediate focal length can be calculated based on the number of image frames needed during the shooting mode switching process. Accordingly, at each relative moment, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is invoked to capture the fourth number of image frames at that intermediate focal length.

[0198] As can be seen from the above, in this embodiment, a fourth number of image frames are captured at each intermediate focal length. Subsequently, the target image frames from the fourth number of image frames captured at each intermediate focal length can be displayed in a preset preview area, so that the preset preview area displays the target image frames with higher clarity, thereby improving the user experience.

[0199] In some embodiments, the first shooting mode and the second shooting mode are two adjacent shooting modes, so the user can directly switch from the first shooting mode to the second shooting mode.

[0200] Accordingly, step S403 may include the following steps: upon reaching each first relative moment, adjusting the focal length to the intermediate focal length corresponding to that first relative moment, and calling the first physical camera to take a picture at that intermediate focal length. After the number of captured image frames reaches a first number, upon reaching each second relative moment, adjusting the focal length to the intermediate focal length corresponding to that second relative moment, and calling the first physical camera and the second physical camera to take a picture at that intermediate focal length.

[0201] Correspondingly, the method may also include the following steps: when the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, stop capturing images using the first physical camera.

[0202] The first relative time is the relative time before the second physical camera is invoked; the second relative time is the relative time after the second physical camera is invoked.

[0203] When switching from the first shooting mode to the second shooting mode, if the second physical camera is activated after the focal length has been adjusted to the focal length used in the second shooting mode, the second physical camera cannot capture images stably upon initial startup. For example, the second physical camera may not have completed 3A convergence, resulting in lower image frame sharpness. Therefore, the second physical camera should be activated earlier during zooming. Consequently, the relative timing during the switch from the first shooting mode to the second shooting mode includes both a first relative timing and a second relative timing.

[0204] Upon reaching each first relative moment, the terminal adjusts the focal length to the intermediate focal length corresponding to that first relative moment and calls the first physical camera to take a picture at that intermediate focal length.

[0205] Once the number of captured image frames reaches a first threshold, indicating that the focal length is close to that used in the second shooting mode, the second physical camera can be used as an auxiliary camera for shooting, and the captured image frames are not sent for display. Furthermore, since the second physical camera cannot capture images stably when it first starts up, the first physical camera is simultaneously used as the primary camera for shooting, and the captured image frames are sent for display. In other words, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and both the first and second physical cameras are used to shoot at that intermediate focal length.

[0206] Furthermore, when the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting mode switching is completed, the second physical camera can capture images stably, and the image frame captured by the second physical camera can be used for display. If the preset preview area still displays the target image frame with higher clarity, then the first physical camera is stopped from being used for shooting.

[0207] As can be seen from the above, in this embodiment of the application, during the zooming process of switching shooting modes, the second physical camera is activated in advance as an auxiliary path for shooting, which can speed up the switching of shooting modes and achieve seamless switching of physical cameras for the user. Furthermore, image frames not yet stably captured by the second physical camera are not displayed; instead, image frames captured by the first physical camera are used for display, ensuring that the preset preview area displays high-resolution target image frames, thus improving the user experience.

[0208] Furthermore, when the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the second physical camera can capture images stably. If the image frame captured by the second physical camera is sent for display, the preset preview area can still display a target image frame with high clarity. In this case, the first physical camera is stopped from being used for shooting, which can reduce the power consumption of the terminal.

[0209] In some embodiments, the second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between the physical camera in the terminal and the shooting mode.

[0210] Accordingly, before calling the second physical camera to take a picture, the method may also include the following steps: according to the focal length constraint strategy, determine the physical camera corresponding to the second shooting mode from the physical cameras in the terminal, and use it as the second physical camera.

[0211] Since different shooting modes require access to different physical cameras on the device, when it is determined that a switch to the second shooting mode is needed, the device can determine the second physical camera corresponding to the second shooting mode from among the physical cameras on the device, according to a focal length constraint strategy. By accessing different physical cameras, different shooting modes are provided to the user, thus improving the user experience.

[0212] In some embodiments, if a third shooting mode is interspersed between the first shooting mode and the second shooting mode, it is necessary to switch from the first shooting mode to the third shooting mode, and then switch from the third shooting mode to the second shooting mode.

[0213] Accordingly, step S403 may include the following steps: at each first relative moment, the focal length is adjusted to the intermediate focal length corresponding to the first relative moment, and the first physical camera is called to take a picture at the intermediate focal length; after the number of captured image frames reaches a first number, at each third relative moment, the focal length is adjusted to the intermediate focal length corresponding to the third relative moment, and the first physical camera and the third physical camera used in the third shooting mode are called to take a picture at the intermediate focal length; after the number of captured image frames reaches a fifth number, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to the second relative moment, and the third physical camera and the second physical camera are called to take a picture at the intermediate focal length.

[0214] Correspondingly, the method may also include the following steps: when the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, stop calling the third physical camera to shoot.

[0215] The first relative time is the relative time before the second physical camera is called; the third relative time is the relative time after the third physical camera is called and before the second physical camera is called; the second relative time is the relative time after the second physical camera is called.

[0216] When switching from the first shooting mode to the third shooting mode, if the third physical camera is activated after the focal length has been adjusted to the focal length used in the third shooting mode, the third physical camera cannot capture images stably upon initial startup. For example, the third physical camera may not have completed 3A convergence, resulting in lower image frame sharpness. Therefore, the third physical camera should be activated earlier during zooming. Consequently, the relative timing during the switch from the first shooting mode to the third shooting mode includes both the first relative timing and the third relative timing.

[0217] Upon reaching each first relative moment, the terminal adjusts the focal length to the intermediate focal length corresponding to that first relative moment and calls the first physical camera to take a picture at that intermediate focal length.

[0218] Once the number of image frames captured by the first physical camera reaches a certain threshold, indicating that the focal length is close to that used in the third shooting mode, the third physical camera can be used as a secondary camera for shooting, and the captured image frames are not displayed. Furthermore, since the third physical camera cannot capture images stably when it first starts up, the first physical camera is used as the primary camera for shooting, and the captured image frames are displayed. In other words, at each third relative moment, the focal length is adjusted to the intermediate focal length corresponding to that third relative moment, and both the first and third physical cameras are used to capture images at that intermediate focal length.

[0219] When the focal length is adjusted to the focal length used in the third shooting mode, and the image frame captured by the third physical camera meets the preset conditions, the shooting mode has been switched to the third shooting mode. The third physical camera can capture images stably, and the image frame captured by the third physical camera can be used for display. If the preset preview area still displays the target image frame with high clarity, then the first physical camera is stopped from being used for shooting.

[0220] Furthermore, after the number of image frames captured by the third physical camera reaches the fifth number, indicating that the focal length is close to that used in the second shooting mode, the second physical camera can be used as a secondary camera for shooting, and the captured image frames are not sent for display. Since the second physical camera cannot capture images stably when it first starts up, the third physical camera is simultaneously used as the primary camera for shooting, and the captured image frames are used for display. In other words, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and both the third and second physical cameras are used to capture images at that intermediate focal length.

[0221] When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting mode has been switched to the second shooting mode. The second physical camera can capture images stably, and the image frame captured by the second physical camera can be used for display. If the preset preview area still displays the target image frame with high clarity, then the third physical camera will stop being used for shooting.

[0222] As can be seen from the above, in this embodiment of the application, during the zooming process of switching shooting modes, the physical camera used in the next shooting mode is activated in advance as an auxiliary path for shooting, which can speed up the switching of shooting modes and achieve seamless switching of physical cameras for the user. Furthermore, image frames not yet stably captured by the physical camera are not displayed; instead, image frames captured by the physical camera used in the previous shooting mode are used for display, ensuring that the preset preview area displays high-resolution target image frames, thus improving the user experience.

[0223] Furthermore, when the focal length is adjusted to the focal length used in the next shooting mode, and the image frame captured by the physical camera used in the next shooting mode meets the preset conditions, the physical camera used in the next shooting mode can capture images stably. If the image frame captured by the physical camera used in the next shooting mode is sent for display, the preset preview area can still display a target image frame with high clarity. In this case, the use of the physical camera used in the previous shooting mode will be stopped, which can reduce the power consumption of the terminal.

[0224] In step S404, the terminal calls the target physical camera to capture multiple image frames at each intermediate focal length. These image frames include images of higher and lower quality. For each relative time, the terminal, according to the third constraint strategy and the first constraint strategy, determines the target image frame that meets the preset conditions from the image frames captured at the intermediate focal length corresponding to that relative time, and displays the target image frame in the preset preview area.

[0225] In some embodiments, the first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames that are delayed before the shooting mode switch is completed.

[0226] Accordingly, step S404 may include the following steps: According to the 3A consistency constraint strategy, determine the number of image frames delayed in display during the shooting mode switching process, as the second number. For each first relative moment, if the image frame captured at that first relative moment meets a preset condition, display that image frame as a target image frame in a preset preview area. If the image frame captured at that first relative moment does not meet the preset condition, determine a second number of target image frames captured before that first relative moment that meet the preset condition, and display the determined target image frames in the preset preview area.

[0227] The second number is set according to requirements. For example, the second number is 3, or the second number is 4, etc., but no specific limitation is made in this embodiment.

[0228] For each first relative moment, it is detected whether the image frame captured at the first relative moment meets the preset conditions. If the preset conditions are met, it indicates that the image frame is a stable image frame captured by the first physical camera, and the image frame has good quality and high clarity. Then, the image frame captured at the first relative moment is displayed as the target image frame in the preset preview area.

[0229] If the image frame captured at the first relative moment does not meet the preset conditions, it indicates that the image frame is not a stable image frame captured by the first physical camera, and the image frame has low clarity. Then, from the image frames captured before the first relative moment, a second number of target image frames that meet the preset conditions are determined, and the determined target image frames are displayed in the preset preview area.

[0230] The method for detecting whether an image frame meets the preset conditions is described in the following embodiments.

[0231] As can be seen from the above, in the embodiments of this application, during the process of switching shooting modes, for each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, it is directly sent for display; if the image frame captured at the first relative moment does not meet the preset conditions, it is delayed for display, that is, the previously captured target image frame is used for display, so that the displayed target image frame is of higher clarity, thereby making the preset preview area display the target image frame of higher clarity, improving the user experience.

[0232] In some embodiments, the third constraint strategy includes: the SAT constraint strategy; the SAT constraint strategy includes: pixel offset between the center points of images captured by different physical cameras.

[0233] Accordingly, step S404 may include the following steps: Calculate the pixel offset between the center point of an image frame captured by the first physical camera and the center point of an image frame captured by the second physical camera, according to the SAT strategy. Process the image frame captured by the second physical camera based on the pixel offset. For each second relative time, if the image frame captured at that second relative time meets a preset condition, display the image frame captured at that second relative time as a target image frame in a preset preview area. If the image frame captured at that second relative time does not meet the preset condition, determine a second number of target image frames captured by the first physical camera before that second relative time that meet the preset condition, and display the determined target image frames in the preset preview area.

[0234] Switching from the first shooting mode to the second shooting mode requires switching the physical camera used, but the center points of images captured by the different physical cameras will differ. To achieve seamless switching of physical cameras for the user, a SAT constraint strategy is used to determine the pixel offset between the center points of image frames captured by the first and second physical cameras. Then, the image frames captured by the second physical camera are processed according to the calculated pixel offset. For example, a spatial alignment transformation is performed on the image frames captured by the second physical camera based on the calculated pixel offset value, ensuring that the center points of the image frames captured by the second physical camera are consistent with those captured by the first physical camera.

[0235] Furthermore, for each second relative moment, it is detected whether the image frame captured at that second relative moment meets the preset conditions. If the preset conditions are met, it indicates that the image frame is a stable image frame captured by the second physical camera, and the image frame has good quality and high clarity. Then, the image frame captured at the second relative moment is displayed as the target image frame in the preset preview area.

[0236] If the image frame captured at the second relative moment does not meet the preset conditions, it indicates that the image frame is not a stable image frame captured by the second physical camera, and the image frame has low clarity, possibly because the second physical camera has just started and has not yet achieved 3A convergence. Accordingly, from the image frames captured by the first physical camera before the second relative moment, a second number of target image frames that meet the preset conditions are determined, and the determined target image frames are displayed in a preset preview area.

[0237] As can be seen from the above, in the embodiments of this application, during the process of switching shooting modes, for each second relative moment, if the image frame captured at the second relative moment meets the preset conditions, it is directly sent for display; if the image frame captured at the second relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame captured by the first physical camera is used for display, so that the displayed image is a target image frame with higher clarity, thereby making the preset preview area display a target image frame with higher clarity, improving the user experience.

[0238] Furthermore, a second number of frames (e.g., 3) need to be inserted between the currently invoked first physical camera and the switch to the second physical camera. These second number of frames use image frames captured by the first physical camera for display. In other words, the second physical camera is started two frames ahead of time. On the one hand, this allows the second physical camera to perform 3A convergence, avoiding the problem of excessive scene differences before and after 3A synchronous switching. On the basis of 3A synchronization, several more frames are converged to achieve the optimal effect of the image frames to be displayed. On the other hand, SAT calculation is triggered within the time of the second number of frames to achieve the optimal effect of the center point offset between physical camera switching, so that the user can complete the shooting mode switch without being aware of it, thus improving the user experience.

[0239] In some embodiments, the following steps are used to detect whether a captured image frame meets a preset condition: For each relative time moment, image metadata acquired at that relative time moment according to the corresponding intermediate focal length is cached in a first storage area. Image metadata is retrieved sequentially from the first storage area according to the chronological order of caching, processed to obtain the corresponding image frame, and the obtained image frame is stored in a second storage area. When the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it is determined that the image frame captured at that relative time moment meets the preset condition.

[0240] The terminal accesses image metadata captured by the physical camera, such as unprocessed (RAW) data. This metadata needs to be processed by ISP to obtain image frames in RGB or YUV format. The terminal can use a pipeline image processing method to perform ISP processing on the image metadata.

[0241] For each relative moment, the terminal caches the image metadata acquired at that relative moment according to the corresponding intermediate focal length in the first storage area. For example, the image metadata is cached in the storage area corresponding to the called Result object.

[0242] Then, according to the chronological order in which the image metadata is cached, the image metadata is retrieved sequentially from the first storage area, processed by the ISP to obtain the corresponding image frames, and the obtained image frames are stored in the second storage area. For example, the image frames are cached in the storage area corresponding to the invoked Request object.

[0243] Since processing image metadata takes a certain amount of time, when an image frame is cached in the second storage area, the first storage area may have already cached new image metadata. Therefore, the focal length corresponding to the image metadata in the first storage area may be different from the focal length corresponding to the image frame in the second storage area. In this case, it indicates that the physical camera has not captured an image stably, and it can be determined that the captured image frame does not meet the preset conditions.

[0244] If the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it indicates that the physical camera is capturing a stable image, and the captured image frame can be determined to meet the preset conditions.

[0245] As can be seen from the above, the solution provided in this embodiment determines whether the physical camera is stably capturing images by detecting whether the image frame meets the preset conditions. Subsequently, the target image frame that meets the preset conditions is displayed in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0246] In some embodiments, a third shooting mode is spaced between the first shooting mode and the second shooting mode; the display method during the process of switching from the first shooting mode to the third shooting mode, and the display method during the process of switching from the third shooting mode to the second shooting mode, are similar to the display method during the process of directly switching from the first shooting mode to the second shooting mode, as described in the foregoing embodiments.

[0247] Similarly, during the switching of shooting modes, for each relative moment, if the image frame captured at that relative moment meets the preset conditions, it is directly displayed; if the image frame captured at that relative moment does not meet the preset conditions, it is displayed with a delay, that is, the target image frame captured by the physical camera called in the previous shooting mode before that relative moment is displayed. For example, if the image frame captured at the third relative moment does not meet the preset conditions, the target image frame is determined from the image frames captured by the first physical camera called before that third relative moment and displayed; or, if the image frame captured at the second relative moment does not meet the preset conditions, the target image frame is determined from the image frames captured by the third physical camera called before that second relative moment and displayed. This ensures that the displayed image frame is a high-resolution target image frame, thus ensuring that the preset preview area displays a high-resolution target image frame, improving the user experience.

[0248] In some embodiments, the method may further include the following steps: after successfully switching to the second shooting mode, displaying the target image frame captured by the second physical camera in a preset preview area.

[0249] When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, it indicates a successful switch to the second shooting mode. The target image frame captured by the second physical camera will then be displayed in the preset preview area. After successfully switching from the first shooting mode to the second shooting mode, the physical camera switching and focal length adjustment are complete, and the camera can stably capture image frames. The captured image frames at this time have higher clarity. Displaying the target image frame captured in the second shooting mode in the preset preview area improves the clarity of the image frames displayed in the preset preview area, thus enhancing the user experience.

[0250] In some embodiments, the constraint strategy determined in step S402 further includes a stability constraint strategy; the stability constraint strategy includes: prohibiting the user from using the shooting function during the shooting mode switching process.

[0251] Accordingly, the method may also include the following steps: during the switching from the first shooting mode to the second shooting mode, the shooting operation is prohibited.

[0252] Switching from the first shooting mode to the second requires adjusting the focal length. The stability of the zoom is poor, resulting in lower image quality. Furthermore, shooting consumes significant CPU resources, impacting device performance. This can affect zoom smoothness on lower-spec devices or in scenarios with high system load, consequently affecting the switching between shooting modes.

[0253] Furthermore, during the process of switching from the first shooting mode to the second shooting mode, the user needs to retrieve data from the Request object and the Result object respectively when using the shooting function. The focal length carried in the data attribute information (Metadata) of the Result object and the Metadata of the Request object may be different. In this case, the stability of the terminal's shooting is poor, the quality of the captured image is low, and the user experience is affected.

[0254] Correspondingly, during the transition from the first shooting mode to the second shooting mode, shooting operations are disabled to improve shooting stability. Shooting operations are only initiated after the shooting mode switch is complete, resulting in higher image quality and a better user experience.

[0255] In some embodiments, the constraint strategy determined in step S402 further includes a power consumption constraint strategy; the power consumption constraint strategy includes: when no shooting mode switching is performed, the number of physical cameras invoked is 1.

[0256] Correspondingly, the method may also include the following steps: when the shooting mode is not switched, prohibit the use of other physical cameras besides the physical camera used in the current shooting mode.

[0257] The more physical cameras a terminal activates, the longer it takes to activate multiple physical cameras, and the higher the terminal's power consumption. Therefore, without changing the zoom level, it is prohibited to call any physical cameras other than those used in the current shooting mode. For example, in the first shooting mode, only the first physical camera is used for shooting, and then other physical cameras are called. Or, in the second shooting mode, only the second physical camera is used for shooting, and then other physical cameras are called.

[0258] As can be seen from the above, in the embodiments of this application, when the shooting mode is not switched, the terminal maintains a single stream, that is, only one physical camera is called, thereby reducing the power consumption of the terminal.

[0259] In some embodiments, to make the shooting mode switching process smoother, changing the physical camera configuration during shooting mode switching is prohibited. If the physical camera configuration is changed before switching shooting modes, the physical camera configuration should be adjusted to the default configuration first, and then the shooting mode should be switched.

[0260] The following combination Figure 7 This section explains the switching process from Ultra Wide (UW) mode to Super Telephoto (ST) mode.

[0261] The ultra-wide-angle mode has an optical zoom of 0.6X, meaning the physical camera's focal length in ultra-wide-angle mode is 0.6 times the equivalent focal length. The wide-angle mode has an optical zoom of 1X, meaning the physical camera's focal length in ultra-wide-angle mode is the equivalent focal length. The telephoto mode has an optical zoom of 3X, meaning the physical camera's focal length in telephoto mode is 3 times the equivalent focal length. The super-telephoto mode has an optical zoom of 6X, meaning the physical camera's focal length in telephoto mode is 6 times the equivalent focal length.

[0262] The shooting mode was switched from ultra-wide-angle mode to super telephoto mode, which means the focal length was adjusted from 0.6X to 6X. Figure 7 The focal lengths between 0.6 and 6 are intermediate focal lengths, such as 0.6375, 0.75, ..., 5.1375, etc. These intermediate focal lengths are based on... Figure 6a The Zoom Ratio function shown determines this. Figure 7 The specific focal length is expressed using multiples of the equivalent focal length. For example, 0.6 represents 0.6 times the equivalent focal length; 0.6375 represents 0.6375 times the equivalent focal length, ..., 5.1375 represents 5.1375 times the equivalent focal length, and so on.

[0263] The mode identifier for each focal length indicates which physical camera is used for shooting at that focal length. 0.6 corresponds to UW, meaning the physical camera corresponding to the ultra-wide-angle mode is used. Therefore, it can be seen that only one physical camera is used for shooting in ultra-wide-angle mode.

[0264] 0.6375, 0.75, and 0.9375 correspond to UW and W, respectively, indicating that the physical camera corresponding to the ultra-wide-angle mode and the physical camera corresponding to the wide-angle mode are used for shooting. This shows that when switching from ultra-wide-angle mode to wide-angle mode, the physical camera corresponding to the wide-angle mode is activated in advance at a focal length of 0.6375, and both the physical cameras corresponding to the ultra-wide-angle mode and the wide-angle mode are used for shooting.

[0265] 1.2, 1.5375, and 1.95 correspond to W and T, indicating that the physical camera corresponding to the wide-angle mode and the physical camera corresponding to the telephoto mode are used for shooting. This shows that when switching from wide-angle mode to telephoto mode, the physical camera corresponding to the telephoto mode is activated in advance at a focal length of 1.2, and both the physical cameras corresponding to the wide-angle mode and the telephoto mode are used for shooting.

[0266] 2.4375, 3, 3.6375, 4.35, and 5.1375 correspond to T and ST, respectively, indicating that the physical camera corresponding to the wide-angle mode and the physical camera corresponding to the telephoto mode are used for shooting. This shows that when switching from telephoto mode to super-telephoto mode, the physical camera corresponding to the super-telephoto mode is activated in advance at a focal length of 2.4375, and both the physical cameras corresponding to the telephoto mode and the super-telephoto mode are used for shooting.

[0267] The symbol 6 corresponds to ST, indicating that the physical camera corresponding to the ultra-wide-angle mode is used for shooting. This shows that in super-telephoto mode, only one corresponding physical camera is used for shooting.

[0268] The Snapshot value indicates whether the user is allowed to use the shooting function at this focal length. A Snapshot value of 1 indicates that the user is allowed to use the shooting function. A Snapshot value of 0 indicates that the user is prohibited from using the shooting function. Figure 7 As can be seen, during the zoom process, the user is prohibited from using the shooting function through stability constraint strategy and smoothness constraint strategy.

[0269] When the focal length is less than 1X, use non-Wide display. That is, when the focal length is less than 1X, the image taken in ultra-wide-angle mode is displayed before switching to wide-angle mode.

[0270] When the focal length is less than 3X, use non-Tele display. That is, when the focal length is less than 3X, the image taken in wide-angle mode is displayed before switching to telephoto mode.

[0271] When the focal length is less than 6X, non-Super Tele display is used. That is, when the focal length is less than 6X, the image taken in super telephoto mode is displayed before switching to super telephoto mode.

[0272] When switching from ultra-wide-angle mode to super-telephoto mode, you need to switch from ultra-wide-angle mode to wide-angle mode first, then from wide-angle mode to telephoto mode, and then from telephoto mode to super-telephoto mode.

[0273] When switching from ultra-wide-angle mode to wide-angle mode, the 3A consistency constraint strategy and the SAT constraint strategy are enabled. Specifically, during the period when the focal length is adjusted from 0.6375 to 0.9375, the physical camera used for wide-angle mode is activated in advance. This allows the physical camera used for wide-angle mode to perform 3A convergence during this period, and the SAT algorithm is enabled. Furthermore, the three frames captured by the physical camera used for wide-angle mode as an auxiliary path during this period are not displayed. After switching to wide-angle mode, i.e., when the focal length is 1.2, the use of the physical camera for ultra-wide-angle mode is stopped.

[0274] Furthermore, when switching from ultra-wide-angle mode to wide-angle mode, a center point constraint strategy needs to be enabled. During the period when the focal length was adjusted from 0.9375 to 1.2, the physical camera used in ultra-wide-angle mode was switched to the physical camera used in wide-angle mode, that is, switching to wide mode during zoom changes.

[0275] When switching from wide-angle mode to telephoto mode, the 3A consistency constraint strategy and the SAT constraint strategy are enabled. Specifically, during the period when the focal length is adjusted from 1.2 to 1.95, the physical camera used for telephoto mode is activated in advance. This allows the physical camera used for telephoto mode to perform 3A convergence during this period, and the SAT algorithm is enabled. Furthermore, the three frames captured by the physical camera used for telephoto mode as an auxiliary path during this period are not displayed. After switching to telephoto mode, i.e., when the focal length is 2.4375, the physical camera used for wide-angle mode is no longer activated.

[0276] Furthermore, when switching from wide-angle mode to telephoto mode, a center point constraint strategy needs to be enabled. During the period when the focal length is adjusted from 1.95 to 2.4375, the physical camera used in wide-angle mode is switched to the physical camera used in telephoto mode, that is, switching to telephoto during zoom changes.

[0277] During the adjustment of the focal length from 2.4375 to 5.1375, a smoothness constraint strategy was enabled, maintaining the default settings throughout the transition. This means that the physical camera configuration was modified during the transition.

[0278] During the adjustment of the focal length from 3 to 6, a smoothness constraint strategy and a power consumption constraint strategy are enabled. This means that unnecessary auxiliary physical cameras are turned off in a timely manner to improve the smoothness of shooting mode switching and reduce the power consumption of the terminal.

[0279] After adjusting the focal length to 6, stability and smoothness constraint strategies are enabled. This means that the user's shooting function is disabled during zooming, and only one physical camera is used to improve the smoothness of shooting mode switching and reduce the power consumption of the terminal.

[0280] The following combination Figure 8 This section explains the switching process from Ultra Wide (UW) mode to Super Telephoto (ST) mode.

[0281] The ultra-wide-angle mode has an optical zoom of 0.6X, meaning the physical camera's focal length in ultra-wide-angle mode is 0.6 times the equivalent focal length. The wide-angle mode has an optical zoom of 1X, meaning the physical camera's focal length in ultra-wide-angle mode is the equivalent focal length. The telephoto mode has an optical zoom of 3X, meaning the physical camera's focal length in telephoto mode is 3 times the equivalent focal length. The super-telephoto mode has an optical zoom of 6X, meaning the physical camera's focal length in telephoto mode is 6 times the equivalent focal length.

[0282] The shooting mode was switched from super telephoto mode to ultra-wide-angle mode, which means the focal length was adjusted from 6X to 0.6X. Figure 8 The focal lengths between 6 and 0.6 are considered intermediate focal lengths, such as 5.9625, 5.85, 5.6625, ..., 1.4625, etc. These intermediate focal lengths are based on... Figure 6b The Zoom Ratio function shown determines this. Figure 8 The specific focal length is expressed using multiples of the equivalent focal length. For example, 6 represents 6 times the equivalent focal length; 5.9625 represents 5.9625 times the equivalent focal length, ..., 1.4625 represents 1.4625 times the equivalent focal length, and so on.

[0283] The mode identifier for each focal length indicates which physical camera is used for shooting at that focal length. 6 corresponds to ST, indicating that the physical camera corresponding to the super telephoto mode is used. Therefore, it can be seen that only one physical camera is used for shooting in super telephoto mode.

[0284] 5.9625, 5.85, and 5.6625 correspond to T and ST, respectively, indicating that the physical camera corresponding to the super telephoto mode and the physical camera corresponding to the telephoto mode are used for shooting. This shows that when switching from super telephoto mode to telephoto mode, the physical camera corresponding to the telephoto mode is activated in advance at a focal length of 5.9625, and both the physical cameras corresponding to the super telephoto mode and the telephoto mode are used for shooting.

[0285] 5.4, ​​5.0625, 4.65, 4.1625, and 3.6 correspond to W and T, indicating that the physical camera corresponding to the wide-angle mode and the physical camera corresponding to the telephoto mode are used for shooting. This shows that when switching from telephoto mode to wide-angle mode, the physical camera corresponding to the wide-angle mode is activated in advance at a focal length of 5.4, and both the physical cameras corresponding to the wide-angle mode and the physical cameras corresponding to the telephoto mode are used for shooting.

[0286] 2.9625, 2.25, and 1.4625 correspond to W and UW, respectively, indicating that the physical camera corresponding to the wide-angle mode and the physical camera corresponding to the ultra-wide-angle mode are used for shooting. This shows that when switching from wide-angle mode to ultra-wide-angle mode, at a focal length of 2.9625, the physical camera corresponding to the ultra-wide-angle mode is activated in advance, and both the physical cameras corresponding to the wide-angle mode and the ultra-wide-angle mode are used for shooting.

[0287] 0.6 corresponds to UW, indicating that the physical camera corresponding to the ultra-wide-angle mode is used for shooting. This shows that only one physical camera is used for shooting in ultra-wide-angle mode.

[0288] in addition, Figure 8In order to meet the 3A and SAT constraint strategies, the focal length needs to be maintained for several frames in super telephoto mode. These frames can be set separately before the start of the shooting mode switch, or they can be achieved through Zoom Ratio. For example, Figure 8 The 6 in parentheses after 5.9625, 5.85, and 5.6625 indicates that the focal length can also be maintained at 6 using Zoom Ratio.

[0289] The Snapshot value indicates whether the user is allowed to use the shooting function at this focal length. A Snapshot value of 1 indicates that the user is allowed to use the shooting function. A Snapshot value of 0 indicates that the user is prohibited from using the shooting function. Figure 8 As can be seen, during the zoom process, the user is prohibited from using the shooting function through stability constraint strategy and smoothness constraint strategy.

[0290] In a specific implementation, this application also provides a terminal, which includes one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, which includes computer instructions, and one or more processors call the computer instructions to cause the terminal to perform some or all of the steps in the above method embodiments.

[0291] This application also provides a computer-readable storage medium including a computer program that, when run on a terminal, causes the terminal to perform some or all of the steps described in the method embodiments. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0292] In a specific implementation, this application also provides a computer program product, which includes executable instructions. When the executable instructions are executed on a terminal, the terminal performs some or all of the steps in the above method embodiments.

[0293] like Figure 9 As shown, this application also provides a chip system applied to a terminal. The chip system includes one or more processors 901. The processors 901 are used to call computer instructions to cause the terminal to input data to be processed into the chip system. The chip system processes the data based on the image display method provided in the embodiments of this application and outputs the processing result.

[0294] In one possible implementation, the chip system also includes input and output interfaces for inputting and outputting data.

[0295] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0296] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

[0297] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0298] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory, random access memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0299] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0300] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0301] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0302] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. An image display method, characterized in that, The method is applied to a terminal, and the method includes: In response to the shooting mode switching operation, based on the current first shooting mode and the second shooting mode to be switched to, a first constraint strategy for the image frame to be displayed, a second constraint strategy for calling the physical camera, a third constraint strategy for the image processing method, and a fourth constraint strategy for the intermediate focal length to be shot are determined when switching shooting modes. According to the fourth constraint strategy, in the first correspondence between the preset relative time and focal length, multiple intermediate focal lengths for shooting during the shooting mode switching process are determined; wherein, the relative time is the time of shooting relative to the time of starting the shooting mode switching; the multiple intermediate focal lengths are different. According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take pictures at that intermediate focal length; wherein, the target physical camera is the physical camera in the first physical camera used in the first shooting mode and the second physical camera used in the second shooting mode. According to the third constraint strategy and the first constraint strategy, a target image frame that meets the preset conditions is determined from the image frames captured by the target physical camera, and the target image frame is displayed in a preset preview area; wherein, the clarity of the target image frame is higher than that of other image frames; The following steps are used to detect whether the captured image frames meet the preset conditions: For each relative moment, the image metadata acquired at that relative moment according to the corresponding intermediate focal length is cached in the first storage area; According to the order in which the image metadata is cached, the image metadata is retrieved from the first storage area in sequence, processed to obtain the corresponding image frame, and the obtained image frame is stored in the second storage area. When the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it is determined that the image frame captured at that relative moment meets the preset condition.

2. The method according to claim 1, characterized in that, The step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and then calling the target physical camera to take a picture at that intermediate focal length includes: Upon reaching each first relative moment, the focal length is adjusted to the intermediate focal length corresponding to that first relative moment, and the first physical camera is invoked to take a picture at that intermediate focal length; the first relative moment is the relative moment before the second physical camera is invoked. After the number of captured image frames reaches a first number, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the first physical camera and the second physical camera are called to take pictures according to that intermediate focal length; the second relative moment is the relative moment after the second physical camera is called. The method further includes: When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting by the first physical camera is stopped.

3. The method according to claim 2, characterized in that, The first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames that are delayed before being displayed before the shooting mode switch is completed; The step of determining a target image frame that meets preset conditions from image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes: According to the 3A consistency constraint strategy, the number of image frames delayed in display during the shooting mode switching process is determined as the second number. For each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, the image frame captured at the first relative moment is displayed as the target image frame in the preset preview area. If the image frame captured at the first relative moment does not meet the preset conditions, a second number of target image frames captured before the first relative moment that meet the preset conditions are determined, and the determined target image frames are displayed in the preset preview area.

4. The method according to claim 3, characterized in that, The third constraint strategy includes: the SAT constraint strategy; the SAT constraint strategy includes: pixel offset between the center points of images captured by different physical cameras; The step of determining a target image frame that meets preset conditions from image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes: According to the SAT strategy, calculate the pixel offset between the center point of the image frame captured by the first physical camera and the center point of the image frame captured by the second physical camera; Based on the pixel offset, the image frames captured by the second physical camera are processed; For each second relative time, if the image frame captured at that second relative time meets the preset conditions, the image frame captured at that second relative time is displayed as the target image frame in the preset preview area. If the image frame captured at the second relative moment does not meet the preset conditions, determine the second number of target image frames that meet the preset conditions and were captured by the first physical camera before the second relative moment, and display the determined target image frames in the preset preview area.

5. The method according to claim 1, characterized in that, The first storage area is the storage area corresponding to the Result object called by the Pipeline image processing method; the second storage area is the storage area corresponding to the Request object called by the Pipeline image processing method.

6. The method according to claim 2, characterized in that, The second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between the physical camera and the shooting mode in the terminal; Before invoking the second physical camera to take a picture, the method further includes: According to the focal length constraint strategy, the physical camera corresponding to the second shooting mode is determined from the physical cameras in the terminal and used as the second physical camera.

7. The method according to claim 1, characterized in that, The fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used by different shooting modes during the shooting mode switching process. According to the fourth constraint strategy, in the preset first correspondence between relative time and focal length, multiple intermediate focal lengths for shooting during the shooting mode switching process are determined, including: According to the center point constraint strategy, the number of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode is determined as the third number. Based on the first correspondence between the preset relative time and focal length, the third number of intermediate focal lengths are determined.

8. The method according to claim 7, characterized in that, After determining a third number of intermediate focal lengths in the first correspondence between preset relative time and focal length, the method further includes: Based on the third number of multiple intermediate focal lengths and the number of image frames required to be captured during the shooting mode switching process, the number of image frames required to be captured at each intermediate focal length is calculated as the fourth number. The step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and then calling the target physical camera to take a picture at that intermediate focal length includes: At each relative moment, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called to take the fourth number of image frames at that intermediate focal length.

9. The method according to claim 2, characterized in that, The relationship between relative time and focal length in the first correspondence is a quadratic curve, or the relationship between relative time and focal length in the first correspondence is a Bézier curve.

10. The method according to any one of claims 1 to 9, characterized in that, The determined constraint strategies also include stability constraint strategies; The stability constraint strategy includes: prohibiting the user from using the shooting function during the shooting mode switching process; The method further includes: During the transition from the first shooting mode to the second shooting mode, shooting operations are prohibited.

11. The method according to any one of claims 1 to 9, characterized in that, The determined constraint strategy also includes a power consumption constraint strategy; the power consumption constraint strategy includes: when no shooting mode switching is performed, the number of physical cameras called is 1; The method further includes: When not switching shooting modes, it is prohibited to call other physical cameras besides the physical camera called by the current shooting mode.

12. The method according to any one of claims 1 to 9, characterized in that, The method further includes: After successfully switching to the second shooting mode, the target image frame captured by the second physical camera will be displayed in the preset preview area.

13. The method according to claim 1, characterized in that, A third shooting mode is spaced between the first shooting mode and the second shooting mode; The step of adjusting the focal length to the intermediate focal length corresponding to each relative moment, and then calling the target physical camera to take a picture at that intermediate focal length includes: Upon reaching each first relative moment, the focal length is adjusted to the intermediate focal length corresponding to that first relative moment, and the first physical camera is invoked to take a picture at that intermediate focal length; the first relative moment is the relative moment before the second physical camera is invoked. After the number of captured image frames reaches a first number, at each third relative moment, the focal length is adjusted to the intermediate focal length corresponding to that third relative moment, and the first physical camera and the third physical camera used by the third shooting mode are called to shoot according to that intermediate focal length; the third relative moment is the relative moment after the third physical camera is called and before the second physical camera is called. After the number of captured image frames reaches the fifth number, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the third physical camera and the second physical camera are called to take pictures according to that intermediate focal length; the second relative moment is the relative moment after the second physical camera is called. The method further includes: When the focal length is adjusted to the focal length used in the second shooting mode, and the image frame captured by the second physical camera meets the preset conditions, the shooting by the third physical camera is stopped.

14. The method according to claim 13, characterized in that, The first shooting mode is: ultra-wide-angle mode; the second shooting mode is: super telephoto mode.

15. The method according to claim 13, characterized in that, The first shooting mode is: super telephoto mode; the second shooting mode is: ultra-wide-angle mode.

16. A terminal, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the terminal to perform the method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, Includes a computer program that, when run on a terminal, causes the terminal to perform the method of any one of claims 1-15.

18. A computer program product, characterized in that, The computer program product includes executable instructions that, when executed on a terminal, cause the terminal to perform the method of any one of claims 1-15.

19. A chip system, characterized in that, The chip system is applied to a terminal. The chip system includes one or more processors. The processors are used to call computer instructions to cause the terminal to input data into the chip system and execute the method described in any one of claims 1-15 to process the data and output the processing result.

Images