Image display method, terminal and storage medium

Abstract

The embodiment of the invention provides an image display method, a terminal and a storage medium, and relates to the technical field of terminals, and the method comprises the following steps: responding to a shooting mode switching operation, and based on a first shooting mode and a second shooting mode; determining a first constraint strategy aiming at a sent and displayed image frame, a second constraint strategy aiming at calling a physical camera, a third constraint strategy aiming at an image processing mode and a fourth constraint strategy aiming at a middle focal length for shooting when the shooting mode is switched; according to a fourth constraint strategy, determining a plurality of intermediate focal lengths in a shooting mode switching process in a preset first corresponding relationship between relative moments and focal lengths; according to a second constraint strategy, the focal length is adjusted to be a corresponding intermediate focal length when each relative moment is reached, and the target physical camera is called for shooting; and according to the third constraint strategy and the first constraint strategy, determining a target image frame from the image frames shot by the target physical camera and displaying the target image frame in the preset preview area, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image display method, a terminal, and a storage medium. Background Art

[0002] With the development of electronic technology, terminals provide more and more functions. For example, users can use the camera application in the terminal to take pictures to record wonderful moments, touching scenes and other beautiful pictures. Figure 1a , Figure 1a The terminal display interface shown in FIG includes multiple application icons. After the terminal detects that the user clicks the icon of the camera application, the terminal determines to start the camera application and displays the following Figure 1b The operation interface of the camera application shown in FIG. The operation interface includes a preset preview area 100 and a shooting mode switching area 110. The preset preview area 100 is used to display the image captured by the camera. For example, Figure 1c The preset preview area shown shows an image of a person R1. The shooting mode switching area 110 includes a plurality of mode marks. Mode marks 111, 112, 113 and 114 represent different shooting modes.

[0003] Users can Figure 1b Select different shooting modes on the operation interface shown. For example, Figure 1b The default shooting mode of the camera application is the shooting mode indicated by the mode mark 112. When it is detected that the user clicks the mode mark 111, it indicates that it is necessary to switch to the shooting mode indicated by the mode mark 111. However, when switching the shooting mode, the clarity of the image displayed in the preset preview area is low, which reduces the user experience. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide an image display method, terminal, and storage medium to improve the clarity of images displayed in a preset preview area and enhance the user experience. The specific technical solutions are as follows:

[0005] In a first aspect, to achieve the above-mentioned objectives, an embodiment of the present application provides an image display method, the method comprising:

[0006] In response to a shooting mode switching operation, determining, based on a current first shooting mode and a desired second shooting mode, a first constraint strategy for image frames sent for display, a second constraint strategy for calling a physical camera, a third constraint strategy for an image processing method, and a fourth constraint strategy for an intermediate focal length for shooting when the shooting mode is switched;

[0007] According to the fourth constraint strategy, a plurality of intermediate focal lengths for shooting during the shooting mode switching process are determined in a preset first correspondence between relative time and focal length; wherein the relative time is the time of shooting relative to the time of starting the shooting mode switching; and the plurality of intermediate focal lengths are different;

[0008] According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and a target physical camera is called to shoot at the intermediate focal length; wherein the target physical camera is a physical camera selected from the first physical camera used in the first shooting mode and the second physical camera used in the second shooting mode;

[0009] According to the third constraint strategy and the first constraint strategy, a target image frame that meets preset conditions is determined from the image frames taken by the target physical camera, and the target image frame is displayed in a preset preview area; wherein the clarity of the target image frame is higher than the clarity of other image frames.

[0010] As can be seen from the above, the solution provided in this embodiment constrains the intermediate focal length of shooting, the image frames sent for display, and the called physical camera from multiple aspects during the shooting mode switching process according to different constraint strategies, thereby improving the clarity of the captured image and displaying the target image frame that meets the preset conditions in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frame displayed in the preset preview area and enhance the user experience.

[0011] In one embodiment of the present application, when each relative moment is reached, adjusting the focal length to an intermediate focal length corresponding to the relative moment, and calling the target physical camera to shoot at the intermediate focal length, includes:

[0012] When each first relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the first relative moment, and the first physical camera is called to shoot at the intermediate focal length;

[0013] After the number of captured image frames reaches a first number, at each second relative moment, adjusting the focal length to an intermediate focal length corresponding to the second relative moment, and calling the first physical camera and the second physical camera to shoot at the intermediate focal length;

[0014] The method further comprises:

[0015] When the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by calling the second physical camera meets a preset condition, calling the first physical camera for shooting is stopped.

[0016] As can be seen from the above, in the embodiments of the present application, when switching between shooting modes and zooming, activating the second physical camera in advance as a secondary camera for shooting can speed up the shooting mode switching process and achieve a user-imperceptible physical camera switch. Furthermore, image frames that are not stabilized by the second physical camera are not displayed, and image frames captured by the first physical camera are still displayed. This ensures that the preset preview area displays the target image frames with higher definition, improving the user experience.

[0017] Moreover, when the focal length is adjusted to the focal length used in the second shooting mode and the image frame captured by the second physical camera meets the preset conditions, the second physical camera can capture images stably, and the image frame captured by the second physical camera can still be displayed in the preset preview area as a target image frame with higher clarity. In this case, the first physical camera is stopped from being called for shooting, which can reduce the power consumption of the terminal.

[0018] In one embodiment of the present application, the first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames delayed for display before completing the shooting mode switch;

[0019] The method of determining a target image frame that meets a preset condition from the image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes:

[0020] Determining, according to the 3A consistency constraint strategy, the number of image frames that are delayed in display during the shooting mode switching process as the second number;

[0021] For each first relative moment, if the image frame captured at the first relative moment meets a preset condition, the image frame captured at the first relative moment is displayed as a target image frame in a preset preview area;

[0022] If the image frame captured at the first relative moment does not meet the preset condition, a second number of target image frames captured before the first relative moment that meet the preset condition are determined, and the determined target image frames are displayed in a preset preview area.

[0023] As can be seen from the above, in the embodiment of the present application, during the process of switching the shooting mode, for each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, it is directly displayed; if the image frame captured at the first relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame captured previously is used for display, so that the target image frame displayed is a higher definition target image frame, and then the preset preview area displays a higher definition target image frame, thereby improving the user experience.

[0024] In one embodiment of the present application, the third constraint strategy includes: a SAT constraint strategy; the SAT constraint strategy includes: a pixel offset between center points of images captured by different physical cameras;

[0025] The method of determining a target image frame that meets a preset condition from the image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes:

[0026] Calculating, according to the SAT strategy, a pixel offset between a center point of an image frame captured by the first physical camera and a center point of an image frame captured by the second physical camera;

[0027] Based on the pixel offset, processing the image frame captured by calling the second physical camera;

[0028] For each second relative moment, if the image frame captured at the second relative moment meets a preset condition, the image frame captured at the second relative moment is displayed as a target image frame in a preset preview area;

[0029] If the image frame captured at the second relative moment does not meet the preset conditions, a second number of target image frames meeting the preset conditions captured by the first physical camera before the second relative moment are determined, and the determined target image frames are displayed in a preset preview area.

[0030] As can be seen from the above, in the embodiment of the present application, during the process of switching the shooting mode, for each second relative moment, if the image frame taken at the second relative moment meets the preset conditions, it is directly displayed; if the image frame taken at the second relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame taken by the first physical camera is used for display, so that the target image frame displayed is a higher definition target image frame, and then the preset preview area displays a higher definition target image frame, thereby improving the user experience.

[0031] In addition, a second number (for example, 3) frames need to be inserted between the currently called first physical camera and the second physical camera to be switched. The second number of frames uses the image frames taken by the first physical camera for display, that is, the second physical camera is started the second number of frames in advance. On the one hand, the second physical camera is allowed to perform 3A convergence to avoid the problem of excessive scene difference before and after 3A synchronization switching, and further converge several frames on the basis of 3A synchronization to achieve the optimal effect of the image frames for display; on the other hand, the SAT calculation is triggered within the second number of frames to achieve the optimal effect of the center point offset between the physical camera switches, so that the user can complete the shooting mode switch without noticing, thereby improving the user experience.

[0032] In one embodiment of the present application, the following steps are performed to detect whether the captured image frame meets the preset conditions:

[0033] For each relative moment, cache the image metadata collected at the relative moment according to the corresponding intermediate focal length in the first storage area;

[0034] sequentially fetching the image metadata from the first storage area in the order in which the image metadata are cached, processing the metadata to obtain corresponding image frames, and storing the obtained image frames in the second storage area;

[0035] When the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it is determined that the image frame captured at the relative moment meets the preset condition.

[0036] As can be seen from the above, the solution provided by this embodiment determines whether the physical camera is stably capturing images by detecting whether the image frame meets the preset conditions. Subsequently, the target image frame that meets the preset conditions is displayed in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frame displayed in the preset preview area and enhance the user experience.

[0037] In one embodiment of the present application, the first storage area is a storage area corresponding to the Result object called by the Pipeline image processing method; the second storage area is a storage area corresponding to the Request object called by the Pipeline image processing method.

[0038] In one embodiment of the present application, the second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between a physical camera in the terminal and a shooting mode;

[0039] Before calling the second physical camera to shoot, the method further includes:

[0040] According to the focal length constraint strategy, a physical camera corresponding to the second shooting mode is determined from the physical cameras in the terminal as the second physical camera.

[0041] As can be seen from the above, the solution provided in this embodiment provides users with different shooting modes by calling different physical cameras, thereby improving user experience.

[0042] In one embodiment of the present application, the fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used in different shooting modes during the shooting mode switching process;

[0043] The determining, according to the fourth constraint strategy, a plurality of intermediate focal lengths for shooting during the shooting mode switching process in the first correspondence between the preset relative time and the focal length includes:

[0044] determining, according to the center point constraint strategy, a number of a plurality of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode as a third number;

[0045] In the preset first correspondence between relative time and focal length, a third number of intermediate focal lengths are determined.

[0046] As can be seen from the above, in the embodiment of the present application, a fourth number of image frames are captured at various intermediate focal lengths. Subsequently, the target image frame from the fourth number of image frames captured at various intermediate focal lengths can be displayed in a preset preview area, so that the preset preview area displays the target image frame with higher clarity, thereby improving the user experience.

[0047] In one embodiment of the present application, after determining a third number of intermediate focal lengths in the first correspondence between the preset relative moments and the focal lengths, the method further includes:

[0048] Calculating the number of image frames required to be captured at each intermediate focal length as a fourth number based on a third number of the plurality of intermediate focal lengths and the number of image frames required to be captured during the shooting mode switching process;

[0049] When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length, including:

[0050] When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to capture a fourth number of image frames at the intermediate focal length.

[0051] As can be seen from the above, in the embodiment of the present application, a fourth number of image frames are captured at various intermediate focal lengths. Subsequently, the target image frame from the fourth number of image frames captured at various intermediate focal lengths can be displayed in a preset preview area, so that the preset preview area displays the target image frame with higher clarity, thereby improving the user experience.

[0052] In one embodiment of the present application, the relationship between the change of the relative time and the focal length in the first corresponding relationship is a quadratic curve, or the relationship between the change of the relative time and the focal length in the first corresponding relationship is a Bezier curve.

[0053] As can be seen from the above, in the embodiment of the present application, the changing trend of the relative moment and focal length in the first corresponding relationship is a curve that is slow at first and then fast. The initial changing trend of the changing relationship of the relative moment and focal length in the first corresponding relationship is relatively slow, and the corresponding field of view angle of the physical camera also changes relatively slowly. The time for preparation before switching the physical camera can be increased, and the physical camera to be switched to can be started first as an auxiliary road. The subsequent changing trend of the changing relationship of the relative moment and focal length in the first corresponding relationship is relatively fast, and the field of view angle changes relatively greatly. When switching the physical camera at this time, even if there is a slight offset between the physical cameras, the user's perception is not strong, thereby improving the user experience.

[0054] In one embodiment of the present application, the determined constraint strategy further includes a stability constraint strategy; the stability constraint strategy includes: during the switching of the shooting mode, prohibiting the user from using the shooting function;

[0055] The method further comprises:

[0056] During the switching from the first shooting mode to the second shooting mode, responding to the shooting operation is prohibited.

[0057] As can be seen from the above, in the embodiment of the present application, during the process of switching from the first shooting mode to the second shooting mode, the shooting operation is prohibited, thereby improving the stability of shooting. After the shooting mode switch is completed, the shooting operation is responded to, and the quality of the captured image is higher, thereby improving the user experience.

[0058] In one embodiment of the present application, the determined constraint strategy further includes a power consumption constraint strategy; the power consumption constraint strategy includes: when the shooting mode is not switched, the number of physical cameras called is 1;

[0059] The method further comprises:

[0060] When the shooting mode is not switched, it is prohibited to call other physical cameras except the physical camera called by the current shooting mode.

[0061] As can be seen from the above, in the embodiment of the present application, when the shooting mode is not switched, the terminal maintains a single stream, that is, only one physical camera is called to reduce the power consumption of the terminal.

[0062] In one embodiment of the present application, the method further includes:

[0063] After successfully switching to the second shooting mode, the target image frame shot by the second physical camera is called to be displayed in a preset preview area.

[0064] As can be seen from the above, in the embodiment of the present application, after successfully switching from the first shooting mode to the second shooting mode, the switching of the physical camera and the adjustment of the focal length have been completed, and the camera can stably capture image frames. At this time, the clarity of the captured image frames is relatively high. The target image frames captured in the second shooting mode are displayed in the preset preview area, which can improve the clarity of the image frames displayed in the preset preview area and improve the user experience.

[0065] In one embodiment of the present application, the first shooting mode and the second shooting mode are separated by a third shooting mode;

[0066] When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length, including:

[0067] When each first relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the first relative moment, and the first physical camera is called to shoot at the intermediate focal length;

[0068] After the number of captured image frames reaches a first number, at each third relative moment, adjusting the focal length to an intermediate focal length corresponding to the third relative moment, and calling the first physical camera and the third physical camera used in the third shooting mode to shoot at the intermediate focal length;

[0069] After the number of captured image frames reaches a fifth number, at each second relative moment, adjusting the focal length to an intermediate focal length corresponding to the second relative moment, and calling the third physical camera and the second physical camera to shoot at the intermediate focal length;

[0070] The method further comprises:

[0071] When the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by calling the second physical camera meets a preset condition, calling the third physical camera for shooting is stopped.

[0072] As can be seen from the above, in the embodiment of the present application, when switching between shooting modes and zooming, the physical camera used in the next shooting mode is activated in advance as an auxiliary channel for shooting, which can speed up the shooting mode switching and achieve the user-unnoticed switching of physical cameras. In addition, image frames that are not stabilized by the physical camera are not displayed, and image frames captured by the physical camera used in the previous shooting mode are still displayed. This ensures that the preset preview area displays the target image frames with higher clarity, thereby improving the user experience.

[0073] Moreover, when the focal length is adjusted to the focal length used in the next shooting mode and the image frame captured by the physical camera used in the next shooting mode meets the preset conditions, the physical camera used in the next shooting mode can capture images stably, and the image frame captured by the physical camera used in the next shooting mode can still be displayed in the preset preview area as a target image frame with higher clarity. Then, the physical camera used in the previous shooting mode is stopped from being called for shooting, which can reduce the power consumption of the terminal.

[0074] In one embodiment of the present application, the first shooting mode is an ultra-wide-angle mode; and the second shooting mode is an ultra-telephoto mode.

[0075] In one embodiment of the present application, the first shooting mode is an ultra-telephoto mode; and the second shooting mode is an ultra-wide-angle mode.

[0076] In a second aspect, an embodiment of the present application further provides a terminal, including:

[0077] one or more processors and memory;

[0078] The memory is coupled to the one or more processors, and is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the terminal to execute any one of the above-mentioned image display methods.

[0079] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, comprising a computer program, which, when executed on a terminal, enables the terminal to execute any one of the above-described image display methods.

[0080] In a fourth aspect, an embodiment of the present application further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a terminal, the terminal executes any one of the above-mentioned image display methods.

[0081] In the fifth aspect, an embodiment of the present application also provides a chip system, which is applied to a terminal. The chip system includes one or more processors, and the processor is used to call computer instructions so that the terminal inputs data into the chip system, and executes any of the above-mentioned image display methods to process the data and output the processing results.

[0082] The beneficial effects of the solutions provided by the embodiments in the second, third, fourth and fifth aspects can be referred to the beneficial effects of the solutions provided by the embodiments in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0084] Figure 1a A schematic diagram of a user interface of a terminal provided in an embodiment of the present application;

[0085] Figure 1b A schematic diagram of a user interface of a camera application provided in an embodiment of the present application;

[0086] Figure 1c A schematic diagram of a first shooting scenario provided in an embodiment of the present application;

[0087] Figure 2 A structural diagram of a terminal provided in an embodiment of the present application;

[0088] Figure 3 A software structure diagram of a terminal provided in an embodiment of the present application;

[0089] Figure 4 A flowchart of an image display method provided in an embodiment of the present application;

[0090] Figure 5a A schematic diagram of a second shooting scenario provided in an embodiment of the present application;

[0091] Figure 5b A schematic diagram of a third shooting scenario provided in an embodiment of the present application;

[0092] Figure 5c A schematic diagram of a fourth shooting scenario provided in an embodiment of the present application;

[0093] Figure 5d A schematic diagram of a fifth shooting scenario provided in an embodiment of the present application;

[0094] Figure 6a A first graph of relative time and focal length provided in an embodiment of the present application;

[0095] Figure 6b A second graph of relative time and focal length provided in an embodiment of the present application;

[0096] Figure 7 A schematic diagram of the first shooting mode switching data provided in an embodiment of the present application;

[0097] Figure 8 A schematic diagram of the second shooting mode switching data provided in an embodiment of the present application;

[0098] Figure 9 A structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0100] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first instruction and the second instruction are intended to distinguish different user instructions and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0101] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0102] The image display method provided in the embodiments of the present application is applied to a terminal. The terminal may be a mobile phone, tablet computer, laptop computer, personal digital assistant (PDA), smart watch, netbook, wearable electronic device, augmented reality (AR) device, virtual reality (VR) device, vehicle-mounted device, smart car, robot, smart glasses, smart TV, or other terminal equipped with a camera. In this way, the terminal can display the captured image in a preset preview area when switching the shooting mode according to the method provided in the embodiments of the present application.

[0103] For example, Figure 2The structure of the terminal 200 is shown. The terminal 200 may include a processor 210, a display screen 220, a camera 230, an internal memory 240, a Subscriber Identification Module (SIM) card interface 250, a Universal Serial Bus (USB) interface 260, a charging management module 270, a battery management module 271, a battery 272 with a cell and a battery protection device, a sensor module 280, a mobile communication module 290, a wireless communication module 300, an antenna 1, and an antenna 2. The sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, a touch sensor 280C, an ambient light sensor 280D, and the like.

[0104] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on terminal 200. In other embodiments of the present application, terminal 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] The processor 210 may include one or more processing units. For example, the processor 210 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent components or integrated into one or more processors. In some embodiments, the terminal 200 may also include one or more processors 210. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. In other embodiments, the processor 210 may also include a memory for storing instructions and data. For example, the memory in the processor 210 may be a cache memory. This memory may store instructions or data that have just been used or are being recycled by the processor 210. If the processor 210 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the terminal 200 in processing data or executing instructions.

[0106] In some embodiments, the processor 210 may include one or more interfaces. The interfaces may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a SIM card interface, and / or a USB interface. The USB interface 260 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 260 may be used to connect a charger to charge the terminal 200, or to transfer data between the terminal 200 and peripheral devices. The USB interface 260 may also be used to connect headphones to play audio through the headphones.

[0107] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is for illustrative purposes only and does not constitute a structural limitation on the terminal 200. In other embodiments of the present application, the terminal 200 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0108] The wireless communication function of the terminal 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 290, the wireless communication module 300, the modem processor, and the baseband processor.

[0109] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in Terminal 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, Antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0110] Terminal 200 implements display functions through a GPU, display screen 220, and an application processor. The GPU is a microprocessor for image processing that connects display screen 220 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or modify display information.

[0111] The display screen 220 is used to display images, videos, etc. The display screen 220 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-o-LED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 200 may include one or more display screens 220.

[0112] In some embodiments of the present application, when the display panel adopts materials such as OLED, AMOLED, FLED, etc., the above Figure 2 The display screen 220 can be bent. Here, the display screen 220 can be bent to any angle at any position and can be maintained at that angle. For example, the display screen 220 can be folded in half from the middle to the left or right. It can also be folded in half from the middle to the top or bottom.

[0113] The display screen 220 of the terminal 200 may be a flexible screen. Currently, flexible screens have attracted much attention due to their unique characteristics and huge potential. Compared with traditional screens, flexible screens are more flexible and bendable, which can provide users with a new way of interaction based on the bendable characteristics, and can meet more user demands for the terminal. For terminals equipped with a foldable display, the foldable display on the terminal can be switched between a small screen in a folded form and a large screen in an unfolded form at any time. Therefore, users are using the split-screen function on terminals equipped with a foldable display more and more frequently.

[0114] The terminal 200 can implement a shooting function through an ISP, a camera 230, a video codec, a GPU, a display screen 220, and an application processor, wherein the camera 230 includes a front camera and a rear camera.

[0115] The ISP processes data fed back by camera 230. For example, when shooting, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 230.

[0116] The camera 230 is used to take photos or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red, green, blue (RGB), YUV, or other format. In some embodiments, the terminal 200 may include 1 or N cameras 230, where N is a positive integer greater than 1.

[0117] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal 200 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0118] Video codecs are used to compress or decompress digital video. Terminal 200 may support one or more video codecs. This allows Terminal 200 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0119] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in the terminal 200, such as image recognition, face recognition, speech recognition, and text comprehension.

[0120] The internal memory 240 can be used to store one or more computer programs, which include instructions. The processor 210 can execute the above instructions stored in the internal memory 240, so that the terminal 200 performs the image display method provided in some embodiments of the present application, as well as various applications and data processing. The internal memory 240 may include a program storage area and a data storage area. The program storage area may store an operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data created during the use of the terminal 200 (such as photos, contacts, etc.). In addition, the internal memory 240 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, a flash memory component, a universal flash memory (UFS), etc. In some embodiments, the processor 210 can execute the image display method provided in the embodiments of the present application, as well as other applications and data processing by executing the instructions stored in the internal memory 240 and / or the instructions stored in the memory provided in the processor 210.

[0121] The internal memory 240 can be used to store the relevant programs of the image display method provided in the embodiment of the present application, and the processor 210 can be used to call the relevant programs of the image display method stored in the internal memory 240 when displaying information to execute the image display method of the embodiment of the present application.

[0122] The sensor module 280 may include a pressure sensor 280A, a fingerprint sensor 280B, a touch sensor 280C, an ambient light sensor 280D, and the like.

[0123] Pressure sensor 280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 280A can be located on display screen 220. There are many types of pressure sensors 280A, including resistive, inductive, and capacitive pressure sensors. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force is applied to pressure sensor 280A, the capacitance between the electrodes changes, and terminal 200 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to display screen 220, terminal 200 detects the touch operation based on pressure sensor 280A. Terminal 200 can also calculate the touch location based on the detection signal from pressure sensor 280A. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed; when a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0124] The fingerprint sensor 280B is used to collect fingerprints. The terminal 200 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0125] Touch sensor 280C, also known as a touch-sensitive device, can be disposed on display screen 220. Touch sensor 280C and display screen 220 form a touch screen, also known as a touchscreen. Touch sensor 280C is configured to detect touch operations applied thereto or in the vicinity thereof. Touch sensor 280C can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via display screen 220. In other embodiments, touch sensor 280C can also be disposed on the surface of terminal 200, at a different location from display screen 220.

[0126] Ambient light sensor 280D is used to sense ambient light brightness. Terminal 200 can adaptively adjust the brightness of display screen 220 based on the perceived ambient light brightness. Ambient light sensor 280D can also be used to automatically adjust white balance during photography. Ambient light sensor 280D can also transmit information about the device's surroundings to the GPU.

[0127] The ambient light sensor 280D is also used to obtain the brightness, light ratio, color temperature, etc. of the acquisition environment in which the camera 230 captures images.

[0128] Figure 3The present invention is a software structure block diagram of a terminal applicable to an embodiment of the present application. The terminal software system can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The layered architecture divides the terminal software system into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into three layers, namely, the application layer (applications), the application framework layer (application framework) and the driver layer (hardware abstract layer, HAL).

[0129] The application layer can include a series of application packages, and the application layer runs applications by calling the application programming interface (API) provided by the application framework layer. Figure 3 As shown, the application package may include multiple applications, such as camera, gallery, browser, music, etc. It can be understood that the port of each of the above applications can be used to receive data.

[0130] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, and a Dynamic Host Configuration Protocol (DHCP) module.

[0131] The driver layer is the layer between hardware and software, responsible for driving the hardware and making it work. The driver layer can contain multiple drivers for driving the hardware. For example, there are drivers for cameras, displays, audio, and sensors.

[0132] In addition, the terminal also includes a hardware layer, which may include a camera, a speaker, a CPU, and an NPU, etc. The hardware layer is connected to the driver layer.

[0133] In an embodiment of the present application, after starting the camera application of the application layer, the camera driver of the driver layer calls the camera of the hardware layer (i.e., the physical camera) to capture images, and the image captured by the camera is displayed in the preset preview area of the display screen.

[0134] Next, the image display solution provided by the embodiment of the present application is described in detail through specific embodiments.

[0135] In one embodiment of the present application, see Figure 4, Figure 4 A flowchart of an image display method provided in an embodiment of the present application, the method comprising the following steps:

[0136] S401: In response to the shooting mode switching operation, based on the current first shooting mode and the second shooting mode to be switched to, determine the first constraint strategy for the image frame sent for display when the shooting mode is switched, the second constraint strategy for calling the physical camera, the third constraint strategy for the image processing method, and the fourth constraint strategy for the intermediate focal length for shooting.

[0137] S402: According to the fourth constraint strategy, a plurality of intermediate focal lengths for shooting during the shooting mode switching process are determined in a preset first correspondence between relative time and focal length.

[0138] The relative time is the time when the shooting is performed relative to the time when the shooting mode switching starts; and the multiple intermediate focal lengths are different.

[0139] S403: According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to the intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length.

[0140] The target physical camera is a physical camera in the first physical camera used in the first shooting mode and a second physical camera used in the second shooting mode.

[0141] S404: According to the third constraint strategy and the first constraint strategy, a target image frame that meets a preset condition is determined from the image frames captured by the target physical camera, and the target image frame is displayed in a preset preview area.

[0142] The definition of the target image frame is higher than that of other image frames.

[0143] As can be seen from the above, the solution provided in this embodiment constrains the intermediate focal length of shooting, the image frames sent for display, and the called physical camera from multiple aspects during the shooting mode switching process according to different constraint strategies, thereby improving the clarity of the captured image and displaying the target image frame that meets the preset conditions in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frame displayed in the preset preview area and enhance the user experience.

[0144] With respect to step S401 , a camera application for shooting is installed in the terminal, and the camera application provides multiple shooting modes.

[0145] Shooting modes include: Ultra Wide (UW) mode, Wide (W) mode, Tele (T) mode, and Super Tele (ST) mode. Among them, the wide-angle mode can also be called the main shooting mode. When in wide-angle mode, the focal length of the physical camera is the equivalent focal length. The equivalent focal length is the first value. For example, the first value is 27mm. For example, Figure 5a The medium mode identifier 112 indicates the wide angle mode. The focal length of the physical camera used in other shooting modes can be determined based on the equivalent focal length.

[0146] When in ultra-wide-angle mode, the focal length of the physical camera is the first multiple of the equivalent focal length. For example, Figure 5a The medium mode identifier 111 indicates the ultra-wide-angle mode. In the ultra-wide-angle mode, the focal length of the physical camera is 0.6 times the equivalent focal length. Figure 5a The above is only an example, and the first magnification can also be set to other values as needed. For example, in ultra-wide-angle mode, the focal length of the physical camera can also be 0.5 times the equivalent focal length.

[0147] When in telephoto mode, the focal length of the physical camera is the second multiple of the equivalent focal length. For example, Figure 5a The medium mode identifier 113 indicates the telephoto mode. In the telephoto mode, the focal length of the physical camera is three times the equivalent focal length. Figure 5a The above is only an example, and the second multiple can also be set to other values as needed. For example, in telephoto mode, the focal length of the physical camera can also be 2.5 times the equivalent focal length.

[0148] In super telephoto mode, the focal length of the physical camera is a multiple of the third of the equivalent focal length. For example, Figure 5a The medium mode identifier 114 indicates the super telephoto mode. In the super telephoto mode, the focal length of the physical camera is 6 times the equivalent focal length. Figure 5a The third factor shown is only an example, and can be set to other values as needed. For example, in super telephoto mode, the focal length of the physical camera can also be 5 times the equivalent focal length.

[0149] The default shooting mode of the camera application after startup is the main shooting mode (i.e. wide-angle mode). Figure 5a As shown, when the shooting mode of the camera application is the wide-angle mode, due to the limited field of view, only the complete image of the person R1 can be captured, but the complete images of the persons R2 and R3 cannot be captured.

[0150] When it is necessary to capture a complete image containing the characters R1, R2, and R3, the focal length of the physical camera needs to be reduced. For example, the camera switches to the ultra-wide-angle mode. The user can click the mode icon 111. In response to the user's shooting mode switching operation, the terminal switches the shooting mode of the camera application to Figure 5bThe ultra-wide-angle mode shown can capture complete images of the persons R1, R2, and R3 when in the ultra-wide-angle mode.

[0151] When it is necessary to capture a complete image containing only the person R1, the focal length of the physical camera needs to be increased. For example, switching to telephoto mode. The user can click the mode identifier 113. In response to the user's shooting mode switching operation, the terminal switches the shooting mode of the camera application to telephoto mode. Figure 5c The telephoto mode shown in FIG. 4 can capture a complete image containing only the person R1 when in the telephoto mode.

[0152] When it is necessary to capture a detailed image of a part of the person R1, the focal length of the physical camera needs to be increased again. For example, the focal length is switched to the super telephoto mode. The user can click the mode icon 114. In response to the user's shooting mode switching operation, the terminal switches the shooting mode of the camera application to Figure 5d The super telephoto mode shown, when in the super telephoto mode, can capture a detailed image containing only the upper body of the person R1.

[0153] Accordingly, upon detecting a shooting mode switching operation, the terminal may determine, in response to the shooting mode switching operation, multiple constraint strategies for switching the shooting mode based on the current first shooting mode and the desired second shooting mode. The multiple constraint strategies include: a first constraint strategy for image frames sent for display, a second constraint strategy for invoking a physical camera, a third constraint strategy for image processing methods, and a fourth constraint strategy for intermediate focal lengths for shooting.

[0154] The following is an illustrative description of multiple constraint strategies in the embodiment of the present application in conjunction with Table 1.

[0155] Table 1

[0156]

[0157]

[0158] The focal length constraint strategy includes: the correspondence between the physical camera in the terminal and the shooting mode.

[0159] The terminal includes multiple physical cameras, each configured to correspond to different shooting modes. This allows the focal length used in ultra-wide-angle mode to be smaller than that used in main camera mode; the focal length used in main camera mode to be smaller than that used in telephoto mode; and the focal length used in telephoto mode to be smaller than that used in super-telephoto mode. Furthermore, by ensuring that focal length constraints are met between shooting modes, discontinuities in the field of view (FOV) can be avoided, improving the user experience.

[0160] In addition, the shooting mode switch cannot jump to the focal length. For example, when the user instructs to switch from ultra-wide-angle mode to telephoto mode, the terminal needs to first switch the shooting mode of the camera application from ultra-wide-angle mode to wide-angle mode, and then switch from wide-angle mode to telephoto mode. It cannot switch directly from ultra-wide-angle mode to telephoto mode.

[0161] The 3A constraint strategy includes the number of image frames delayed for display before completing the shooting mode switch. 3A refers to the physical camera's auto focus (AF), automatic exposure (AE), and auto white balance (AWB).

[0162] When the second physical camera is called, the image quality captured before the second physical camera achieves 3A convergence is low. Therefore, the image frames captured by the second physical camera need to be displayed with a delay. That is, after the second physical camera is started, the first few image frames (for example, the first three frames) captured by the second physical camera need to be used as a secondary channel and not displayed. The images captured after the second physical camera achieves 3A convergence will be displayed. Displaying means that the captured image frames are displayed in the preset preview area.

[0163] The SAT constraint strategy includes: pixel offsets between the center points of images taken by different physical cameras.

[0164] The SAT algorithm requires dual streams, meaning two physical cameras (the first and second physical cameras are used for illustration) are used for capture. Based on the image frames captured by the first and second physical cameras, the pixel offset (i.e., offset) between the center points of the images captured by the different physical cameras is calculated. This pixel offset calculation process takes approximately three frames. Therefore, it takes three frames before the image warp can be added to the zoom process. This means that it takes three frames before the captured image frames can be spatially aligned according to the calculated pixel offset.

[0165] The center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used in different shooting modes during the shooting mode switching process.

[0166] If the focal length remains unchanged, switching physical cameras will cause a significant shift in the center points of the image frames captured by the different physical cameras, which is noticeable to the user. Therefore, it is necessary to switch physical cameras during zooming. Furthermore, simultaneously increasing the zoom speed and the speed of switching physical cameras can make the switching of physical cameras almost imperceptible to the user. Therefore, reducing the number of intermediate focal lengths between the focal lengths used in different shooting modes can achieve faster zoom speeds and the speed of switching physical cameras.

[0167] The smoothness constraint strategy includes: prohibiting the modification of the physical camera configuration during the shooting mode switching process, and limiting the number of image frames captured by two physical cameras at the same time during the shooting mode switching process.

[0168] Each physical camera switch results in frame loss or longer frame intervals. Modifying the physical camera configuration during the shooting mode switch will affect the smoothness of the shooting mode switch. Therefore, it is prohibited to modify the physical camera configuration during the shooting mode switch to improve the smoothness of the zoom preview. In addition, the power consumption of calling two physical cameras simultaneously during the shooting mode switch is high. Therefore, the dual-stream time is reduced. That is, the number of image frames captured by calling two physical cameras simultaneously during the shooting mode switch is set to a smaller number, and the calling of three physical cameras is prohibited to reduce the terminal power consumption.

[0169] The stability constraint strategy includes: prohibiting users from using the shooting function during the shooting mode switching process.

[0170] Photo stability during zooming is relatively poor. This is because the image metadata captured by the physical camera undergoes ISP processing according to the Pipeline processing method. The pre-processed image metadata is stored in the Result object, while the processed image frame is stored in the Request object. When the user uses the capture function, data is retrieved from both the Request and Result objects. Based on this data, the image saved at the time of the user's instruction is generated. However, there can be a gap of several frames between the data in the Request and Result objects, resulting in poor photo stability and low image quality. Therefore, users are prohibited from using the capture function during zooming to improve shooting stability.

[0171] The power consumption constraint strategy includes: when the shooting mode is not switched, the number of physical cameras called is 1.

[0172] The more physical cameras a terminal activates, and the longer it takes to activate multiple physical cameras, the higher the terminal's power consumption. Therefore, when zooming is not adjusted, keep the terminal in a single stream, that is, activate only one physical camera, to reduce terminal power consumption.

[0173] In some embodiments, the first shooting mode and the second shooting mode are two adjacent shooting modes. For example, the first shooting mode is an ultra-wide-angle mode, and the second shooting mode is a wide-angle mode. Alternatively, the first shooting mode is a wide-angle mode, and the second shooting mode is a telephoto mode, etc.

[0174] In some embodiments, the first shooting mode and the second shooting mode are separated by a third shooting mode. There can be multiple third shooting modes. For example, the first shooting mode is the ultra-wide-angle mode; the second shooting mode is the ultra-telephoto mode. The third shooting mode between the ultra-wide-angle mode and the ultra-telephoto mode includes: wide-angle mode, telephoto mode. That is, from Figure 5b The shooting mode shown switches to Figure 5d Shooting modes shown.

[0175] Alternatively, the first shooting mode is the super telephoto mode; the second shooting mode is the super wide-angle mode. Figure 5d The shooting mode shown switches to Figure 5b Shooting modes shown.

[0176] The third shooting mode can be one. For example, the first shooting mode is the ultra-wide-angle mode, and the second shooting mode is the telephoto mode. The ultra-wide-angle mode and the telephoto mode are separated by the wide-angle mode.

[0177] The above are just examples; the first and second shooting modes can also be other shooting modes. For example, the first shooting mode can be telephoto mode, and the second shooting mode can be wide-angle mode. Alternatively, the first shooting mode can be wide-angle mode, and the second shooting mode can be super telephoto mode, etc.

[0178] For step S402 , the relative time is the time when the shooting is performed relative to the time when the shooting mode switching is started.

[0179] Relative time can be expressed using captured image frames. For example, relative time can be expressed as: the time at which the first image frame is captured after the start of the shooting mode switch, the time at which the second image frame is captured after the start of the shooting mode switch, the time at which the third image frame is captured after the start of the shooting mode switch, and so on, until the time at which the nth image frame is captured after the start of the shooting mode switch. n represents the number of image frames required to capture during the shooting mode switch process.

[0180] Correspondingly, the first correspondence can be expressed as: the focal length of the physical camera when the first image frame is captured after the shooting mode switching starts is f1, the focal length of the physical camera when the second image frame is captured after the shooting mode switching starts is f2, the focal length of the physical camera when the third image frame is captured after the shooting mode switching starts is f3, ..., the focal length of the physical camera when the nth image frame is captured after the shooting mode switching starts is fn.

[0181] Relative moments can also be expressed as time. For example, relative moments can be expressed as: 30 milliseconds after the start of the shooting mode switch, 60 milliseconds after the start of the shooting mode switch, 90 milliseconds after the start of the shooting mode switch, and so on, to m milliseconds after the start of the shooting mode switch. m represents the duration required for the shooting mode switch.

[0182] Accordingly, the first correspondence can be expressed as: the focal length of the physical camera at 30 milliseconds after the start of the shooting mode switch is f1, the focal length of the physical camera at 60 milliseconds after the start of the shooting mode switch is f2, the focal length of the physical camera at 90 milliseconds after the start of the shooting mode switch is f3, ..., and the focal length of the physical camera at m milliseconds after the start of the shooting mode switch is fn. n represents the number of image frames required to be captured during the shooting mode switch process.

[0183] The first corresponding relationship may be a Zoom Ratio function, which may be any one of a linear curve, a quadratic curve, and a Bezier curve. The Zoom Ratio function represents the relationship between relative time and focal length change.

[0184] In some embodiments, the Zoom Ratio function may select a curve whose changing trend is initially slow and then fast. For example, a quadratic curve or a Bezier curve may be selected. That is, the changing relationship between the relative time and the focal length in the first corresponding relationship is a quadratic curve, or the changing relationship between the relative time and the focal length in the first corresponding relationship is a Bezier curve.

[0185] Switching physical cameras while the field of view changes little or nothing at all can result in significant differences in the captured image frames, which can be noticeable to the user and negatively impact the user experience. Accordingly, the Zoom Ratio function uses a curve that changes slowly at first and then quickly. The Zoom Ratio function's initial change is slow, and accordingly, the physical camera's field of view changes slowly. This allows for more time to prepare for switching physical cameras, allowing the physical camera to be switched to start as a secondary camera. The Zoom Ratio function's subsequent change is faster, resulting in a significant change in the field of view. Switching physical cameras at this point means that even if there's a slight offset between the cameras, the user won't notice it, improving the user experience.

[0186] The following combination Figure 6a and Figure 6b The first corresponding relationship is exemplified. Figure 6a and Figure 6bT in the middle means telephoto mode; ST means super telephoto mode; W means wide angle mode; UW means ultra wide angle mode. Figure 6a and Figure 6b The curve shows a slow-moving trend. ST indicates the focal length used in super telephoto mode; T indicates the focal length used in telephoto mode; W indicates the focal length used in wide-angle mode; and UW indicates the focal length used in ultra-wide-angle mode. Figure 6a In the figure, UW+W indicates the intermediate focal length when switching from ultra-wide-angle mode to wide-angle mode. W+T indicates the intermediate focal length when switching from wide-angle mode to telephoto mode. T+ST indicates the intermediate focal length when switching from telephoto mode to ultra-telephoto mode. Figure 6b In the figure, ST+T represents the intermediate focal length when switching from super telephoto mode to telephoto mode. T+W represents the intermediate focal length when switching from telephoto mode to wide-angle mode. W+UW represents the intermediate focal length when switching from wide-angle mode to ultra wide-angle mode.

[0187] In some embodiments, the first shooting mode is an ultra-wide angle mode; the second shooting mode is an ultra-telephoto mode. In the first corresponding relationship, the focal length increases as the relative time increases. For example, the relative time and the focal length in the first corresponding relationship are Figure 6a The curve shown changes. Figure 6a In the , the focal length increases gradually from the focal length used in ultra-wide-angle mode, through several intermediate focal lengths, to the focal length used in ultra-telephoto mode.

[0188] In some embodiments, the first shooting mode is an ultra-telephoto mode; the second shooting mode is an ultra-wide-angle mode. In the first corresponding relationship, the focal length decreases as the relative time increases. In the first corresponding relationship, the relative time and the focal length decrease according to Figure 6b The curve shown changes. Figure 6b In the , the focal length decreases from the focal length used in super telephoto mode, through several intermediate focal lengths, to the focal length used in ultra wide-angle mode.

[0189] In some embodiments, the fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between focal lengths used in different shooting modes during the shooting mode switching process.

[0190] Correspondingly, step S402 may include the following steps: determining, according to the center point constraint strategy, the number of multiple intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode as a third number; and determining the third number of intermediate focal lengths in the first corresponding relationship between the preset relative moment and the focal length.

[0191] The terminal determines a third number of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode according to the center point constraint strategy, and determines the third number of intermediate focal lengths according to the first corresponding relationship.

[0192] The third number is set as needed. When the third number is set to a smaller value, the number of intermediate focal lengths used for shooting during the shooting mode switch can be reduced, thereby reducing the time required for shooting mode switching. This can speed up the shooting mode switch and shorten the time that the image frames displayed in the preset preview area during the shooting mode switch, allowing the user to switch shooting modes without noticing, thereby improving the user experience.

[0193] With respect to step S403, after determining each intermediate focal length, at each relative moment, the terminal adjusts the focal length to the focal length corresponding to that relative moment and calls the target physical camera to capture at that intermediate focal length. If the number of captured image frames does not reach the first number, the target physical camera is the first physical camera. If the number of captured image frames reaches the first number, the target physical camera includes the first physical camera and the second physical camera.

[0194] For example, at the 30th millisecond after the start of the shooting mode switch, the focal length is adjusted to f1, and the target physical camera is called to shoot at the intermediate focal length; at the 60th millisecond after the start of the shooting mode switch, the focal length is adjusted to f2, and the target physical camera is called to shoot at the intermediate focal length, and so on, until the shooting mode switch is completed.

[0195] In some embodiments, after determining the third number of intermediate focal lengths, the method may further include the following steps: based on the third number of multiple intermediate focal lengths and the number of image frames required to be captured during the shooting mode switching process, calculating the number of image frames required to be captured at each intermediate focal length as a fourth number.

[0196] Accordingly, step S403 includes the following steps: when each relative moment is reached, adjusting the focal length to an intermediate focal length corresponding to the relative moment, and calling the target physical camera to shoot a fourth number of image frames at the intermediate focal length.

[0197] Once the third number of intermediate focal lengths has been determined, the fourth number of image frames required to be captured at each intermediate focal length can be calculated based on the number of image frames required to be captured during the capture mode switch. Accordingly, at each relative moment, the focal length is adjusted to the intermediate focal length corresponding to that relative moment, and the target physical camera is called upon to capture the fourth number of image frames at that intermediate focal length.

[0198] As can be seen from the above, in the embodiment of the present application, a fourth number of image frames are captured at various intermediate focal lengths. Subsequently, the target image frame from the fourth number of image frames captured at various intermediate focal lengths can be displayed in a preset preview area, so that the preset preview area displays the target image frame with higher clarity, thereby improving the user experience.

[0199] In some embodiments, the first shooting mode and the second shooting mode are two adjacent shooting modes, and the first shooting mode can be directly switched to the second shooting mode.

[0200] Accordingly, step S403 may include the following steps: at each first relative moment, adjusting the focal length to an intermediate focal length corresponding to the first relative moment, and invoking the first physical camera to shoot at the intermediate focal length. After the number of captured image frames reaches the first number, at each second relative moment, adjusting the focal length to an intermediate focal length corresponding to the second relative moment, and invoking the first physical camera and the second physical camera to shoot at the intermediate focal length.

[0201] Accordingly, the method may further include the following steps: when the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by the second physical camera meets a preset condition, stopping shooting with the first physical camera.

[0202] The first relative moment is the relative moment before the second physical camera is called; the second relative moment is the relative moment after the second physical camera is called.

[0203] When switching from the first shooting mode to the second shooting mode, if the second physical camera is called after the focal length is adjusted to the focal length used in the second shooting mode, the second physical camera may not be able to capture stable images when it is first started. For example, the second physical camera has not yet completed 3A convergence, and the captured image frames will have low clarity. Therefore, during the zoom process, the second physical camera is started in advance. Accordingly, the relative time during the process of switching from the first shooting mode to the second shooting mode includes the first relative time and the second relative time.

[0204] When each first relative moment is reached, the terminal adjusts the focal length to an intermediate focal length corresponding to the first relative moment, and calls the first physical camera to shoot at the intermediate focal length.

[0205] After the number of captured image frames reaches the first number, indicating that the focal length is close to the focal length used in the second shooting mode, the second physical camera can be activated as the auxiliary channel for shooting, and the captured image frames are not sent for display. Furthermore, because the second physical camera cannot capture stable images when it is first activated, the first physical camera is simultaneously activated as the primary channel for shooting, and the captured image frames are sent for display. That is, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the first and second physical cameras are activated to shoot at this intermediate focal length.

[0206] Furthermore, when the focal length is adjusted to the focal length used in the second shooting mode and the image frame captured by the second physical camera meets the preset conditions, the shooting mode switching is completed, the second physical camera can capture images stably, and the image frame captured by the second physical camera can be used for display. The preset preview area still displays the target image frame with higher clarity, and then the first physical camera is stopped from being called for shooting.

[0207] As can be seen from the above, in the embodiments of the present application, when switching between shooting modes and zooming, activating the second physical camera in advance as a secondary camera for shooting can speed up the shooting mode switching process and achieve a user-imperceptible physical camera switch. Furthermore, image frames that are not stabilized by the second physical camera are not displayed, and image frames captured by the first physical camera are still displayed. This ensures that the preset preview area displays the target image frames with higher definition, improving the user experience.

[0208] Moreover, when the focal length is adjusted to the focal length used in the second shooting mode and the image frame captured by the second physical camera meets the preset conditions, the second physical camera can capture images stably, and the image frame captured by the second physical camera can still be displayed in the preset preview area as a target image frame with higher clarity. In this case, the first physical camera is stopped from being called for shooting, which can reduce the power consumption of the terminal.

[0209] In some embodiments, the second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between a physical camera in the terminal and a shooting mode.

[0210] Accordingly, before calling the second physical camera for shooting, the method may further include the following steps: according to the focal length constraint strategy, determining a physical camera corresponding to the second shooting mode from the physical cameras in the terminal as the second physical camera.

[0211] Because different shooting modes require different physical cameras in the terminal, when it is determined that a second shooting mode needs to be switched, the terminal can determine the second physical camera corresponding to the second shooting mode from the physical cameras in the terminal according to the focal length constraint policy. By invoking different physical cameras, users can be provided with different shooting modes, thereby improving the user experience.

[0212] In some embodiments, the first shooting mode and the second shooting mode are separated by a third shooting mode, and it is necessary to switch from the first shooting mode to the third shooting mode, and then switch from the third shooting mode to the second shooting mode.

[0213] Accordingly, step S403 may include the following steps: when each first relative moment is reached, adjusting the focal length to the intermediate focal length corresponding to the first relative moment, and calling the first physical camera to shoot at the intermediate focal length; after the number of captured image frames reaches the first number, when each third relative moment is reached, adjusting the focal length to the intermediate focal length corresponding to the third relative moment, and calling the first physical camera and the third physical camera used in the third shooting mode to shoot at the intermediate focal length; after the number of captured image frames reaches the fifth number, when each second relative moment is reached, adjusting the focal length to the intermediate focal length corresponding to the second relative moment, and calling the third physical camera and the second physical camera to shoot at the intermediate focal length.

[0214] Accordingly, the method may further include the following steps: when the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by the second physical camera meets a preset condition, stopping shooting by the third physical camera.

[0215] The first relative time is the relative time before calling the second physical camera; the third relative time is the relative time after calling the third physical camera and before calling the second physical camera; and the second relative time is the relative time after calling the second physical camera.

[0216] When switching from the first capture mode to the third capture mode, if the third physical camera is activated after the focal length is adjusted to that used for the third capture mode, the third physical camera may not be able to capture stable images upon initial activation. For example, if the third physical camera has not yet completed 3A convergence, the captured image frames may be of low clarity. Therefore, during the zoom process, the third physical camera is activated in advance. Accordingly, the relative time during the switch from the first capture mode to the third capture mode includes the first relative time and the third relative time.

[0217] When each first relative moment is reached, the terminal adjusts the focal length to an intermediate focal length corresponding to the first relative moment, and calls the first physical camera to shoot at the intermediate focal length.

[0218] After the number of image frames captured by the first physical camera reaches a first number, indicating that the focal length is close to the focal length used in the third shooting mode, the third physical camera can be activated as the auxiliary channel for shooting, and the captured image frames are not sent for display. Furthermore, because the third physical camera cannot capture stable images upon initial activation, the first physical camera is simultaneously activated as the primary channel for shooting, and the captured image frames are sent for display. Specifically, at each third relative moment, the focal length is adjusted to the intermediate focal length corresponding to that third relative moment, and the first and third physical cameras are activated to shoot at that intermediate focal length.

[0219] When the focal length is adjusted to the focal length used in the third shooting mode and the image frame captured by the third physical camera meets the preset conditions, the shooting mode has been switched to the third shooting mode. The third physical camera can capture images stably, and the image frame captured by the third physical camera can be used for display. The preset preview area still displays the target image frame with higher clarity, and then the first physical camera is stopped from being called for shooting.

[0220] Furthermore, after the number of image frames captured by the third physical camera reaches the fifth number, indicating that the focal length is close to the focal length used in the second shooting mode, the second physical camera can be activated as the auxiliary channel for shooting, and the captured image frames are not sent for display. Furthermore, because the second physical camera cannot capture stable images when it is first activated, the third physical camera is simultaneously activated as the primary channel for shooting, and the captured image frames are sent for display. In other words, at each second relative moment, the focal length is adjusted to the intermediate focal length corresponding to that second relative moment, and the third and second physical cameras are activated to shoot at this intermediate focal length.

[0221] When the focal length is adjusted to the focal length used in the second shooting mode and the image frame captured by the second physical camera meets the preset conditions, the shooting mode has been switched to the second shooting mode. The second physical camera can capture images stably, and the image frame captured by the second physical camera can be used for display. The preset preview area still displays the target image frame with higher clarity, and then the third physical camera is stopped from being called for shooting.

[0222] As can be seen from the above, in the embodiment of the present application, when switching between shooting modes and zooming, the physical camera used in the next shooting mode is activated in advance as an auxiliary channel for shooting, which can speed up the shooting mode switching and achieve the user-unnoticed switching of physical cameras. In addition, image frames that are not stabilized by the physical camera are not displayed, and image frames captured by the physical camera used in the previous shooting mode are still displayed. This ensures that the preset preview area displays the target image frames with higher clarity, thereby improving the user experience.

[0223] Moreover, when the focal length is adjusted to the focal length used in the next shooting mode and the image frame captured by the physical camera used in the next shooting mode meets the preset conditions, the physical camera used in the next shooting mode can capture images stably, and the image frame captured by the physical camera used in the next shooting mode can still be displayed in the preset preview area as a target image frame with higher clarity. Then, the physical camera used in the previous shooting mode is stopped from being called for shooting, which can reduce the power consumption of the terminal.

[0224] In step S404, the terminal uses the target camera to capture multiple image frames at each intermediate focal length. These multiple image frames include both high-quality and low-quality images. At each relative moment, the terminal uses the third constraint strategy and the first constraint strategy to determine a target image frame that meets the preset conditions from the image frames captured at the intermediate focal length corresponding to that relative moment. The terminal then displays the target image frame in a preset preview area.

[0225] In some embodiments, the first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames that are delayed for display before completing the shooting mode switch.

[0226] Accordingly, step S404 may include the following steps: determining, in accordance with the 3A consistency constraint strategy, the number of image frames to be delayed for display during the shooting mode switch as a second number. For each first relative moment, if the image frame captured at the first relative moment meets a preset condition, displaying the image frame captured at the first relative moment as the target image frame in a preset preview area. If the image frame captured at the first relative moment does not meet the preset condition, determining a second number of target image frames captured before the first relative moment that meet the preset condition, and displaying the determined target image frames in the preset preview area.

[0227] The second number is set according to the requirements. For example, the second number is 3, or the second number is 4, etc., which is not specifically limited in the embodiments of the present application.

[0228] For each first relative moment, it is detected whether the image frame captured at the first relative moment meets the preset conditions. If the preset conditions are met, it indicates that the image frame is an image frame stably captured by the first physical camera, and the quality of the image frame is good and the clarity is high. Then, the image frame captured at the first relative moment is displayed as the target image frame in the preset preview area.

[0229] If the image frame captured at the first relative moment does not meet the preset conditions, indicating that the image frame is not an image frame stably captured by the first physical camera and the clarity of the image frame is low, a second number of target image frames that meet the preset conditions are determined from the image frames captured before the first relative moment, and the determined target image frames are displayed in the preset preview area.

[0230] For the method of detecting whether the image frame meets the preset conditions, please refer to the relevant description of the subsequent embodiments.

[0231] As can be seen from the above, in the embodiment of the present application, during the process of switching the shooting mode, for each first relative moment, if the image frame captured at the first relative moment meets the preset conditions, it is directly displayed; if the image frame captured at the first relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame captured previously is used for display, so that the target image frame displayed is a higher definition target image frame, and then the preset preview area displays a higher definition target image frame, thereby improving the user experience.

[0232] In some embodiments, the third constraint strategy includes: a SAT constraint strategy; the SAT constraint strategy includes: pixel offsets between center points of images captured by different physical cameras.

[0233] Accordingly, step S404 may include the following steps: calculating, according to the SAT strategy, a pixel offset between the center point of the image frame captured by the first physical camera and the center point of the image frame captured by the second physical camera. Based on the pixel offset, processing the image frame captured by the second physical camera. For each second relative moment, if the image frame captured at the second relative moment meets a preset condition, displaying the image frame captured at the second relative moment as a target image frame in a preset preview area. If the image frame captured at the second relative moment does not meet the preset condition, determining a second number of target image frames captured by the first physical camera before the second relative moment that meet the preset condition, and displaying the determined target image frames in the preset preview area.

[0234] To switch from the first shooting mode to the second shooting mode, the physical camera to be called needs to be switched from the first physical camera to the second physical camera, and there is a deviation in the center point of the images captured by different physical cameras. In order to achieve the user-imperceptible switching of physical cameras, the pixel offset between the center point of the image frame captured by the first physical camera and the center point of the image frame captured by the second physical camera is determined based on the SAT constraint strategy. Then, the image frame captured by the second physical camera is processed according to the calculated pixel offset. For example, according to the calculated pixel offset value, the image frame captured by the second physical camera is spatially aligned, so that the center point of the image frame captured by the second physical camera is consistent with that of the image frame captured by the first physical camera.

[0235] Furthermore, for each second relative moment, it is detected whether the image frame captured at the second relative moment meets a preset condition. If the preset condition is met, it indicates that the image frame is an image frame stably captured by the second physical camera, and the quality of the image frame is good and the clarity is high. Then, the image frame captured at the second relative moment is displayed as the target image frame in the preset preview area.

[0236] If the image frame captured at the second relative moment does not meet the preset conditions, it indicates that the image frame was not stably captured by the second physical camera and that the image frame has low clarity, possibly because the second physical camera was just started and has not yet achieved 3A convergence. Accordingly, a second number of target image frames that meet the preset conditions are determined from the image frames captured by the first physical camera before the second relative moment, and the determined target image frames are displayed in a preset preview area.

[0237] As can be seen from the above, in the embodiment of the present application, during the process of switching the shooting mode, for each second relative moment, if the image frame taken at the second relative moment meets the preset conditions, it is directly displayed; if the image frame taken at the second relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame taken by the first physical camera is used for display, so that the target image frame displayed is a higher definition target image frame, and then the preset preview area displays a higher definition target image frame, thereby improving the user experience.

[0238] In addition, a second number (for example, 3) frames need to be inserted between the currently called first physical camera and the second physical camera to be switched. The second number of frames uses the image frames taken by the first physical camera for display, that is, the second physical camera is started the second number of frames in advance. On the one hand, the second physical camera is allowed to perform 3A convergence to avoid the problem of excessive scene difference before and after 3A synchronization switching, and further converge several frames on the basis of 3A synchronization to achieve the optimal effect of the image frames for display; on the other hand, the SAT calculation is triggered within the second number of frames to achieve the optimal effect of the center point offset between the physical camera switches, so that the user can complete the shooting mode switch without noticing, thereby improving the user experience.

[0239] In some embodiments, the following steps are used to detect whether a captured image frame meets a preset condition: for each relative moment, image metadata captured at the corresponding intermediate focal length at that relative moment is cached in a first storage area. The image metadata is sequentially retrieved from the first storage area in the chronological order in which the image metadata was cached, processed to obtain corresponding image frames, and the resulting image frames are stored in a second storage area. If the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, the image frame captured at that relative moment is determined to meet the preset condition.

[0240] The terminal uses the physical camera to capture image metadata, such as unprocessed (RAW) data. This image metadata needs to be processed by ISP to obtain image frames in RGB or YUV format. The terminal can use the pipeline image processing method to perform ISP processing on image metadata.

[0241] For each relative moment, the terminal caches the image metadata collected at the relative moment according to the corresponding intermediate focal length in the first storage area. For example, the image metadata is cached in the storage area corresponding to the called Result object.

[0242] Then, in the order in which the image metadata was cached, the image metadata is retrieved from the first storage area, ISP processed, and the corresponding image frames are stored in the second storage area. For example, the image frames are cached in the storage area corresponding to the called Request object.

[0243] Since it takes a certain amount of time to process the image metadata, when the image frame is cached in the second storage area, the new image metadata may have been cached in the first storage area. In this case, the focal length corresponding to the image metadata in the first storage area may be different from the focal length corresponding to the image frame in the second storage area. At this time, it indicates that the physical camera has not stably captured the image, and it can be determined that the captured image frame does not meet the preset conditions.

[0244] If the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it indicates that the physical camera captures the image stably, and it can be determined that the captured image frame meets the preset conditions.

[0245] As can be seen from the above, the solution provided by this embodiment determines whether the physical camera is stably capturing images by detecting whether the image frame meets the preset conditions. Subsequently, the target image frame that meets the preset conditions is displayed in the preset preview area. The clarity of the target image frame is higher than that of other image frames, which can improve the clarity of the image frame displayed in the preset preview area and enhance the user experience.

[0246] In some embodiments, the first shooting mode and the second shooting mode are separated by a third shooting mode; the display method during the process of switching from the first shooting mode to the third shooting mode, and the display method during the process of switching from the third shooting mode to the second shooting mode are similar to the display method during the process of directly switching from the first shooting mode to the second shooting mode, and refer to the relevant introduction of the aforementioned embodiments.

[0247] Similarly, during the process of switching shooting modes, for each relative moment, if the image frame captured at the relative moment meets the preset conditions, it is directly displayed; if the image frame captured at the relative moment does not meet the preset conditions, it is delayed for display, that is, the target image frame captured by the physical camera called by the previous shooting mode before the relative moment is displayed. For example, if the image frame captured at the third relative moment does not meet the preset conditions, the target image frame is determined from the image frames captured by the first physical camera called before the third relative moment for display; or, if the image frame captured at the second relative moment does not meet the preset conditions, the target image frame is determined from the image frames captured by the third physical camera called before the second relative moment for display. In this way, the target image frame displayed is of higher definition, and the preset preview area displays the target image frame of higher definition, thereby improving the user experience.

[0248] In some embodiments, the method may further include the following steps: after successfully switching to the second shooting mode, displaying the target image frame shot by calling the second physical camera in a preset preview area.

[0249] When the focal length is adjusted to that used in the second shooting mode and the image frames captured by the second physical camera meet the preset conditions, indicating a successful switch to the second shooting mode, the target image frames captured by the second physical camera are then displayed in the preset preview area. After successfully switching from the first shooting mode to the second shooting mode, the physical camera switch and focal length adjustment are complete, and the camera can stably capture image frames. At this time, the captured image frames have a higher clarity. Displaying the target image frames captured in the second shooting mode in the preset preview area can improve the clarity of the image frames displayed in the preset preview area, thereby enhancing the user experience.

[0250] In some embodiments, the constraint strategy determined in step S402 further includes a stability constraint strategy; the stability constraint strategy includes: during the process of switching the shooting mode, prohibiting the user from using the shooting function.

[0251] Accordingly, the method may further include the following step: during the process of switching from the first shooting mode to the second shooting mode, prohibiting responding to the shooting operation.

[0252] Switching from the first shooting mode to the second requires adjusting the focal length, resulting in poor shooting stability and low image quality. Furthermore, shooting consumes a significant amount of CPU resources, impacting terminal performance. This can affect the smoothness of zooming on lower-end devices or in scenarios with high terminal loads, further impacting shooting mode switching.

[0253] Moreover, when switching from the first shooting mode to the second shooting mode, the user needs to obtain data from the Request object and the Result object respectively when using the shooting function, and the focal length carried in the data attribute information (Metadata) of the Result object and the Metada of the Request object may be different. In this case, the terminal's photography stability is poor, and the captured image quality is low, affecting the user experience.

[0254] Accordingly, in the process of switching from the first shooting mode to the second shooting mode, the shooting operation is prohibited, thereby improving the stability of shooting. After the shooting mode switching is completed, the shooting operation is responded to, and the quality of the captured image is high, thereby improving the user experience.

[0255] In some embodiments, the constraint strategy determined in step S402 further includes a power consumption constraint strategy; the power consumption constraint strategy includes: when the shooting mode is not switched, the number of physical cameras called is 1.

[0256] Accordingly, the method may further include the following steps: when the shooting mode is not switched, prohibiting the calling of other physical cameras except the physical camera called by the current shooting mode.

[0257] The more physical cameras a terminal activates, and the longer it takes to activate multiple physical cameras, the higher the terminal's power consumption. Therefore, when zooming is not enabled, it is prohibited to call other physical cameras except the one activated in the current shooting mode. For example, when in the first shooting mode, only the first physical camera is called for shooting, and then other physical cameras are called. Alternatively, when in the second shooting mode, only the second physical camera is called for shooting, and then other physical cameras are called.

[0258] As can be seen from the above, in the embodiment of the present application, when the shooting mode is not switched, the terminal maintains a single stream, that is, only one physical camera is called to reduce the power consumption of the terminal.

[0259] In some embodiments, to streamline the shooting mode switching process, switching the physical camera configuration during the shooting mode switching process is prohibited. If the physical camera configuration is switched before the shooting mode is switched, the physical camera configuration is first adjusted to the default configuration before switching the shooting mode.

[0260] The following combination Figure 7 , the switching process from ultra-wide-angle (UW) mode to super-telephoto (ST) mode is explained.

[0261] The optical zoom point in ultra-wide-angle mode is 0.6X, which means the focal length of the physical camera in ultra-wide-angle mode is 0.6 times the equivalent focal length. The optical zoom point in wide-angle mode is 1X, which means the focal length of the physical camera in ultra-wide-angle mode is the equivalent focal length. The optical zoom point in telephoto mode is 3X, which means the focal length of the physical camera in telephoto mode is 3 times the equivalent focal length. The optical zoom point in super-telephoto mode is 6X, which means the focal length of the physical camera in telephoto mode is 6 times the equivalent focal length.

[0262] The shooting mode is switched from ultra-wide-angle mode to ultra-telephoto mode, that is, the focal length is adjusted from 0.6X to 6X. Figure 7 The focal lengths between 0.6 and 6 are all intermediate focal lengths, for example, 0.6375, 0.75, ..., 5.1375, etc. The multiple intermediate focal lengths are calculated based on the Figure 6a The Zoom Ratio function shown is determined. Figure 7 The specific focal length is expressed as a multiple of the equivalent focal length. For example, 0.6 represents 0.6 times the equivalent focal length; 0.6375 represents 0.6375 times the equivalent focal length; and so on. 5.1375 represents 5.1375 times the equivalent focal length.

[0263] The mode identifier for each focal length indicates that the physical camera corresponding to that shooting mode is used for shooting at that focal length. 0.6 corresponds to UW, which means that the physical camera corresponding to the ultra-wide-angle mode is used for shooting. This shows that only one corresponding physical camera is used for shooting in ultra-wide-angle mode.

[0264] 0.6375, 0.75, and 0.9375 correspond to UW and W, indicating that the physical camera corresponding to ultra-wide-angle mode and the physical camera corresponding to wide-angle mode are used for shooting. This shows that when switching from ultra-wide-angle mode to wide-angle mode, the physical camera corresponding to wide-angle mode is started in advance at a focal length of 0.6375, and the physical camera corresponding to ultra-wide-angle mode and the physical camera corresponding to wide-angle mode are called for shooting.

[0265] 1.2, 1.5375, and 1.95 correspond to W and T, indicating that the physical camera corresponding to wide-angle mode and the physical camera corresponding to telephoto mode are used for shooting. This shows that when switching from wide-angle mode to telephoto mode, the physical camera corresponding to telephoto mode is started in advance when the focal length is 1.2, and the physical cameras corresponding to wide-angle mode and telephoto mode are called for shooting.

[0266] 2.4375, 3, 3.6375, 4.35, and 5.1375 correspond to T and ST, indicating that the physical camera corresponding to wide-angle mode and the physical camera corresponding to telephoto mode are used for shooting. This shows that when switching from telephoto mode to super-telephoto mode, the physical camera corresponding to super-telephoto mode is pre-activated at a focal length of 2.4375, and the physical cameras corresponding to telephoto mode and super-telephoto mode are called for shooting.

[0267] 6 corresponds to ST, which means that the physical camera corresponding to the ultra-wide-angle mode is used for shooting. From this, it can be seen that only one corresponding physical camera is called for shooting in the ultra-telephoto mode.

[0268] Snapshot is used to indicate whether the user can use the shooting function at this focal length. Snapshot is 1, which means the user is allowed to use the shooting function. Snapshot is 0, which means the user is prohibited from using the shooting function. Figure 7 It can be seen that during the zooming process, the user is prohibited from using the shooting function through the stability constraint strategy and the smoothness constraint strategy.

[0269] When the focal length is less than 1X, non-Wide display is used. That is, when the focal length is less than 1X, before switching to wide-angle mode, images shot in ultra-wide-angle mode are displayed.

[0270] When the focal length is less than 3X, non-Tele mode is used for display. That is, when the focal length is less than 3X and the telephoto mode has not been switched, the image shot in wide-angle mode is displayed.

[0271] When the focal length is less than 6X, non-Super Tele mode is used for display. That is, when the focal length is less than 6X and has not yet been switched to super telephoto mode, images shot in super telephoto mode are displayed.

[0272] When switching from ultra-wide-angle mode to ultra-telephoto mode, you need to first switch from ultra-wide-angle mode to wide-angle mode, then switch from wide-angle mode to telephoto mode, and then switch from telephoto mode to ultra-telephoto mode.

[0273] When switching from ultra-wide-angle mode to wide-angle mode, the 3A consistency constraint strategy and the SAT constraint strategy are enabled. Specifically, during the period when the focal length is adjusted from 0.6375 to 0.9375, the physical camera used in wide-angle mode is pre-activated. This allows the physical camera used in wide-angle mode to perform 3A convergence and enable the SAT algorithm during this period. Furthermore, the three image frames captured by the physical camera used in wide-angle mode as a secondary image during this period are not displayed. After switching to wide-angle mode, when the focal length reaches 1.2, the physical camera used in ultra-wide-angle mode is no longer used.

[0274] Furthermore, when switching from ultra-wide-angle mode to wide-angle mode, the center point constraint strategy must be enabled. During the period when the focal length is adjusted from 0.9375 to 1.2, the physical camera used in ultra-wide-angle mode is switched to the physical camera used in wide-angle mode, that is, the zoom is switched to wide during the change.

[0275] When switching from wide-angle mode to telephoto mode, the 3A consistency constraint strategy and the SAT constraint strategy are enabled. Specifically, during the period when the focal length is adjusted from 1.2 to 1.95, the physical camera used in telephoto mode is pre-activated. This allows the physical camera in telephoto mode to perform 3A convergence and enable the SAT algorithm during this period. Furthermore, the three image frames captured by the physical camera in telephoto mode as a secondary image during this period are not displayed. After switching to telephoto mode, when the focal length reaches 2.4375, the physical camera in wide-angle mode is no longer used.

[0276] Furthermore, when switching from wide-angle mode to telephoto mode, the center point constraint strategy must be enabled. During the period when the focal length is adjusted from 1.95 to 2.4375, the physical camera used in wide-angle mode switches to the physical camera used in telephoto mode, effectively switching from zoom to telephoto mode.

[0277] During the focal length adjustment from 2.4375 to 5.1375, the smoothness constraint strategy is enabled and the default setting is maintained throughout the switching process. This means that the physical camera configuration is modified during the switching process.

[0278] During the process of adjusting the focal length from 3 to 6, the smoothness constraint strategy and the power consumption constraint strategy are enabled. That is, the physical cameras of the unnecessary auxiliary channels are turned off in time to improve the smoothness of the shooting mode switching and reduce the power consumption of the terminal.

[0279] After the focal length is adjusted to 6, the stability constraint policy and the smoothness constraint policy are enabled. That is, the user's shooting function is prohibited during the zoom process, and only one physical camera is called, which improves the smoothness of shooting mode switching and reduces terminal power consumption.

[0280] The following combination Figure 8 , the switching process from ultra-wide-angle (UW) mode to super-telephoto (ST) mode is explained.

[0281] The optical zoom point in ultra-wide-angle mode is 0.6X, which means the focal length of the physical camera in ultra-wide-angle mode is 0.6 times the equivalent focal length. The optical zoom point in wide-angle mode is 1X, which means the focal length of the physical camera in ultra-wide-angle mode is the equivalent focal length. The optical zoom point in telephoto mode is 3X, which means the focal length of the physical camera in telephoto mode is 3 times the equivalent focal length. The optical zoom point in super-telephoto mode is 6X, which means the focal length of the physical camera in telephoto mode is 6 times the equivalent focal length.

[0282] The shooting mode is switched from super telephoto mode to ultra wide-angle mode, that is, the focal length is adjusted from 6X to 0.6X. Figure 8 The focal lengths between 6 and 0.6 are all intermediate focal lengths, for example, 5.9625, 5.85, 5.6625, ..., 1.4625, etc. The multiple intermediate focal lengths are calculated according to Figure 6b The Zoom Ratio function shown is determined. Figure 8 The equivalent focal length is expressed as a multiple of the focal length. For example, 6 represents 6 times the equivalent focal length; 5.9625 represents 5.9625 times the equivalent focal length; and 1.4625 represents 1.4625 times the equivalent focal length.

[0283] The mode identifier corresponding to each focal length is used to indicate that the physical camera corresponding to the shooting mode is used to shoot at that focal length. 6 corresponding to ST indicates that the physical camera corresponding to the super telephoto mode is used to shoot. This shows that in super telephoto mode, only one corresponding physical camera is used for shooting.

[0284] 5.9625, 5.85, and 5.6625 correspond to T and ST, indicating that the physical camera corresponding to the super telephoto mode and the physical camera corresponding to the telephoto mode are used for shooting. This shows that when switching from super telephoto mode to telephoto mode, the physical camera corresponding to the telephoto mode is started in advance when the focal length is 5.9625, and the physical cameras corresponding to the super telephoto mode and the telephoto mode are called for shooting.

[0285] 5.4, 5.0625, 4.65, 4.1625, and 3.6 correspond to W and T, indicating that the physical camera corresponding to wide-angle mode and the physical camera corresponding to telephoto mode are used for shooting. This shows that when switching from telephoto mode to wide-angle mode, the physical camera corresponding to wide-angle mode is started in advance when the focal length is 5.4, and the physical cameras corresponding to wide-angle mode and telephoto mode are called for shooting.

[0286] 2.9625, 2.25, and 1.4625 correspond to W and UW, indicating that the physical camera corresponding to wide-angle mode and the physical camera corresponding to ultra-wide-angle mode are used for shooting. This shows that when switching from wide-angle mode to ultra-wide-angle mode, the physical camera corresponding to ultra-wide-angle mode is pre-activated at a focal length of 2.9625, and the physical camera corresponding to wide-angle mode and ultra-wide-angle mode are called for shooting.

[0287] 0.6 corresponds to UW, which means that the physical camera corresponding to the ultra-wide-angle mode is used for shooting. From this, it can be seen that only one corresponding physical camera is called for shooting in the ultra-wide-angle mode.

[0288] in addition, Figure 8In order to meet the 3A constraint strategy and the SAT constraint strategy, the focus needs to be kept in the super telephoto mode for several frames. These frames can be set separately before starting the shooting mode switch, or they can be achieved through the Zoom Ratio. For example, Figure 8 The 6 in parentheses after 5.9625, 5.85, and 5.6625 indicates that the focal length can also be maintained at 6 through the Zoom Ratio.

[0289] Snapshot is used to indicate whether the user can use the shooting function at this focal length. Snapshot is 1, which means the user is allowed to use the shooting function. Snapshot is 0, which means the user is prohibited from using the shooting function. Figure 8 It can be seen that during the zooming process, the user is prohibited from using the shooting function through the stability constraint strategy and the smoothness constraint strategy.

[0290] In a specific implementation, the present application also provides a terminal, which includes one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the terminal to execute some or all of the steps in the above method embodiments.

[0291] The present application also provides a computer-readable storage medium including a computer program. When the computer program is executed on a terminal, the terminal executes some or all of the steps in the above method embodiment. The above storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0292] In a specific implementation, an embodiment of the present application further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a terminal, the terminal executes some or all of the steps in the above method embodiment.

[0293] like Figure 9 As shown, the present application also provides a chip system, which is applied to a terminal. The chip system includes one or more processors 901. The processor 901 is used to call computer instructions so that the terminal inputs the data to be processed into the chip system. The chip system processes the data based on the image display method provided in the embodiment of the present application and outputs the processing results.

[0294] In one possible implementation, the chip system further includes input and output interfaces for inputting and outputting data.

[0295] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0296] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0297] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0298] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, instructions can be distributed over a network or by other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, floppy disks, optical disks, optical disks, compact disc read-only memories (Compact Disc Read Only Memory, CD-ROMs), magneto-optical disks, read-only memories, random access memories, erasable programmable read-only memories (Erasable Programmable Read Only Memory, EPROM), electrically erasable programmable read-only memories (Electrically Erasable Programmable Read Only Memory, EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the Internet in electrical, optical, acoustic or other forms of propagation signals. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0299] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0300] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0301] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0302] Although the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. An image display method, characterized in that: The method is applied to a terminal, and includes: In response to a shooting mode switching operation, determining, based on a current first shooting mode and a desired second shooting mode, a first constraint strategy for image frames sent for display, a second constraint strategy for calling a physical camera, a third constraint strategy for an image processing method, and a fourth constraint strategy for an intermediate focal length for shooting when the shooting mode is switched; According to the fourth constraint strategy, a plurality of intermediate focal lengths for shooting during the shooting mode switching process are determined in a preset first correspondence between relative time and focal length; wherein the relative time is the time of shooting relative to the time of starting the shooting mode switching; and the plurality of intermediate focal lengths are different; According to the second constraint strategy, when each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and a target physical camera is called to shoot at the intermediate focal length; wherein the target physical camera is a physical camera selected from the first physical camera used in the first shooting mode and the second physical camera used in the second shooting mode; According to the third constraint strategy and the first constraint strategy, a target image frame that meets preset conditions is determined from the image frames taken by the target physical camera, and the target image frame is displayed in a preset preview area; wherein the clarity of the target image frame is higher than the clarity of other image frames.

2. The method according to claim 1, characterized in that When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length, including: When each first relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the first relative moment, and the first physical camera is called to shoot at the intermediate focal length; After the number of captured image frames reaches a first number, at each second relative moment, adjusting the focal length to an intermediate focal length corresponding to the second relative moment, and calling the first physical camera and the second physical camera to shoot at the intermediate focal length; The method further comprises: When the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by calling the second physical camera meets a preset condition, calling the first physical camera for shooting is stopped.

3. The method according to claim 2, characterized in that The first constraint strategy includes: a 3A consistency constraint strategy; the 3A consistency strategy includes: the number of image frames that are delayed before completing the shooting mode switch; The method of determining a target image frame that meets a preset condition from the image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes: Determining, according to the 3A consistency constraint strategy, the number of image frames that are delayed in display during the shooting mode switching process as the second number; For each first relative moment, if the image frame captured at the first relative moment meets a preset condition, the image frame captured at the first relative moment is displayed as a target image frame in a preset preview area; If the image frame captured at the first relative moment does not meet the preset condition, a second number of target image frames captured before the first relative moment that meet the preset condition are determined, and the determined target image frames are displayed in a preset preview area.

4. The method according to claim 3, characterized in that The third constraint strategy includes: a SAT constraint strategy; the SAT constraint strategy includes: pixel offset between center points of images captured by different physical cameras; The method of determining a target image frame that meets a preset condition from the image frames captured by the target physical camera according to the third constraint strategy and the first constraint strategy, and displaying the target image frame in a preset preview area, includes: Calculating, according to the SAT strategy, a pixel offset between a center point of an image frame captured by the first physical camera and a center point of an image frame captured by the second physical camera; Based on the pixel offset, processing the image frame captured by calling the second physical camera; For each second relative moment, if the image frame captured at the second relative moment meets a preset condition, the image frame captured at the second relative moment is displayed as a target image frame in a preset preview area; If the image frame captured at the second relative moment does not meet the preset conditions, a second number of target image frames meeting the preset conditions captured by the first physical camera before the second relative moment are determined, and the determined target image frames are displayed in a preset preview area.

5. The method according to claim 4, characterized in that Use the following steps to check whether the captured image frame meets the preset conditions: For each relative moment, cache the image metadata collected at the relative moment according to the corresponding intermediate focal length in the first storage area; sequentially fetching the image metadata from the first storage area in the order in which the image metadata are cached, processing the metadata to obtain corresponding image frames, and storing the obtained image frames in the second storage area; When the focal length corresponding to the image metadata in the first storage area is the same as the focal length corresponding to the image frame in the second storage area, it is determined that the image frame captured at the relative moment meets the preset condition.

6. The method according to claim 5, characterized in that The first storage area is the storage area corresponding to the Result object called by the Pipeline image processing method; the second storage area is the storage area corresponding to the Request object called by the Pipeline image processing method.

7. The method according to claim 2, characterized in that The second constraint strategy includes: a focal length constraint strategy; the focal length constraint strategy includes: a second correspondence between a physical camera in the terminal and a shooting mode; Before calling the second physical camera to shoot, the method further includes: According to the focal length constraint strategy, a physical camera corresponding to the second shooting mode is determined from the physical cameras in the terminal as the second physical camera.

8. The method according to claim 1, characterized in that The fourth constraint strategy includes: a center point constraint strategy; the center point constraint strategy includes: the number of intermediate focal lengths between the focal lengths used in different shooting modes during the shooting mode switching process; The determining, according to the fourth constraint strategy, a plurality of intermediate focal lengths for shooting during the shooting mode switching process in the first correspondence between the preset relative moments and the focal lengths includes: determining, according to the center point constraint strategy, a number of a plurality of intermediate focal lengths between the focal length used in the first shooting mode and the focal length used in the second shooting mode as a third number; In the preset first correspondence between relative time and focal length, a third number of intermediate focal lengths are determined.

9. The method according to claim 8, characterized in that After determining a third number of intermediate focal lengths in the first correspondence between the preset relative moments and the focal lengths, the method further includes: Calculating the number of image frames required to be captured at each intermediate focal length as a fourth number based on a third number of the plurality of intermediate focal lengths and the number of image frames required to be captured during the shooting mode switching process; When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length, including: When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to capture a fourth number of image frames at the intermediate focal length.

10. The method according to claim 2, characterized in that The changing relationship between the relative time and the focal length in the first corresponding relationship is a quadratic curve, or the changing relationship between the relative time and the focal length in the first corresponding relationship is a Bezier curve.

11. The method according to any one of claims 1 to 10, characterized in that The identified constraint strategies also include stability constraint strategies; The stability constraint strategy includes: prohibiting the user from using the shooting function during the shooting mode switching process; The method further comprises: During the switching from the first shooting mode to the second shooting mode, responding to the shooting operation is prohibited.

12. The method according to any one of claims 1 to 10, characterized in that The determined constraint strategy also includes a power consumption constraint strategy; the power consumption constraint strategy includes: when the shooting mode is not switched, the number of physical cameras called is 1; The method further comprises: When the shooting mode is not switched, it is prohibited to call other physical cameras except the physical camera called by the current shooting mode.

13. The method according to any one of claims 1 to 10, characterized in that The method further comprises: After successfully switching to the second shooting mode, the target image frame shot by the second physical camera is called to be displayed in a preset preview area.

14. The method according to claim 1, wherein The first shooting mode and the second shooting mode are separated by a third shooting mode; When each relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the relative moment, and the target physical camera is called to shoot at the intermediate focal length, including: When each first relative moment is reached, the focal length is adjusted to an intermediate focal length corresponding to the first relative moment, and the first physical camera is called to shoot at the intermediate focal length; After the number of captured image frames reaches a first number, at each third relative moment, adjusting the focal length to an intermediate focal length corresponding to the third relative moment, and calling the first physical camera and the third physical camera used in the third shooting mode to shoot at the intermediate focal length; After the number of captured image frames reaches a fifth number, at each second relative moment, adjusting the focal length to an intermediate focal length corresponding to the second relative moment, and calling the third physical camera and the second physical camera to shoot at the intermediate focal length; The method further comprises: When the focal length is adjusted to the focal length used in the second shooting mode and the image frame shot by calling the second physical camera meets a preset condition, calling the third physical camera for shooting is stopped.

15. The method according to claim 14, characterized in that The first shooting mode is: ultra-wide-angle mode; the second shooting mode is: ultra-telephoto mode.

16. The method according to claim 14, characterized in that The first shooting mode is: ultra-telephoto mode; the second shooting mode is: ultra-wide-angle mode.

17. A terminal, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and is configured to store computer program codes, where the computer program codes include computer instructions. The one or more processors invoke the computer instructions to enable the terminal to execute the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that The method comprises a computer program, which, when executed on a terminal, causes the terminal to execute the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that The computer program product comprises executable instructions, and when the executable instructions are executed on a terminal, the terminal is caused to execute the method according to any one of claims 1 to 16.

20. A chip system, characterized in that: The chip system is applied to a terminal, and the chip system includes one or more processors, which are used to call computer instructions to enable the terminal to input data into the chip system, and execute the method described in any one of claims 1-16 to process the data and output the processing results.

Images