Shooting method and electronic equipment

Abstract

The invention discloses a shooting method and electronic equipment, the electronic equipment can default to start only one camera (namely the current camera), when the operation of clicking zoom by a user is detected, the electronic equipment can start the other camera (namely the target camera), and the zoom operation of the target camera is carried out based on images acquired by the two cameras. According to the embodiment of the invention, the electronic equipment can enable the FOV center of the image collected by the current camera to be gradually close to the FOV center of the image collected by the target camera by executing the FOV center offset algorithm, so that smooth transition of the FOV centers is realized, FOV center jump is avoided, and meanwhile, another camera is started in a click zoom scene, so that the power consumption of the electronic equipment is saved.

Description

[0001] This application is a divisional application. The application number of the original application is 2021106121480, and the original application date is May 31, 2021. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a shooting method and electronic equipment. Background Art

[0003] With the development of electronic technology, some electronic devices such as mobile phones and tablets are equipped with more and more cameras, which can achieve zoom shooting with a larger magnification range (for example, 0.5x to 30x), meeting the various shooting needs of users. However, the center jump of the field of view (FOV) will occur during the zoom process, resulting in poor user experience. In addition, in scenes where zoom shooting is not required, some electronic devices will turn on all cameras, resulting in high power consumption of the electronic devices. Summary of the invention

[0004] The embodiments of the present application provide a shooting method and an electronic device, which can avoid FOV center jumps during zoom shooting, achieve smooth FOV transition, improve user experience, and, moreover, can turn on the corresponding camera on the electronic device as needed, saving power consumption of the electronic device.

[0005] In a first aspect, an embodiment of the present application provides a shooting method, which is applied to an electronic device including a display screen and multiple cameras, the multiple cameras including a first camera and a second camera, the method comprising: the electronic device displays a first preview image at a first zoom ratio on the display screen, and the first preview image is captured by the first camera; the electronic device detects a user's click operation on a second zoom ratio option; the electronic device generates and displays at least one second preview image on the display screen based on at least one third zoom ratio, and the at least one third zoom ratio is between the first zoom ratio and the second zoom ratio; the electronic device displays a third preview image at the second zoom ratio on the display screen, and the third preview image is captured by the second camera.

[0006] By providing the shooting method, the electronic device can, based on the images captured by the above two cameras, execute the FOV center offset algorithm so that the FOV center of the image captured by the current camera gradually approaches the FOV center of the image captured by the target camera, thereby achieving a smooth transition of the FOV center and avoiding FOV center jumps.

[0007] In a possible implementation, the click operation includes: the user's finger or handwriting device first contacts the second zoom ratio option, and then leaves the second zoom ratio option.

[0008] In this way, the click operation mentioned in the embodiment of the present application can be distinguished from the operation of the user's finger or handwriting device simply touching the second zoom ratio option first.

[0009] In one possible implementation, the electronic device generates and displays on a display screen at least one second preview image based on at least one third zoom ratio, specifically including: when the electronic device detects that a user's finger or handwriting device touches a second zoom ratio option, the electronic device starts to generate and display on the display screen at least one second preview image based on at least one third zoom ratio.

[0010] In this way, more internal processing time can be reserved for the electronic device, so that the electronic device can respond to user needs more quickly after the user performs an operation.

[0011] In a possible implementation, the method for determining the number N of second preview images is as follows:

[0012] N=T / t

[0013] Among them, T is the time interval between the electronic device detecting the user's click operation on the second zoom ratio option and the electronic device displaying the third preview image at the second zoom ratio on the display screen, and t is the time interval between two adjacent preview frames of the current camera system.

[0014] In this way, the number of image frames that need to be displayed on the display screen can be calculated.

[0015] In a possible implementation, a method for determining the zoom value zoomValue corresponding to the third zoom ratio is as follows:

[0016]

[0017] Wherein, S is the distance between the first zoom ratio option and the second zoom ratio option on the display screen, and n is a positive integer less than the number N of the second preview images.

[0018] In this way, the zoom value corresponding to the third zoom ratio can be calculated.

[0019] In a possible implementation, the FOV center of the first preview image is consistent with the FOV center of the first camera, and the FOV center of the third preview image is consistent with the FOV center of the second camera.

[0020] In this way, it can be ensured that the FOV center of the image captured by the camera coincides with the FOV center of the preview image.

[0021] In a possible implementation, before the electronic device detects a click operation of the user on the second zoom ratio option, the method further includes: the electronic device powers on the second camera to start and run the second camera.

[0022] In this way, the electronic device can run multiple cameras at the same time, saving the time of turning on the camera in scenarios where zooming is required.

[0023] In a possible implementation manner, the FOV center of at least one second preview image gradually approaches the FOV center of the second camera.

[0024] In this way, smooth zooming can be achieved and FOV center jumps can be avoided.

[0025] In a possible implementation, after the electronic device detects a click operation of the user on the second zoom ratio option, the method further includes: the electronic device powers on the second camera, and starts and runs the second camera.

[0026] In this way, the camera can be turned on as needed, saving power consumption of electronic devices.

[0027] In one possible implementation, before the electronic device runs the second camera, the FOV center of at least one second preview image is consistent with the FOV center of the first camera; after the electronic device runs the second camera, the FOV center of at least one second preview image begins to gradually approach the FOV center of the second camera.

[0028] In this way, while saving the power consumption of the electronic device, smooth zooming can be achieved and FOV center jumps can be avoided.

[0029] In a second aspect, an embodiment of the present application provides an electronic device, comprising a display screen, multiple cameras with different focal lengths, a memory and a processor coupled to the memory, multiple applications, and one or more programs; wherein the optical centers of the multiple cameras do not overlap, the multiple cameras include a first camera and a second camera, the first camera and the second camera are two cameras with adjacent focal lengths among the multiple cameras, and when the processor runs one or more programs, the electronic device executes a method as any possible implementation method of any of the above aspects.

[0030] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes a method as any possible implementation method of any of the above aspects.

[0031] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute a method in any possible implementation of any of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0033] Figure 2A-2B It is a schematic diagram of the appearance structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 3A-3B is a user interface schematic diagram of a typical shooting scene provided in an embodiment of the present application;

[0035] Figure 4A-4H is a schematic diagram of a user interface for zooming in and out in a preview scenario provided in an embodiment of the present application;

[0036] Figure 5A-5H is a schematic diagram of a user interface for zoom reduction in a preview scenario provided in an embodiment of the present application;

[0037] Figure 6A-6H is a schematic diagram of a user interface for zooming in a video recording scenario provided in an embodiment of the present application;

[0038] Figure 7 It is a flowchart of a method for achieving smooth transition of the FOV center in a point-to-zoom scenario provided by an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of a specific execution process of the FOV center offset algorithm provided in an embodiment of the present application;

[0040] Figure 9A-9D is a schematic diagram of the states of multiple cameras being turned on and off provided in an embodiment of the present application;

[0041] Fig.10 It is a schematic diagram of the cooperation of some software and hardware of an electronic device when zooming up provided by an embodiment of the present application;

[0042] Fig.11 It is a schematic diagram of the cooperation of some software and hardware of an electronic device when zoom is reduced provided by an embodiment of the present application;

[0043] Fig.12 It is a flowchart of a shooting method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0045] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0046] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0047] The embodiment of the present application provides a shooting method, which can avoid FOV center jump during zoom shooting, achieve FOV smooth transition, and improve user experience. Moreover, the corresponding camera on the electronic device can be turned on as needed, saving power consumption of the electronic device.

[0048] The shooting method provided in the embodiment of the present application can be applied to electronic devices with multiple cameras with different focal lengths. These multiple cameras can be ordinary cameras, telephoto cameras, wide-angle cameras, etc. Since the positions of these multiple cameras on the electronic device are different, the optical centers of these multiple cameras do not overlap, resulting in inconsistent FOV centers of these multiple cameras. Assume that the optical zoom ratio of the ordinary camera is 1x (expressed as 1X), the optical zoom ratio of the telephoto camera is 5x, expressed as 5X (subsequent X represents the zoom ratio), and the optical zoom ratio of the wide-angle camera is 0.4X. Generally, since the optical centers of these multiple cameras do not overlap, the optical zoom process involving camera switching will cause FOV center jumps. For example, in the process of switching from an ordinary camera to a telephoto camera, the center point of the shot will fall on a significantly different position, affecting the user's focus. At present, in scenes where zoom shooting is not required, some electronic devices will also turn on all cameras, resulting in high power consumption of the electronic equipment.

[0049] The shooting method provided by the embodiment of the present application can solve the problem of FOV center jump occurring during the optical zoom process. Moreover, in the scenario where zooming is not required, only one default camera can be turned on, and in the scenario where zooming is required, other corresponding cameras on the electronic device can be turned on to save power consumption of the electronic device. The electronic device can turn on only one camera with a larger FOV (such as an ordinary camera) by default. When the user increases the zoom ratio, the electronic device can turn on another camera with a smaller FOV (such as a telephoto camera). After the camera with a smaller FOV is running, the electronic device can use the FOV center offset algorithm to crop the image captured by the camera with a larger FOV, so that the FOV center of the cropped image gradually approaches the FOV center of the camera with a smaller FOV, so as to achieve a smooth transition of the FOV and avoid FOV center jump. Here, the camera with a larger FOV and the camera with a smaller FOV refer to the two cameras involved in a camera switch, such as switching from an ordinary camera to a telephoto camera. The FOV center offset algorithm is used to perform eccentric cropping on the image taken by the camera with a larger FOV. It mainly means that during the zooming process, as the zoom ratio approaches the zoom ratio of the camera switching, the image taken by the camera with a larger FOV is cropped multiple times in succession, so that the FOV center of each cropped image (for example, the image cropped at 3.1X, 3.2X, 3.3X, ... 4.9X) gradually approaches the FOV center of the camera with a smaller FOV, and even the image cropped at 4.9X can overlap with the FOV center of the telephoto camera, thereby avoiding FOV center jumps. The implementation of eccentric cropping of images taken by a camera with a larger FOV based on the FOV center offset algorithm will be described in detail in the following content, which will not be expanded here.

[0050] In this way, when the user increases the zoom ratio (such as from 1X to 5X) to achieve the purpose of observing distant scenes in detail, the shooting method provided by the embodiment of the present application facilitates the user to "lock" a certain target scene during the zoom process, avoiding the situation of "losing" the target scene due to camera switching, and also avoids the user from repeatedly framing and focusing, thereby improving the efficiency and convenience of zoom shooting. At the same time, when zooming is required, other corresponding cameras on the electronic device are turned on, saving power consumption of the electronic device.

[0051] Here we only take increasing the zoom ratio as an example. The shooting method provided in the embodiment of the present application is also applicable to the usage scenario of reducing the zoom ratio (for example, switching from a normal camera to a wide-angle camera).

[0052] The above-mentioned electronic device can be a mobile phone, a tablet computer, a wearable device, a car-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a dedicated camera (such as a SLR camera, a card camera), etc. This application does not impose any restrictions on the specific types of the above-mentioned electronic devices.

[0053] In an embodiment of the present application, when the zoom ratio is increased, the zoom ratio of the camera switching is 4.9X-5X. The camera switching specifically refers to the switching from a normal camera to a telephoto camera. That is to say, the zoom ratio switching point when switching from a normal camera to a telephoto camera is 5X.

[0054] Figure 1 The structure of an electronic device 100 provided in an embodiment of the present application is exemplarily shown.

[0055] like Figure 1 As shown, the electronic device 100 may have multiple cameras 193, such as a normal camera, a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, etc. The smaller the focal length of the camera 193, the larger its viewing angle, the larger the viewing range, and more scenes can be captured. Conversely, the larger the focal length of the camera 193, the smaller its viewing angle, the smaller the viewing range, and fewer but farther scenes can be captured. For example, the focal length of an ultra-wide-angle camera is generally about 12 millimeters (mm)-24mm, and the viewing angle of an ultra-wide-angle camera is generally 84°-120°; the focal length of a wide-angle camera is generally about 24mm-35mm, and the viewing angle of a wide-angle camera is generally 63°-84°; the focal length of an ordinary camera is generally around 50mm, and the viewing angle of an ordinary camera is generally around 46°; the focal length of a telephoto camera is generally about 135mm-500mm, and the viewing angle of a telephoto camera is generally 5°-18°; the focal length of an ultra-telephoto camera is generally more than 500mm, and the viewing angle of an ultra-telephoto camera is generally 0°-5°. The performance of these cameras in terms of viewing angle is as follows: an ultra-wide-angle camera is better than a wide-angle camera, a wide-angle camera is better than an ordinary camera, an ordinary camera is better than a telephoto camera, and a telephoto camera is better than an ultra-telephoto camera.

[0056] Since the positions of these multiple cameras on the electronic device are different, the optical centers of these multiple cameras do not overlap, resulting in inconsistent FOV centers of these multiple cameras. Assume that the optical zoom ratio of the normal camera is 1X, the optical zoom ratio of the telephoto camera is 5X, and the optical zoom ratio of the wide-angle camera is 0.4X. Generally, since the optical centers of these multiple cameras do not overlap, the optical zoom process involving camera switching will cause FOV center jumps. For example, in the process of switching from a normal camera to a telephoto camera, the center point of the shot will fall at a significantly different position, affecting the user's focus.

[0057] In addition, the electronic device 100 may also include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0058] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0059] In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0060] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0061] The charging management module 140 is used to receive charging input from a charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. In some other embodiments, the power management module 141 can also be set in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0062] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0063] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to electronic device 100. Wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to electronic device 100. In some embodiments, antenna 1 of electronic device 100 is coupled with mobile communication module 150, and antenna 2 is coupled with wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0064] The wireless communication technologies mentioned above may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation systems (SBAS).

[0065] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area.

[0066] The electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor. For example, music playing, recording, etc. The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0067] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0068] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100, such as the angular velocity of the electronic device 100 around three axes (i.e., x, y and z axes). The air pressure sensor 180C is used to measure the air pressure. The magnetic sensor 180D includes a Hall sensor. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focus.

[0069] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The ambient light sensor 180L is used to sense the brightness of ambient light. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K may be arranged on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The bone conduction sensor 180M may obtain a vibration signal.

[0070] The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0071] The electronic device 100 can realize the shooting function through an image signal processor (ISP), a camera 193, a video codec, a GPU, a display screen 194 and an application processor.

[0072] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. The ISP is not limited to being integrated in the processor 110, and can also be set in the camera 193.

[0073] The camera 193 includes a lens and a photosensitive element (also called an image sensor) for capturing still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the ISP for conversion into a digital image signal, such as an image signal in a standard RGB, YUV, or other format.

[0074] In some embodiments, the camera 193 can be used to collect depth data. For example, the camera 193 can have a time of flight (TOF) 3D sensing module or a structured light 3D sensing module for acquiring depth information. The camera used to collect depth data can be a front camera or a rear camera.

[0075] The video codec is used to compress or decompress digital images. The electronic device 100 may support one or more image codecs. In this way, the electronic device 100 may open or save pictures or videos in multiple encoding formats.

[0076] The electronic device 100 can realize the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0077] The display screen 194 is used to display images, videos, etc., such as images captured by the camera 193. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or more display screens 194.

[0078] It should be understood that Figure 1 The electronic device 100 shown is only an example, and the electronic device 100 may have more Figure 1 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0079] Figure 2A and Figure 2B The external structure of an electronic device 100 provided in an embodiment of the present application is exemplarily shown. Figure 2A The side of the electronic device 100 where the display screen 194 is located is shown. Figure 2B The side where the back cover of the electronic device 100 is located is shown.

[0080] The electronic device 100 may have multiple cameras 193. The electronic device 100 may include multiple front cameras. Figure 2A As shown, the front camera 193-1 and the front camera 193-2 can be arranged at the top of the electronic device 100, such as the "bangs" position of the electronic device 100 (i.e. Figure 2A In addition to the camera 193, the area AA may also include a speaker 170A, etc. Figure 2BAs shown, the electronic device 100 may include multiple rear cameras, such as a rear camera 193-3, a rear camera 193-4, and a rear camera 193-5. The rear camera 193-3, the rear camera 193-4, and the rear camera 193-5 may be a normal camera, a wide-angle camera, and a telephoto camera, respectively. A flash 196 may also be configured near the camera 193.

[0081] The camera 193 can change the viewing angle of the preview image in the preview frame by digital zoom, can also change the viewing angle of the preview image in the preview frame by optical zoom, and can also change the viewing angle of the preview image by combining optical zoom and digital zoom (also called hybrid zoom). In other words, zooming can include digital zoom, optical zoom or hybrid zoom. Hybrid zoom is taken as an example below.

[0082] The electronic device 100 can realize that the size of the preview viewing angle presented by a series of images displayed in the preview frame is gradually changed by changing the camera used for shooting among the multiple cameras 193 and combining digital zoom. The above-mentioned camera used for shooting can refer to the camera whose captured image is displayed in the preview frame. The above-mentioned digital zoom can enlarge the area of ​​each pixel in the image captured by the camera 193 for the electronic device 100 to achieve the purpose of changing the focal length. This is equivalent to the electronic device 100 cropping the image captured by a camera, and then enlarging the cropped image, that is, adjusting the resolution of the cropped image to the same resolution as the image before cropping. In this way, compared with the image before cropping, the area of ​​each pixel in the cropped and enlarged image becomes larger. And the image within the partial viewing angle is cropped, so that the preview viewing angle presented by the cropped and enlarged image becomes smaller, which is similar to the effect of increasing the focal length. But in fact, the above-mentioned digital zoom does not change the focal length of the above-mentioned one camera.

[0083] Assuming that the image captured by the ordinary camera 193-3 is displayed in the current preview frame, the current zoom ratio is 1X, and the telephoto camera 193-5 is not turned on, when the electronic device 100 detects that the user clicks the zoom point "5X", the electronic device 100 will start the telephoto camera 193-5 and zoom in a certain zoom step (for example, 0.1X) between 1X and 5X (for example, from 1X, 1.1X, 1.2X...4.9X to 5X). In the process of zooming from 1X to 5X, the electronic device 100 can first crop the image captured by the ordinary camera 193-3, and the cropping at this stage can be centered cropping. When the telephoto camera 193-5 is running, the electronic device 100 can use the FOV center offset algorithm to eccentrically crop the image captured by the ordinary camera 193-3, so that the FOV center of the cropped image gradually approaches the FOV center of the telephoto camera 193-5, and the FOV center changes more smoothly, rather than suddenly jumping to the FOV center of the telephoto camera 193-5. That is, the electronic device 100 performs multiple digital zooms on the image captured by the ordinary camera 193-3. When the zoom ratio reaches the optical zoom ratio of the telephoto camera 193-5 (for example, 5X), the electronic device 100 switches to the telephoto camera 193-5, and the image displayed in the preview frame becomes the image captured by the telephoto camera 193-5, that is, the electronic device 100 performs optical zoom. That is to say, when the zoom ratio is continuously increased, the image display can include two stages: the stage of capturing images using the ordinary camera 193-3 and the stage of capturing images using the telephoto camera 193-5. During the stage of capturing images using the ordinary camera 193-3, the zoom ratio gradually increases from 1X, for example, from 1X to 3X, and from 3X to 4.9X. When the ratio increases to 5X, it switches to the stage of capturing images using the telephoto camera 193-5.

[0084] When using a camera to capture images, the FOV presented by the image from the camera displayed in the preview frame may generally be smaller than the FOV of this camera. At this time, the image in the preview frame is cropped from the image captured by the camera. When the zoom ratio is changed to a specific ratio, the FOV presented by the image displayed in the preview frame can be as large as the FOV of this camera. For example, in the above-mentioned stage of using the image captured by the ordinary camera 193-3 (i.e., 1X to 4.9X), the FOV presented by the image displayed in the preview frame is generally smaller than the FOV of the ordinary camera 193-3. Even when the zoom ratio is 1X, the FOV presented by the image displayed in the preview frame may be smaller than the FOV of the ordinary camera 193-3.

[0085] How the electronic device 100 performs off-center cropping of the image captured by the ordinary camera 193 - 3 based on the FOV center offset algorithm will be described in detail in subsequent embodiments and will not be expanded here.

[0086] A typical shooting scene involved in this application is described below.

[0087] like Figure 3A-3B As shown, the electronic device 100 can detect the user's operation of opening the "Camera" application, for example, Figure 3A In response to this operation, the electronic device 100 may display Figure 3B The exemplary user interface 301 is a user interface of a “camera” application. Figure 3A The main interface shown may include a status bar 211, a tray 215 with a list of commonly used applications, a calendar indicator 212, a weather indicator 213, a navigation bar 216, and other application icons 214, etc. "Camera" is an image capture application on an electronic device 100 such as a smartphone or tablet computer, and the application does not limit the name of the application. Figure 3A-3B As shown, the user can also open the user interface 301 in other applications.

[0088] User interface 301 may be a user interface of a default camera mode of a "camera" application. The default camera mode may be a default camera mode of a rear-mounted ordinary camera, or may be other modes, which are not limited here. Figure 3B As shown, the user interface 301 may include: a setting control 310, a flash control 309, a zoom bar 308, a zoom ratio 307, a preview box 306, a camera flip control 305, a gallery shortcut control 304, a shutter control 303, and a camera mode option 302. Among them:

[0089] The setting control 310 can be used to adjust the parameters of taking photos (such as resolution, filter, etc.) and turn on or off some modes for taking photos (such as timed photo taking, smile capture, voice-activated photo taking, etc.). The setting control 310 can be used to set more other shooting functions, which are not limited in the embodiments of the present application.

[0090] The flash control 309 can be used to turn the flash on or off.

[0091] There are multiple zoom points on the zoom bar 308 for indicating different zoom ratios. Different zoom points indicate different zoom ratios.

[0092] The zoom ratio 307 may be used to indicate the current zoom ratio. The larger the zoom ratio 307 is, the smaller the FOV presented by the image displayed in the preview frame 306 is. Conversely, the smaller the zoom ratio 307 is, the larger the FOV presented by the image displayed in the preview frame 306 is. Figure 3B As shown, 1X can be the default zoom ratio of the camera application. The default zoom ratio can also be other values, and the embodiment of the present application does not limit the default zoom ratio.

[0093] The preview frame 306 may be used to display the image captured in real time by the camera 193. The electronic device 100 may refresh the display content therein in real time so that the user can preview the image currently captured by the camera 193.

[0094] The camera switching control 305 may be used to monitor a user operation that triggers camera switching. In response to the operation, the electronic device 100 may switch cameras, for example, switching a rear camera to a front camera.

[0095] The gallery shortcut key 304 can be used to open the gallery application. In response to a user operation on the gallery shortcut key 304, such as a click operation, the electronic device 100 can open the gallery application. In this way, the user can conveniently view the photos and videos taken without having to exit the camera application first and then open the gallery application. The gallery application is an image management application on electronic devices such as smartphones and tablets, and can also be called an "album". This embodiment does not limit the name of the application. The gallery application can support users to perform various operations on pictures stored on the electronic device 100, such as browsing, editing, deleting, selecting, etc. In addition, the electronic device 100 can also display thumbnails of the saved images in the gallery shortcut key 304.

[0096] The shutter control 303 may be used to monitor a user operation that triggers a photo. In response to the operation, the electronic device 100 may save the image in the preview box 307 as a picture in a gallery application.

[0097] One or more shooting mode options may be displayed in the camera mode option 302. The one or more shooting mode options may include: a large aperture mode option 302A, a video recording mode option 302B, a photo mode option 302C, a portrait mode option 302D, and more options 302E. When a user operation on a shooting mode option is detected, the electronic device 100 may turn on the shooting mode selected by the user. In particular, when a user operation on more options 302E is detected, the electronic device 100 may further display more other shooting mode options, such as a slow motion shooting mode option, etc., to show the user more abundant camera functions. Not limited to Figure 3BAs shown, the more options 302E may not be displayed in the camera mode option 302 , and the user may browse other shooting mode options by sliding left / right in the camera mode option 302 .

[0098] Based on the above shooting scenarios, some user interfaces (UI) implemented on the electronic device 100 are introduced below.

[0099] Figure 4A-4H The user interface for zooming in on a preview scene is shown as an example. Figure 4A-4H In the embodiment, assuming that the optical zoom magnification of the telephoto camera is 5X, and the optical zoom magnification of the normal camera is 1X, the camera turned on by default in the electronic device 100 is the normal camera. After detecting that the user presses the 5X zoom point on the zoom bar 308, the electronic device 100 turns on the telephoto camera.

[0100] Figure 4A A preview scenario is exemplarily shown: the image displayed in the preview frame 306 (which may be referred to as a preview image) is from a common camera, and the zoom ratio 307 is 1X.

[0101] exist Figure 4A In the preview scenario shown, the electronic device 100 can detect the user's operation of increasing the zoom ratio (for example, the user clicks on the 5X zoom point on the zoom bar 308). In response to this operation, the electronic device 100 can reduce the FOV presented by the preview image. At the same time, the zoom ratio 307 displayed in the preview frame 306 will gradually increase. When the zoom ratio increases to 5X, the electronic device 100 can display the image captured by the telephoto camera in the preview frame 306, that is, the preview image is switched to the telephoto camera, that is, the electronic device 100 performs optical zoom. Assuming that the minimum change unit of the zoom ratio is 0.1X, then 4.9X-5X is the optical zoom ratio when the ordinary camera switches to the telephoto camera.

[0102] like Figure 4A-4B As shown, illustratively, during the process of increasing the zoom ratio 307 from 1X to 3X, the ordinary camera is always in the running state, and the telephoto camera is in the startup state and has not yet entered the running state. Therefore, during this process, only the ordinary camera collects images, and the electronic device 100 gradually reduces the FOV presented by the preview image (i.e., the telephoto camera 100 is in the startup state and has not yet entered the running state). Figure 4E The image M shown is cropped in the center), and the FOV center O1 presented in the preview image at 1X coincides with the FOV center O2 presented in the preview image at 3X.

[0103] like Figure 4C-4DAs shown, illustratively, when the zoom ratio 307 increases to 3.1X, the ordinary camera is in operation, and the telephoto camera begins to enter the operation state, that is, the ordinary camera and the telephoto camera begin to collect images at the same time. In the process of increasing the zoom ratio 307 from 3.1X to 4.9X, the ordinary camera and the telephoto camera are always in operation and collect images at the same time, until the zoom ratio 307 increases to 5X, the ordinary camera is turned off, and the telephoto camera is still in operation. At this time, only the telephoto camera collects images. In the process of increasing the zoom ratio 307 from 3.1X to 5X, in addition to gradually reducing the FOV presented by the preview image, the electronic device 100 can also move the FOV center of the preview image toward the FOV center of the telephoto camera, instead of directly jumping from the FOV center of the ordinary camera to the FOV center of the telephoto camera. As the zoom ratio continues to increase to the optical zoom ratio of 5X where the telephoto camera is located, the FOV center of the preview image will gradually approach, and even eventually overlap with the FOV center of the telephoto camera, avoiding FOV center jumps.

[0104] Specifically, Figure 4A , Figure 4B , Figure 4C , Figure 4D The preview images at 1X, 3X, 3.1X, and 5X are shown, and their FOV centers are at positions O1, O2, O3, and O4, respectively. It can be seen that O3 is closer to O4 than O2, so when the zoom ratio increases from 1X to 5X, the FOV center of the preview image will not suddenly change from O1 to O4, but will transition more smoothly without jumping.

[0105] Combine the following Figure 4E-4H ,illustrate Figure 4A-4D The implementation principle of the zoom enlargement process is shown by way of example.

[0106] like Figure 4E-4H As shown, image M is an image captured by an ordinary camera, FOV1 is the FOV of the ordinary camera, and FOV2 is the FOV of the telephoto camera. The FOV of the ordinary camera covers the FOV of the telephoto camera, and the distance between the FOV center O4 of the telephoto camera and the FOV center O1 of the ordinary camera is relatively far. This farther distance is caused by the fact that the optical centers of the telephoto camera and the ordinary camera do not overlap. In order to avoid this causing the FOV center to jump during the zooming process, when cropping the image M captured by the ordinary camera, the electronic device 100 can gradually move the cropping center of the cropping area closer to the FOV center of the telephoto camera, that is, execute the FOV center offset algorithm to achieve eccentric cropping.

[0107] Specifically, Figure 4EAs shown, Image 1 is a preview image at 1X. Image 1 can be obtained by cropping Image M. The FOV center of Image 1 coincides with the center O1 of FOV1. Figure 4F As shown, Image 2 is a preview image at 3X, and Image 2 can be obtained by cropping Image M. The FOV center of Image 2 coincides with the center O1 of FOV1. Figure 4G As shown, Image 3 is a preview image at 3.1X, and Image 3 can be obtained by cropping Image M. The FOV center of Image 3 no longer coincides with the center O1 of FOV1, but deviates from O1 and is closer to the FOV center O4 of the telephoto camera. Figure 4H As shown, image T is a preview image at 5X, and image T is an image captured by a telephoto camera. The FOV center of image T coincides with the FOV center O4 of the telephoto camera.

[0108] Figure 4E-4H Only by taking the execution of the FOV center offset algorithm when the telephoto camera starts to run as an example, an implementation method of the FOV center offset algorithm is explained, that is, the electronic device 100 executes the FOV center offset algorithm during the entire digital zoom process from 3X to 3.1X, 3.2X, ..., to 4.9X. Not limited to this, the electronic device 100 may also execute the FOV center offset algorithm for a period of time after the telephoto camera starts to run (for example, after the electronic device 100 zooms to 4X), for example, the electronic device 100 may execute the FOV center offset algorithm during the entire digital zoom process from 4X to 4.1X, 4.2X, ..., to 4.9X. In addition, the embodiment of the present application does not limit the speed of using the FOV center offset algorithm to cut toward the FOV center of the telephoto camera. For example, the electronic device 100 may move toward the FOV center of the telephoto camera once for every 0.1X change in zoom ratio, or may move toward the FOV center of the telephoto camera once for every 0.2X change in zoom ratio.

[0109] In some embodiments, the FOV of image 1 may also be FOV1, that is, image M may be a preview image at 1X.

[0110] Figure 4A-4H The method of increasing the zoom ratio described in the embodiment is also applicable to the scene where the wide-angle camera switches to the ordinary camera, the scene where the ultra-wide-angle camera switches to the wide-angle camera, and the scene where the telephoto camera switches to the ultra-telephoto camera, but the embodiment of the present application is not limited to this.

[0111] It should be noted that the embodiment of the present application only takes the example of starting to operate the telephoto camera after zooming to 3X to illustrate one implementation method of the FOV center offset algorithm. In some embodiments, the telephoto camera can also be started after zooming to other magnifications (for example, 2X, 3.5X, 4X, etc.), and the embodiment of the present application is not limited to this.

[0112] Figure 5A-5H The user interface for zoom reduction in the preview scene is shown as an example. Figure 5A-5H In the embodiment, assuming that the optical zoom magnification of the ordinary camera is 1X, and the optical zoom magnification of the wide-angle camera is 0.4X (wide), the camera turned on by default by the electronic device 100 is the ordinary camera. After detecting that the user presses the 0.4X zoom point on the zoom bar 308, the electronic device 100 turns on the wide-angle camera.

[0113] Figure 5A A preview scenario is exemplarily shown: the image displayed in the preview frame 306 (which may be referred to as a preview image) is from a common camera, and the zoom ratio 307 is 1X.

[0114] exist Figure 5A In the preview scenario shown, the electronic device 100 can detect the user's operation of reducing the zoom ratio (for example, the user clicks on the 0.4X zoom point on the zoom bar 308). In response to this operation, the electronic device 100 can expand the FOV presented by the preview image. At the same time, the zoom ratio 307 displayed in the preview frame 306 will gradually decrease. When the zoom ratio is reduced to 0.4X, the electronic device 100 can display the image captured by the wide-angle camera in the preview frame 306, that is, the preview image is switched to the wide-angle camera, that is, the electronic device 100 performs optical zoom. Assuming that the minimum change unit of the zoom ratio is 0.1X, then 0.5X-0.4X is the zoom ratio when the ordinary camera switches to the wide-angle camera.

[0115] like Figure 5A-Figure 5B As shown, for example, during the process of the zoom ratio 307 decreasing from 1X to 0.9X, the ordinary camera is always in the running state, and the wide-angle camera is in the startup state and has not yet entered the running state. Therefore, during this process, only the ordinary camera collects images, and the electronic device 100 gradually expands the FOV presented by the preview image (i.e., the wide-angle camera 100 is in the startup state and has not yet entered the running state). Figure 5E The image M' shown is cropped in the center), and the FOV center O1 presented in the preview image at 1X coincides with the FOV center O2 presented in the preview image at 0.9X.

[0116] like Figure 5C-5DAs shown, illustratively, when the zoom ratio 307 is reduced to 0.8X, the ordinary camera is in operation, and the wide-angle camera begins to enter the operation state, that is, the ordinary camera and the wide-angle camera begin to collect images at the same time. In the process of reducing the zoom ratio 307 from 0.8X to 0.4X, the ordinary camera and the wide-angle camera are always in operation and collect images at the same time, until the zoom ratio 307 is reduced to 0.4X, the ordinary camera is turned off, and the wide-angle camera is still in operation. At this time, only the wide-angle camera collects images. In the process of reducing the zoom ratio 307 from 0.8X to 0.4X, in addition to gradually expanding the FOV presented by the preview image, the electronic device 100 can also move the FOV center of the preview image toward the FOV center of the wide-angle camera, instead of directly jumping from the FOV center of the ordinary camera to the FOV center of the wide-angle camera. As the zoom ratio continues to decrease to the optical zoom ratio of 0.4X where the wide-angle camera is located, the FOV center of the preview image will gradually approach, and even eventually overlap with the FOV center of the wide-angle camera, avoiding FOV center jumps.

[0117] Specifically, Figure 5A , Figure 5B , Figure 5C , Figure 5D The preview images at 1X, 0.9X, 0.8X, and 0.4X are shown, and the FOV centers are at positions O1', O2', O3', and O4', respectively. It can be seen that O3' is closer to O4' than O2', so that when the zoom ratio is reduced from 1X to 0.4X, the FOV center of the preview image will not suddenly change from O1' to O4', but will transition more smoothly without jumping.

[0118] Combine the following Figure 5E-Figure 5H ,illustrate Figure 5A-5D The implementation principle of the zoom reduction process is shown by way of example.

[0119] like Figure 5E-Figure 5H As shown, image M' is the image captured by the ordinary camera, FOV1' is the FOV of the ordinary camera, and FOV2' is the FOV of the wide-angle camera. The FOV of the wide-angle camera covers the FOV of the ordinary camera, and the distance between the FOV center O4' of the wide-angle camera and the FOV center O1' of the ordinary camera is relatively far. This long distance is caused by the fact that the optical centers of the wide-angle camera and the ordinary camera do not overlap. To avoid this, the FOV center jumps during the zoom process, as shown in Figure 5G-Figure 5H As shown, when cropping the image M' captured by the ordinary camera, the electronic device 100 can gradually move the cropping center of the cropping area closer to the FOV center of the wide-angle camera, that is, execute the FOV center offset algorithm to achieve eccentric cropping.

[0120] Specifically, Figure 5E As shown, image 1' is a preview image at 1X. Image 1' can be obtained by cropping image M'. The FOV center of image 1' coincides with the center O1' of FOV1'. Fig. 5F As shown, image 2' is a preview image at 0.9X, and image 2' can be obtained by cropping image M'. The FOV center of image 2' coincides with the center O1' of FOV1'. Figure 5G As shown, image 3' is a preview image at 0.8X, and image 3' can be obtained by cropping image M'. The FOV center of image 3' no longer coincides with the center O1' of FOV1', but deviates from O1' and is closer to the FOV center O4' of the wide-angle camera. Figure 5H As shown, image T' is a preview image at 0.4X, and image T' is an image captured by a wide-angle camera. The FOV center of image T' coincides with the FOV center O4' of the wide-angle camera.

[0121] Figure 5E-Figure 5H Only by taking the example of executing the FOV center offset algorithm when the wide-angle camera starts to run, an implementation method of the FOV center offset algorithm is explained, that is, the electronic device 100 executes the FOV center offset algorithm during the entire digital zoom process from 0.9X to 0.8X, 0.7X, ..., until 0.4X. Not limited to this, the electronic device 100 may also execute the FOV center offset algorithm after a period of time after the wide-angle camera starts to run (for example, after the electronic device 100 zooms to 0.6X). For example, the electronic device 100 may execute the FOV center offset algorithm during the entire digital zoom process from 0.6X to 0.5X, 0.4X. In addition, the embodiment of the present application does not limit the speed of using the FOV center offset algorithm to cut toward the FOV center of the wide-angle camera. For example, the electronic device 100 may move toward the FOV center of the telephoto camera once every 0.1X change in zoom ratio, or once every 0.2X change in zoom ratio.

[0122] In an embodiment of the present application, the FOV of image 1' needs to be smaller than FOV1'. In this way, it can be ensured that in the process of switching from an ordinary camera to a wide-angle camera, a smooth transition can be achieved by cropping the image M' captured by the ordinary camera, avoiding the phenomenon of FOV center jump. The following is a possible implementation method of image 1' being smaller than FOV': Assuming that the zoom ratio corresponding to the FOV of image 1' is A, and the zoom ratio corresponding to FOV' is B, then A=B*C, where C is a constant greater than 1. For example, if B=1X, C=2, then A=2X, that is, when the zoom ratio 307 is displayed as 1X, the preview image actually corresponds to the image at a zoom ratio of 2X. It is easy to understand that by mapping the actual zoom ratio with the zoom ratio displayed by the zoom ratio 307, it can be achieved that the FOV of image 1' is smaller than FOV'.

[0123] Figure 5A-5H The method of reducing the zoom ratio described in the embodiment is also applicable to the scenario where the wide-angle camera switches to the ultra-wide-angle camera, the scenario where the telephoto camera switches to the ordinary camera, and the scenario where the ultra-telephoto camera switches to the telephoto camera, but the embodiment of the present application does not limit this.

[0124] It should be noted that the embodiment of the present application only uses the example of starting to run the wide-angle camera after zooming to 0.9X to illustrate one implementation method of the FOV center offset algorithm. In some embodiments, the wide-angle camera can also be started after zooming to other magnifications (for example, 0.8X, 0.7X, etc.), and the embodiment of the present application is not limited to this.

[0125] Figure 4A-4D , Figure 5A-5D The exemplary zoom preview user interface can also be used for the user to take photos during the zooming process.

[0126] For example, based on Figure 4A-4D The user interface for increasing the zoom ratio in the preview scenario shown, the electronic device 100 can also detect the user's photo operation, such as taking photos at 3X, 3.1X, 3.3X, ..., 5X. In response to the operation, the electronic device 100 can save the images in the preview boxes at these different zoom ratios as photos. Then, the FOV center presented in these photos is smoothly transitioned.

[0127] For example, based on Figure 5A-5D The user interface for reducing the zoom ratio in the preview scenario shown, the electronic device 100 can also detect the user's photo operation, such as taking photos at 0.9X, 0.8X, 0.7X, ..., 0.4X. In response to the operation, the electronic device 100 can save the images in the preview boxes at these different zoom ratios as photos, then, the FOV center presented in these photos is smoothly transitioned.

[0128] Figure 6A-6H The user interface for zooming in a video recording scene is exemplarily shown.

[0129] (1) Figure 6A-6D The user interface for zooming in in a video recording scene is exemplarily shown.

[0130] exist Figure 6A-6D In the embodiment, assuming that the optical zoom magnification of the telephoto camera is 5X, and the optical zoom magnification of the normal camera is 1X, the camera turned on by default in the electronic device 100 is the normal camera. After detecting that the user presses the 5X zoom point on the zoom bar 308, the electronic device 100 turns on the telephoto camera.

[0131] and Figure 4A-4D As in the embodiment shown, Fig. 6A , Figure 6B , Figure 6C , Fig.6D During the video recording process with increased zoom shown, the electronic device 100 can detect the user's operation of increasing the zoom ratio (for example, the user clicks on the 5X zoom point on the zoom bar 308). In response to this operation, during the process of increasing the zoom ratio 307 from 1X to 3X, the electronic device 100 can gradually reduce the FOV presented by the preview image. During the process of increasing the zoom ratio 307 from 3.1X to 5X, in addition to gradually reducing the FOV presented by the preview image, the electronic device 100 can also move the FOV center of the preview image toward the FOV center of the telephoto camera, rather than directly jumping from the FOV center of the ordinary camera to the FOV center of the telephoto camera. As the zoom ratio continues to increase to the optical zoom ratio of 5X where the telephoto camera is located, the FOV center of the preview image will gradually approach, and even eventually overlap with the FOV center of the telephoto camera, avoiding the FOV center jump.

[0132] and Figure 4A-4D The embodiment shown differs in that Figure 6A-6D The image in the preview box, that is, the recorded image, can be saved as a video.

[0133] (2) Figure 6E-6H The user interface for zooming out in a video recording scenario is exemplarily shown.

[0134] exist Figure 6E-6H In the embodiment, assuming that the optical zoom magnification of the ordinary camera is 1X, and the optical zoom magnification of the wide-angle camera is 0.4X (wide), the camera turned on by default by the electronic device 100 is the ordinary camera. After detecting that the user presses the 0.4X zoom point on the zoom bar 308, the electronic device 100 turns on the wide-angle camera.

[0135] and Figure 5A-5D As in the embodiment shown, Fig. 6E , Fig. 6F , Figure 6G , Figure 6H During the video recording process with reduced zoom shown, the electronic device 100 can detect the user's operation of reducing the zoom ratio (for example, the user clicks on the 0.4X zoom point on the zoom bar 308). In response to the operation, in the process of reducing the zoom ratio 307 from 1X to 0.9X, the electronic device 100 gradually expands the FOV presented by the preview image. In the process of reducing the zoom ratio 307 from 0.8X to 0.4X, in addition to gradually expanding the FOV presented by the preview image, the electronic device 100 can also move the FOV center of the preview image toward the FOV center of the wide-angle camera, rather than directly jumping from the FOV center of the ordinary camera to the FOV center of the wide-angle camera. As the zoom ratio continues to decrease to the optical zoom ratio of 0.4X where the wide-angle camera is located, the FOV center of the preview image will gradually approach, and even eventually overlap with the FOV center of the wide-angle camera, avoiding the FOV center jump.

[0136] and Figure 5A-5D The embodiment shown differs in that Figure 6E-6H The image in the preview box, that is, the recorded image, can be saved as a video.

[0137] Based on the aforementioned UI embodiment, a method flow for the electronic device 100 to achieve smooth transition of the FOV center in a point-to-zoom scenario is introduced below.

[0138] In the embodiment of the present application, "click zoom" refers to the process in which the user switches the current zoom point to the target zoom point by clicking on a zoom point other than the current zoom point (also referred to as the first zoom ratio option) on the zoom bar 308, for example, Figure 4A In the example, the user switches the current zoom point 1X zoom point to the target zoom point 5X zoom point by clicking the 5X zoom point on the zoom bar 308. For example, Figure 5A The user switches the current zoom point 1X zoom point to the target zoom point 0.4X zoom point by clicking the 0.4X zoom point on the zoom bar 308.

[0139] Figure 7 The following is an example of a method flow for the electronic device 100 to achieve smooth transition of the FOV center in a point-to-zoom scenario. The following takes the current zoom point as a 1X zoom point, the target zoom point as a 5X zoom point, and the user zooming from a 1X zoom point to a 5X zoom point as an example to describe the specific steps of the method in detail:

[0140] S701: The electronic device 100 detects that a user presses a display screen with a finger.

[0141] S702: The electronic device 100 determines whether the position where the user's finger presses the display screen is at a target zoom point, where the target zoom point is different from the current zoom point.

[0142] Specifically, after detecting the operation of the user's finger pressing the display screen, the electronic device 100 can determine whether the position where the user's finger presses the display screen is within the hot zone of the target zoom point. If so, it can be indicated that the position where the user's finger presses the display screen is on the target zoom point. The hot zone of the target zoom point refers to a certain area on the display screen that includes the position of the target zoom point.

[0143] S703. The camera application of the electronic device 100 sends the zoom value corresponding to the target zoom point and the zoom value corresponding to the zoom sequence to the hardware abstraction layer (HAL).

[0144] The above zoom sequence refers to a set of multiple zoom ratios between a target zoom point (eg, a 5X zoom point) and a current zoom point (eg, a 1X zoom point).

[0145] Exemplarily, if the electronic device 100 determines that the position where the user's finger presses the display screen is within the hot zone of the 5X zoom point, it means that the position where the user's finger presses the display screen is on the 5X zoom point. Afterwards, the camera application of the electronic device 100 can send the zoom value corresponding to the 5X zoom point and the zoom value corresponding to the zoom sequence to the hardware abstraction layer. In this way, it can be avoided that the hardware abstraction layer does not have enough time to execute subsequent related steps due to a certain delay in the camera application sending the zoom value corresponding to the 5X zoom point to the hardware abstraction layer. Among them, the zoom value corresponding to the above-mentioned 5X zoom point is the zoom value corresponding to the zoom ratio of 5X, and the zoom value corresponding to the above-mentioned zoom sequence is the zoom value corresponding to each zoom ratio in the zoom sequence.

[0146] The calculation formula of the zoom value zoomValue corresponding to the zoom sequence is:

[0147]

[0148] Wherein, S is the distance between the current zoom point and the target zoom point on the zoom bar on the display screen; T is the time required for a smooth transition from the current zoom point to the target zoom point (i.e., the time required from the user clicking the current zoom point to the display screen displaying the preview image corresponding to the target zoom point), which can be obtained from the hardware abstraction layer; t is the time interval between two adjacent preview frames of the current camera system, which can be obtained from the hardware abstraction layer; n is the sequence number of the zoom ratio in the zoom sequence, n is a positive integer, and the value range is (0, N], N = T / t, that is, N is the total number of frames that need to be inserted within the T time.

[0149] It is easy to understand that the number of zoom ratios included in the above zoom sequence is determined by N. For example, the current zoom point is 1X, the target zoom point is 5X, and the minimum change unit of the zoom ratio is 0.1X. In a possible implementation, the zoom sequence may be [1.1X 1.2X1.3X…4.9X 5X], the value range of the sequence number n of the zoom sequence is (0,40], n=1 corresponds to a zoom magnification of 1.1X, n=2 corresponds to a zoom magnification of 1.2X, and so on. When n=40, the corresponding zoom magnification is 5X. In another possible implementation, the zoom sequence may also be [1.2X 1.4X…4.8X 5X], the value range of the sequence number n of the zoom sequence is (0,20], n=1 corresponds to a zoom magnification of 1.2X, n=2 corresponds to a zoom magnification of 1.4X, and so on. When n=20, the corresponding zoom magnification is 5X.

[0150] It is easy to understand that since the time interval for the camera application to send the zoom value corresponding to each zoom ratio in the zoom sequence to the hardware abstraction layer is determined by the preview frame time interval of the current camera system, the time required for the camera application to send the zoom values ​​corresponding to the above zoom sequence one by one in sequence is the time T required for the smooth transition from the current zoom point to the target zoom point.

[0151] In one possible implementation, the camera application may send all zoom values ​​corresponding to the above zoom sequence to the hardware abstraction layer at one time, and the hardware abstraction layer may match the image preview frames corresponding to each zoom value one by one according to the preview frame time interval of the current camera system, thereby completing a smooth zoom process.

[0152] S704: The hardware abstraction layer of the electronic device 100 starts executing the first processing flow.

[0153] The hardware abstraction layer of the electronic device 100 can start executing the first processing flow based on the zoom value corresponding to the zoom point and the zoom value corresponding to the zoom sequence sent by the camera application. The specific process of executing the first processing flow will be described in detail later and will not be expanded here.

[0154] S705: The electronic device 100 detects that the user's finger leaves the display screen.

[0155] S706: The electronic device 100 determines whether the position where the user's finger leaves the display screen is at the target zoom point.

[0156] Specifically, after detecting that the user's finger leaves the display screen, the electronic device 100 can determine whether the position where the user's finger leaves the display screen is within the hot zone of the above-mentioned focus point. If so, it can be indicated that the position where the user's finger leaves the display screen is on the above-mentioned focus point. The hot zone of the above-mentioned focus point refers to a certain area on the display screen that includes the position of the above-mentioned focus point.

[0157] Typically, during the point-to-zoom process, the time from when the user's finger presses the display screen to when it leaves the display screen is approximately 100ms-300ms, which does not exceed the total time of the smooth zoom process.

[0158] In some embodiments, the user may also use a writing device (such as a stylus) to press or release the display screen.

[0159] S707: The camera application of the electronic device 100 sends a confirmation zoom instruction to the hardware abstraction layer.

[0160] For example, if the electronic device 100 determines that the position where the user's finger leaves the display screen is within the hot zone of the 5X zoom point, it means that the position where the user's finger leaves the display screen is at the 5X zoom point. Afterwards, the camera application of the electronic device 100 can send the above-mentioned confirmation zoom instruction to the hardware abstraction layer, wherein the above-mentioned confirmation zoom instruction is used to instruct the hardware abstraction layer to execute the second processing flow.

[0161] S708: The hardware abstraction layer of the electronic device 100 starts executing the second processing flow.

[0162] After receiving the above-mentioned confirmation zoom instruction, the hardware abstraction layer of the electronic device 100 can execute the second processing flow based on the zoom value corresponding to the above-mentioned zoom point and the zoom value corresponding to the zoom sequence sent by the camera application. The specific process of executing the second processing flow will be described in detail later and will not be expanded here.

[0163] S709: The display screen of the electronic device 100 displays the zoomed image.

[0164] Specifically, after the hardware abstraction layer of the electronic device 100 executes the first processing flow and the second processing flow, the display screen of the electronic device 100 can display multiple zoomed images. For example, when the zoom sequence is [1.1X 1.2X 1.3X ... 4.9X 5X], the display screen can sequentially display images at 1.1X, 1.2X, 1.3X, ..., 4.9X, and 5X.

[0165] S710: The camera application of the electronic device 100 sends a cancel zoom instruction to the hardware abstraction layer.

[0166] For example, if the electronic device 100 determines that the position where the user's finger leaves the display screen is not within the hot zone of the 5X zoom point, it means that the position where the user's finger leaves the display screen is not at the 5X zoom point. Afterwards, the camera application of the electronic device 100 can send the above-mentioned cancel zoom instruction to the hardware abstraction layer, wherein the above-mentioned cancel zoom instruction is used to instruct the hardware abstraction layer to stop executing the first processing flow and the second processing flow.

[0167] S711. The hardware abstraction layer of the electronic device 100 stops executing the first processing flow and the second processing flow.

[0168] After receiving the above-mentioned cancel zoom instruction, the hardware abstraction layer of the electronic device 100 can stop executing the first processing flow and the second processing flow. The specific process of executing the first processing flow and the second processing flow will be described in detail later and will not be expanded here.

[0169] It should be noted that the method for the electronic device 100 to achieve a smooth transition of the FOV center in the above-mentioned point zoom scenario is only illustrated by taking the switching of an ordinary camera to a telephoto camera as an example. The above-mentioned method is also applicable to switching between other cameras, for example, switching an ordinary camera to a wide-angle camera. The embodiments of the present application are not limited to this.

[0170] The specific process of the hardware abstraction layer of the electronic device 100 executing the first processing flow and the second processing flow in the point-to-zoom scenario is described in detail below.

[0171] In the embodiment of the present application, the first processing flow and the second processing flow can be collectively referred to as a smooth zoom processing process, and the specific process is as follows:

[0172] (1) Increase the zoom ratio

[0173] For example, the current zoom point is 1X zoom point, the target zoom point is 5X zoom point, and the user clicks to zoom from 1X zoom point to 5X zoom point (that is, the user clicks to switch from a normal camera to a telephoto camera).

[0174] As shown in Table 1, when the current zoom point is 1X zoom point, only the ordinary camera is running. At this time, the image displayed on the display screen (for example Figure 4A The preview image shown) is from a regular camera.

[0175] When the electronic device 100 detects that the user presses the 5X zoom point, the camera application can send the zoom value corresponding to the 5X zoom point to the hardware abstraction layer. After receiving the zoom value corresponding to the 5X zoom point, the hardware abstraction layer can confirm that the camera ID (Identifier) ​​corresponding to the zoom value is a telephoto camera based on the zoom value, and start the telephoto camera. The ordinary camera is still running. At this time, the image displayed on the display screen (for example Figure 4A The preview image shown) still comes from the normal camera.

[0176] When the camera application sends the zoom value corresponding to the 5X zoom point to the hardware abstraction layer, it also sends the zoom value corresponding to each zoom ratio in the zoom sequence to the hardware abstraction layer in sequence. For example, when the zoom sequence is [1.1X 1.2X1.3X…4.9X 5X], during the process of sending the zoom value from 1.1X to 3X, the ordinary camera is still running, while the telephoto camera is in the startup process and has not yet started running. This process can center-crop the image captured by the ordinary camera. At this time, the image displayed on the display (for example Figure 4B The preview image shown) still comes from the normal camera.

[0177] When the camera application sends the zoom value from 3.1X to 4.9X, the normal camera is still running and the telephoto camera starts running, that is, the telephoto camera also starts to collect images. Based on the images collected by the normal camera and the telephoto camera, the hardware abstraction layer starts to execute the FOV center offset algorithm to gradually move the FOV center of the image collected by the normal camera toward the FOV center of the telephoto camera. During this process, the image displayed on the display (for example Figure 4C The preview image shown) still comes from the normal camera.

[0178] When the camera application sends a 5X zoom value to the hardware abstraction layer, the normal camera stops running (i.e. shuts down), the telephoto camera is still running, and the image displayed on the display (e.g. Figure 4D The preview image shown in FIG. 1 is from the telephoto camera. At this point, the camera application has sent all the zoom values ​​corresponding to each zoom ratio in the above zoom sequence.

[0179] from Figure 4A-4HIt can be seen that the electronic device 100 can implement smooth zoom by executing the FOV center offset algorithm and cropping the image captured by the ordinary camera so that the FOV center of the image displayed on the display screen smoothly transitions from the FOV center of the ordinary camera to the FOV center of the telephoto camera.

[0180]

[0181] Table 1

[0182] (2) Reduce the zoom ratio

[0183] For example, the current zoom point is 1X zoom point, the target zoom point is 0.4X zoom point, and the user clicks to zoom from 1X zoom point to 0.4X zoom point (that is, the user clicks to switch from a normal camera to a wide-angle camera).

[0184] As shown in Table 2, when the current zoom point is 1X zoom point, only the ordinary camera is running. At this time, the image displayed on the display screen (for example Figure 5A The preview image shown) is from a regular camera.

[0185] When the electronic device 100 detects that the user presses the 0.4X zoom point, the camera application can send the zoom value corresponding to the 0.4X zoom point to the hardware abstraction layer. After receiving the zoom value corresponding to the 0.4X zoom point, the hardware abstraction layer can confirm that the camera ID (Identifier) ​​corresponding to the zoom value is a wide-angle camera based on the zoom value, and start the wide-angle camera. The ordinary camera is still running. At this time, the image displayed on the display screen (for example Figure 5A The preview image shown) still comes from the normal camera.

[0186] When the camera application sends the zoom value corresponding to the 0.4X zoom point to the hardware abstraction layer, it also sends the zoom value corresponding to each zoom ratio in the zoom sequence to the hardware abstraction layer in sequence. For example, if the zoom sequence is [0.9X 0.8X…0.5X 0.4X], when the zoom value of 0.9X is sent, the ordinary camera is still running, while the wide-angle camera is in the startup process and has not yet started running. This process can center-crop the image captured by the ordinary camera. At this time, the image displayed on the display (for example Figure 5B The preview image shown) still comes from the normal camera.

[0187] When the camera application sends the zoom value from 0.8X to 0.5X, the ordinary camera is still running and the wide-angle camera starts running, that is, the wide-angle camera also starts to collect images. Based on the images collected by the ordinary camera and the wide-angle camera, the hardware abstraction layer starts to execute the FOV center offset algorithm to gradually move the FOV center of the image collected by the ordinary camera toward the FOV center of the wide-angle camera. During this process, the image displayed on the display (for example Figure 5C The preview image shown) still comes from the normal camera.

[0188] When the camera application sends a zoom value of 0.5X to the hardware abstraction layer, the normal camera stops running (i.e., turns off), the wide-angle camera is still running, and the image displayed on the display (e.g. Figure 5D The preview image shown in FIG. 1 is from the wide-angle camera. At this point, the camera application has sent the zoom values ​​corresponding to all zoom magnifications in the above zoom sequence.

[0189] from Figure 5A-5H It can be seen that the electronic device 100 can implement smooth zoom by executing the FOV center offset algorithm and cropping the image captured by the ordinary camera so that the FOV center of the image displayed on the display screen smoothly transitions from the FOV center of the ordinary camera to the FOV center of the wide-angle camera.

[0190]

[0191] Table 2

[0192] The following describes the specific implementation process of the FOV center offset algorithm. Figure 8 The specific execution process of the FOV center offset algorithm is exemplarily shown, wherein the target camera is the camera to be switched from the current camera, and the process includes the following steps:

[0193] S801. The hardware abstraction layer of the electronic device 100 adjusts the cropping area of ​​the current camera (including the position and size of the cropping area) based on the feature point coordinates of the target camera (for example, the FOV center point coordinates of the target camera), and calculates the feature point coordinates of the current camera (for example, the FOV center point coordinates of the current camera cropping area).

[0194] (1) Increase the zoom ratio

[0195] Taking the current camera as a normal camera and the target camera as a telephoto camera as an example, when both the normal camera and the telephoto camera are in operation, the hardware abstraction layer of the electronic device 100 can be based on the feature point coordinates of the telephoto camera (for example Figure 4G The FOV center point coordinates O4 of the telephoto camera shown in FIG. 4 are used to adjust the cropping area of ​​the ordinary camera (for example, the cropping area is Figure 4F The area where image 2 is located is adjusted to Figure 4G The area where the image 3 is located in the image 3 is located), and the coordinates of the feature points of the ordinary camera are calculated based on the above-mentioned cropped area (for example Figure 4G The FOV center point coordinates of the ordinary camera shown are O3).

[0196] (2) Reduce the zoom ratio

[0197] Taking the current camera as a normal camera and the target camera as a wide-angle camera as an example, when both the normal camera and the wide-angle camera are in operation, the hardware abstraction layer of the electronic device 100 can be based on the feature point coordinates of the wide-angle camera (for example Figure 5G The FOV center point coordinates O4' of the wide-angle camera shown in FIG. 1 are used to adjust the cropping area of ​​the ordinary camera (for example, the cropping area is Fig. 5F The area where image 2' is located is adjusted to Figure 5G The area where the image 3' is located in the cut area is calculated based on the above cut area. Figure 5G The coordinates of the FOV center point of the ordinary camera shown are O3').

[0198] S802: The hardware abstraction layer of the electronic device 100 determines whether the coordinates of the feature points of the target camera coincide with the coordinates of the feature points of the current camera.

[0199] Specifically, after executing step S801, the hardware abstraction layer of the electronic device 100 can determine whether the feature point coordinates of the target camera coincide with the feature point coordinates of the current camera calculated based on the cropping area of ​​the current camera. If so, the FOV center offset algorithm is terminated; if not, steps S801 and S802 are continued to be executed in sequence until the feature point coordinates of the target camera coincide with the feature point coordinates of the current camera. In other words, the feature point coordinates of the current camera can be gradually approached and coincided with the feature point coordinates of the target camera by adjusting the cropping area of ​​the current camera multiple times and calculating the feature point coordinates of the current camera multiple times, so that the feature point coordinates of the current camera gradually approach and coincide with the feature point coordinates of the target camera, thereby achieving a smooth transition of the FOV center of the image displayed on the display screen from the FOV center of the ordinary camera to the FOV center of the wide-angle camera.

[0200] (1) Increase the zoom ratio

[0201] Taking the current camera as an ordinary camera and the target camera as a telephoto camera as an example, assuming that the zoom ratio presents a trend of gradually increasing from 3.1X, 3.2X, 3.3X, ..., 4.9X, 5X, the hardware abstraction layer of the electronic device 100 can adjust the cropping area of ​​the current camera in sequence under the zoom ratio of 3.1X, 3.2X, 3.3X, ..., 4.9X, 5X, and calculate the feature point coordinates of the current camera in sequence, thereby presenting a trend of the FOV center point coordinates of the ordinary camera gradually moving toward the FOV center point coordinates of the telephoto camera. In this way, during the zooming process, the FOV center of the preview image will not jump.

[0202] (2) Reduce the zoom ratio

[0203] Taking the current camera as an ordinary camera and the target camera as a wide-angle camera as an example, assuming that the zoom ratio shows a trend of gradually decreasing from 0.8X, 0.7X, 0.6X, 0.5X, and 0.4X, the hardware abstraction layer of the electronic device 100 can adjust the cropping area of ​​the current camera in sequence at the zoom ratios of 0.8X, 0.7X, 0.6X, 0.5X, and 0.4X, and calculate the feature point coordinates of the current camera in sequence, thereby showing a trend of the FOV center point coordinates of the ordinary camera gradually moving toward the FOV center point coordinates of the wide-angle camera. In this way, during the zooming process, the FOV center of the preview image will not jump.

[0204] It should be noted that Figure 8 The specific execution process of the FOV center offset algorithm is described in detail by taking the switching of an ordinary camera to a telephoto camera and the switching of an ordinary camera to a wide-angle camera as examples. The FOV algorithm is also applicable to switching between other cameras, and the embodiments of the present application do not limit this.

[0205] In the aforementioned embodiment, the electronic device 100 turns on only one camera by default, and turns on another camera only when a scene is selected for zooming, so as to achieve a smooth transition of the FOV center.

[0206] In the embodiment of the present application, the electronic device 100 may also open the current camera and the cameras with adjacent focal lengths (or also called adjacent optical zoom ratios) by default. In multiple scenarios such as point-to-zoom, pinch-to-zoom, and drag-to-zoom, only the camera corresponding to the target zoom point and the cameras with adjacent focal lengths are opened, which can also achieve a smooth transition of the FOV center and save the time of opening the camera during the zoom process. The following is a detailed introduction:

[0207] Assume that the electronic device 100 has four cameras: a wide-angle camera, a normal camera, a telephoto camera 1, and a telephoto camera 2. Figure 9A-9DThe above four cameras are shown as being turned on and off, wherein the camera represented by the black solid circle is the camera from which the preview image displayed on the display screen of the electronic device 100 comes.

[0208] Assume that the optical zoom ratio of the wide-angle camera is 0.4X, the optical zoom ratio of the normal camera is 1X, the optical zoom ratio of the telephoto camera 1 is 5X, and the optical zoom ratio of the telephoto camera 2 is 6X. Figure 9A-9D It is easy to see that the optical zoom ratios of the above four cameras increase in sequence, that is, the optical zoom ratio of the wide-angle camera < the optical zoom ratio of the ordinary camera < the optical zoom ratio of the telephoto camera 1 < the optical zoom ratio of the telephoto camera 1. It is easy to understand that from the perspective of the optical zoom ratio of each camera, the wide-angle camera is only adjacent to the ordinary camera, the ordinary camera is adjacent to the wide-angle camera and telephoto camera 1, the telephoto camera 1 is adjacent to the ordinary camera and telephoto camera 2, and the telephoto camera 2 is only adjacent to telephoto camera 2.

[0209] Assume that the electronic device 100 turns on the wide-angle camera by default. Fig.9A As shown, after the electronic device 100 detects the user's operation of opening the camera application, in response to the operation, the electronic device 100 opens the camera application, and the preview image comes from the image captured by the wide-angle camera. At the same time, since the wide-angle camera is adjacent to the ordinary camera, the electronic device 100 starts to start the ordinary camera, so that the ordinary camera enters the running state. In the case where the ordinary camera has entered the running state, if the electronic device 100 detects the user's operation of switching from the wide-angle camera to the ordinary camera, in response to the operation, the electronic device 100 can directly execute the FOV center offset algorithm based on the image captured by the ordinary camera and the image captured by the wide-angle camera, so that the FOV center of the wide-angle camera smoothly transitions to the FOV center of the ordinary camera, avoiding the FOV center from jumping, and achieving smooth zooming.

[0210] like Fig. 9BAs shown, after the above-mentioned smooth zoom process is finished, the preview image comes from the image captured by the ordinary camera. At this time, since the ordinary camera is adjacent to the wide-angle camera and the telephoto camera 1, the electronic device 100 still keeps the wide-angle camera running, and starts the telephoto camera 1 at the same time, so that the telephoto camera 1 enters the running state. When the telephoto camera 1 has entered the running state, if the electronic device 100 detects the user's operation of switching from the ordinary camera to the telephoto camera 1, in response to the operation, the electronic device 100 can directly execute the FOV center offset algorithm based on the image captured by the ordinary camera and the image captured by the telephoto camera 1, so that the FOV center of the ordinary camera smoothly transitions to the FOV center of the telephoto camera 1, avoiding the FOV center from jumping, and realizing smooth zoom.

[0211] like Fig. 9C As shown, after the above-mentioned smooth zoom process is finished, the preview image comes from the image captured by the telephoto camera 1. At this time, since the telephoto camera 1 is adjacent to the ordinary camera and the telephoto camera 2, but not adjacent to the wide-angle camera, the electronic device 100 starts to turn off the wide-angle camera, still keeps the ordinary camera running, and starts to start the telephoto camera 2 at the same time, so that the telephoto camera 2 enters the running state. When the telephoto camera 2 has entered the running state, if the electronic device 100 detects the user's operation of switching from the telephoto camera 1 to the telephoto camera 2, in response to the operation, the electronic device 100 can directly execute the FOV center offset algorithm based on the image captured by the telephoto camera 1 and the image captured by the telephoto camera 2, so that the FOV center of the telephoto camera 1 smoothly transitions to the FOV center of the telephoto camera 2, avoiding the FOV center from jumping, and achieving smooth zoom.

[0212] like Fig.9D As shown, after the above-mentioned smooth zoom process is completed, the preview image comes from the image captured by the telephoto camera 2. At this time, since the telephoto camera 2 is adjacent to the telephoto camera 1 and not adjacent to the ordinary camera, the electronic device 100 starts to turn off the ordinary camera and keeps the telephoto camera 1 running.

[0213] It can be seen that the difference from only turning on one camera by default in the click-to-zoom scenario is that the embodiment of the present application can directly execute the FOV center offset algorithm in the scenario where it is detected that the user needs to zoom (for example, click-to-zoom, two-finger pinch-to-zoom, drag-to-zoom bar, and other scenarios), without having to wait until the camera corresponding to the target zoom point is running before executing the FOV center offset algorithm. In this way, while achieving smooth zooming, the time of turning on the camera during the zooming process is saved.

[0214] It should be noted that the above text is only based on the example that the electronic device 100 turns on the wide-angle camera by default. Figure 9A-9DFor example, the electronic device 100 may also turn on a normal camera by default. When it is detected that the user switches from the normal camera to the wide-angle camera, the camera can be turned on and off by the user. Fig. 9B Switch to Fig.9A When it is detected that the user switches from the normal camera to the telephoto camera 1, the camera opening and closing status can be determined by Fig. 9B Switch to Fig. 9C .

[0215] It is easy to understand that the embodiments of the present application are also applicable to the scenario where the user directly clicks the zoom button from the current camera to switch to a camera that is not adjacent to the current camera. For example, in the case of increasing the zoom ratio, if the electronic device 100 detects that the user directly clicks the zoom button from the wide-angle camera to the telephoto camera 2, in response to the operation, the electronic device 100 can turn on or off the above-mentioned zoom buttons in sequence. Fig.9A , Fig. 9B , Fig. 9C , Fig.9D For another example, in the case of reducing the zoom ratio, if the electronic device 100 detects that the user directly switches the zoom from the telephoto camera 2 to the wide-angle camera, in response to the operation, the electronic device 100 can turn on or off the above-mentioned Fig.9D , Fig. 9C , Fig. 9B , Fig.9A The corresponding camera is located in the middle and the FOV center offset algorithm in the smooth zoom process is executed in sequence.

[0216] Next, from the perspective of the software and hardware collaboration of the electronic device 100, Fig.10 and Fig.11 Describes in detail the specific implementation of the FOV center smooth transition process.

[0217] like Fig.10 and Fig.11 As shown, the camera is used to capture images. When the reflected light of the scene passes through the lens, it is refracted on the lens and converges on the image sensor. The image sensor converts the optical image into an analog electrical signal, which is then output through a digital-to-analog converter as the original digital image captured by the camera.

[0218] The hardware abstraction layer is used to receive the zoom value corresponding to the zoom ratio reported from the camera application, start the target camera corresponding to the zoom ratio (such as a telephoto camera or a wide-angle camera), start the FOV center offset algorithm, etc. Specifically, after receiving the zoom value corresponding to the zoom ratio reported from the camera application, the hardware abstraction layer can determine and start the target camera corresponding to the above zoom value. Then, the hardware abstraction layer can execute the FOV center offset algorithm based on the image captured by the current camera (such as a normal camera) and the image captured by the target camera (such as a telephoto camera or a wide-angle camera), obtain the cropping parameters, and send the cropping parameters to the cropping module of the ISP. Among them, the above cropping parameters may include the position (such as the coordinates of the FOV center point) and size (width and height) of the cropping area.

[0219] ISP is used to convert the data from the camera into images in standard formats, such as YUV, RGB, etc. Specifically, ISP can use the cropping module to crop the image captured by the camera based on the received cropping parameters, and then perform post-processing on the cropped image, such as black level correction, lens shading correction, bad pixel compensation, color interpolation, etc., and then send the YUV / RGB image to the processor for processing through the I / O control interface.

[0220] The processor includes application processor, baseband processor, multimedia processor, etc., which can run various image processing algorithms and control peripheral devices. In the preview scene, the image in YUV, RGB and other formats obtained by the ISP can be directly sent to the display screen for display. In the photo or video scene, the photo or video can be saved to the memory.

[0221] The display screen can monitor user operations for adjusting the preview images in various areas of the display screen through the UI, and report the monitored user operations to the hardware abstraction layer. The user operations may include but are not limited to the touch operations on the target object detected by the electronic device 100 in the preview frame mentioned in the above UI embodiment.

[0222] like Fig.10 As shown, the specific implementation of increasing the zoom ratio is introduced below (taking the switching of a normal camera to a telephoto camera as an example).

[0223] In the preview scene, the display screen displays a preview interface. The preview box in the preview interface is used to display the image captured by the ordinary camera. The zoom ratio displayed in the preview interface is 1X.

[0224] In the case where only the ordinary camera is turned on by default, the display screen detects the user's operation of increasing the zoom ratio to 5X (for example, the user clicks on the zoom point 5X), and sends the zoom value corresponding to the zoom ratio selected by the user to the hardware abstraction layer. The hardware abstraction layer can determine that the camera corresponding to the zoom point 5X is a telephoto camera, and then the hardware abstraction layer can start the telephoto camera. During the startup process of the telephoto camera, that is, before the telephoto camera starts running, the ISP can center the image captured by the ordinary camera based on the cropping parameters calculated by the hardware abstraction layer. After the telephoto camera starts running, the hardware abstraction layer can execute the FOV center offset algorithm based on the image captured by the ordinary camera and the image captured by the telephoto camera to obtain the cropping parameters, and send the above-mentioned cropping parameters to the cropping module of the ISP. The cropping module of the ISP can eccentrically crop the image captured by the ordinary camera based on the above-mentioned cropping parameters. Afterwards, the ISP can perform further post-processing on the cropped image and send it to the processor to generate the image to be displayed. Afterwards, the processor can send the above-mentioned image to be displayed to the display to instruct the display screen to display the above-mentioned image to be displayed in the preview box.

[0225] When the normal camera is turned on by default, and the cameras adjacent to the normal camera (such as the telephoto camera and the wide-angle camera) are turned on, the display screen detects the user's operation of increasing the zoom ratio to 5X (such as the user clicking on the zoom point 5X), and sends the zoom value corresponding to the zoom ratio selected by the user to the hardware abstraction layer. The hardware abstraction layer can determine that the camera corresponding to the zoom point 5X is a telephoto camera. Afterwards, since the telephoto camera has been turned on and running in advance, the hardware abstraction layer can directly execute the FOV center offset algorithm based on the images captured by the normal camera and the images captured by the telephoto camera to obtain the cropping parameters, and send the above cropping parameters to the cropping module of the ISP. The cropping module of the ISP can perform eccentric cropping on the image captured by the normal camera based on the above cropping parameters. Afterwards, the ISP can perform further post-processing on the cropped image and send it to the processor to generate the image to be displayed. Afterwards, the processor can send the above image to be displayed to the display to instruct the display screen to display the above image to be displayed in the preview box.

[0226] It is easy to understand that in the process of increasing the zoom ratio, the electronic device 100 can crop the image captured by the ordinary camera through the collaboration of the hardware abstraction layer and the ISP, so as to achieve the purpose of gradually reducing the preview viewing angle and gradually changing the FOV center of the displayed image.

[0227] like Fig.11 As shown, the specific implementation of reducing the zoom ratio is introduced below (taking the switching of a normal camera to a wide-angle camera as an example).

[0228] In the preview scene, the display screen displays a preview interface. The preview box in the preview interface is used to display the image captured by the ordinary camera. The zoom ratio displayed in the preview interface is 1X.

[0229] In the case where only the ordinary camera is turned on by default, the display screen detects the user's operation of reducing the zoom ratio to 0.4X (for example, the user clicks on the zoom point 0.4X), and sends the zoom value corresponding to the zoom ratio selected by the user to the hardware abstraction layer. The hardware abstraction layer can determine that the camera corresponding to the zoom point 0.4X is a wide-angle camera. After that, the hardware abstraction layer can start the wide-angle camera. During the wide-angle camera startup process, that is, before the wide-angle camera starts running, the ISP can center the image captured by the ordinary camera based on the cropping parameters calculated by the hardware abstraction layer. After the wide-angle camera starts running, the hardware abstraction layer can execute the FOV center offset algorithm based on the image captured by the ordinary camera and the image captured by the wide-angle camera to obtain the cropping parameters, and send the above-mentioned cropping parameters to the cropping module of the ISP. The cropping module of the ISP can eccentrically crop the image captured by the ordinary camera based on the above-mentioned cropping parameters. After that, the ISP can perform further post-processing on the cropped image and send it to the processor to generate the image to be displayed. After that, the processor can send the above-mentioned image to be displayed to the display to instruct the display screen to display the above-mentioned image to be displayed in the preview box.

[0230] When the normal camera is turned on by default, and the cameras adjacent to the normal camera (such as the telephoto camera and the wide-angle camera) are turned on, the display screen detects the user's operation of reducing the zoom ratio to 0.4X (such as the user clicking on the zoom point 0.4X), and sends the zoom value corresponding to the zoom ratio selected by the user to the hardware abstraction layer. The hardware abstraction layer can determine that the camera corresponding to the zoom point 0.4X is a wide-angle camera. Afterwards, since the wide-angle camera has been turned on and running in advance, the hardware abstraction layer can directly perform the FOV center offset algorithm based on the image captured by the normal camera and the image captured by the wide-angle camera to obtain the cropping parameters, and send the above cropping parameters to the cropping module of the ISP. The cropping module of the ISP can perform eccentric cropping on the image captured by the normal camera based on the above cropping parameters. Afterwards, the ISP can perform further post-processing on the cropped image and send it to the processor to generate the image to be displayed. Afterwards, the processor can send the above image to be displayed to the display to instruct the display screen to display the above image to be displayed in the preview box.

[0231] It is easy to understand that in the process of reducing the zoom ratio, the electronic device 100 can crop the image captured by the ordinary camera through the collaboration of the hardware abstraction layer and the ISP, so as to achieve the purpose of gradually increasing the preview viewing angle and gradually changing the FOV center of the displayed image.

[0232] In the photo shooting scene, unlike the above two preview scenes, the display screen can also detect the user's photo shooting operation. In response to the operation, the electronic device 100 can save the image displayed in the preview box as a photo. Specifically, after entering the shooting preview interface, the display screen can detect not only the user's operation of increasing / decreasing the zoom ratio, but also the user's touch operation on the shooting control. In response to the operation, the electronic device 100 can save the image in the preview box sent to the display screen by the ISP when the above touch operation is detected, that is, the ISP can further encode and compress the obtained YUV format data into a JPEG format photo, and the processor then saves the above photo to the memory.

[0233] In the recording scene, unlike the above two preview scenes, the electronic device 100 can further save the image in the preview frame, specifically as a video file. Specifically, after entering the shooting preview interface, the display screen can detect not only the user's operation of increasing / decreasing the zoom ratio, but also the user's two touch operations on the recording control. In response to the above two touch operations, the electronic device 100 can save the image frame output in the time period between the above two touch operations, that is, the video generated in the recording mode, and the processor then saves the generated video in the memory.

[0234] Based on the electronic device 100 and the aforementioned UI embodiment introduced in the foregoing content, the overall process of the shooting method provided in the embodiment of the present application is introduced below.

[0235] like Fig.12 As shown, the shooting method can be applied to an electronic device 100 including a display screen and multiple cameras, where the multiple cameras may include a first camera and a second camera. The specific steps of executing the shooting method are as follows:

[0236] S1201: The electronic device 100 displays a first preview image at a first zoom ratio on a display screen, where the first preview image is captured by a first camera.

[0237] Specifically, the electronic device 100 may detect the user's operation of opening the "Camera" application, for example, Figure 3A The operation of clicking the camera icon 215D in the main interface shown in FIG. In response to this operation, the electronic device 100 can open the "Camera" application and display a first preview image at a first zoom factor on the display screen, the first preview image being captured by the first camera.

[0238] The first camera may be a common camera, the first zoom ratio may be an optical zoom ratio of a common camera or a zoom ratio within the range of a digital zoom ratio of a common camera, and the first preview image may be Figure 4A or Figure 5A The image in the preview box is shown.

[0239] S1202: The electronic device 100 detects a click operation of the user on a second zoom ratio option.

[0240] The second zoom ratio option may also be referred to as a target zoom point, such as a target zoom point of 5X. The click operation may include: the user's finger or handwriting device first touches the second zoom ratio option, and then leaves the second zoom ratio option.

[0241] S1203: The electronic device 100 generates and displays on the display screen at least one second preview image based on at least one third zoom magnification, where the at least one third zoom magnification is between the first zoom magnification and the second zoom magnification.

[0242] The third zoom ratio may be one or more zoom ratios in the zoom sequence mentioned in the above embodiment, and the second preview image may be a preview image at the third zoom ratio during the smooth zoom process. The second zoom ratio may be an optical zoom ratio of the second camera. The second camera may be a telephoto camera or a wide-angle camera.

[0243] For example, in the case of increasing the zoom ratio, the second zoom ratio may be the optical zoom ratio of the telephoto camera, the third zoom ratio may be 3X, and the corresponding second preview image may be Figure 4B Alternatively, the third zoom ratio may also be 3.1X, and the corresponding second preview image may be Figure 4C The image in the preview box is shown.

[0244] For example, in the case of reducing the zoom ratio, the second zoom ratio may be the optical zoom ratio of the wide-angle camera, the third zoom ratio may be 0.9X, and the corresponding second preview image may be Figure 5B Alternatively, the third zoom ratio may also be 0.8X, and the corresponding second preview image may be Figure 5C The image in the preview box is shown.

[0245] S1204: The electronic device 100 displays a third preview image at the second zoom factor on the display screen, where the third preview image is captured by the second camera.

[0246] For example, when the zoom ratio is increased, the second camera may be a telephoto camera, and the third preview image may be Figure 4D The image in the preview box is shown.

[0247] For example, when the zoom ratio is reduced, the second camera may be a wide-angle camera, and the third preview image may be Figure 5D The image in the preview box is shown.

[0248] For the contents not mentioned in the above overall method flow, reference can be made to the relevant contents in the aforementioned embodiments, which will not be repeated here.

[0249] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A shooting method, applied to an electronic device including a display screen, a first camera and a second camera, characterized in that: The method comprises: The electronic device displays a preview image at a first zoom ratio on the display screen, wherein the preview image at the first zoom ratio is acquired based on the first camera; the FOV center of the preview image at the first zoom ratio coincides with the FOV center of the first camera; The electronic device detects that a user triggers an operation of increasing the zoom ratio of the electronic device; After detecting the operation, the electronic device starts to activate the second camera; In the process of starting the second camera to the running state, the electronic device gradually changes the field of view FOV size of the preview image; in the process, the FOV center of the preview image coincides with the FOV center of the preview image under the first zoom factor; the preview image in the process is obtained by cropping the image captured by the first camera; After the second camera enters the operating state, the electronic device continues to gradually change the FOV size of the preview image and moves the FOV center of the preview image toward the FOV center of the second camera; the preview image after the second camera enters the operating state is obtained by cropping the image captured by the second camera; The electronic device displays a preview image at a second zoom ratio on the display screen, where the preview image at the second zoom ratio is captured based on the second camera, and the FOV center of the preview image at the second zoom ratio coincides with the FOV center of the second camera.

2. The method according to claim 1, characterized in that The operation includes: a user's finger or handwriting device first contacts the option corresponding to the second zoom ratio, and then leaves the option corresponding to the second zoom ratio.

3. The method according to claim 1 or 2, characterized in that: The second zoom ratio is greater than the first zoom ratio, and the optical zoom ratio of the second camera is greater than the optical zoom ratio of the first camera; The gradually changing the size of the field of view FOV of the preview image specifically includes: gradually reducing the size of the FOV of the preview image.

4. The method according to any one of claims 1 to 3, characterized in that The second zoom ratio is smaller than the first zoom ratio, and the optical zoom ratio of the second camera is smaller than the optical zoom ratio of the first camera; The gradually changing the size of the field of view FOV of the preview image specifically includes: gradually expanding the size of the FOV of the preview image.

5. The method according to any one of claims 1 to 4, characterized in that The step of moving the FOV center of the preview image toward the FOV center of the second camera specifically occurs after the second camera has been running for a period of time.

6. The method according to any one of claims 1 to 5, characterized in that The zoom value zoomValue corresponding to the zoom ratio of the preview image in the process of gradually changing the FOV size is determined as follows: Wherein, S is the distance between the option corresponding to the first zoom ratio and the option corresponding to the second zoom ratio on the display screen, and n is a positive integer less than the number N of the second preview images.

7. The method according to any one of claims 1 to 6, characterized in that After the second camera enters the running state, the method further includes: The electronic device turns off the first camera.

8. The method according to any one of claims 1 to 7, characterized in that The focal length of the first camera is different from the focal length of the second camera.

9. The method according to claim 8, characterized in that The electronic device includes a plurality of cameras, and the first camera and the second camera are two cameras with adjacent focal lengths among the plurality of cameras.

10. An electronic device comprising a display screen, a first camera, a second camera, a memory and a processor coupled to the memory, a plurality of application programs, and one or more programs; wherein, The optical centers of the first camera and the second camera do not overlap, and the processor, when running the one or more programs, enables the electronic device to execute the method as described in any one of claims 1 to 9.

11. A computer storage medium, characterized in that: The computer storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

Images