Shooting method and related equipment thereof

Abstract

The invention relates to the field of image processing, and provides a shooting method and related equipment thereof, the shooting method is applied to electronic equipment, the electronic equipment comprises a first camera and a second camera, and the method comprises the following steps: starting a camera application program; displaying a first image, wherein the first image is obtained by collecting an image by a first camera; receiving a first zooming operation; and a second image is displayed and stored, and the second image is obtained after the OIS motor of the second camera pushes the image sensor to reduce the deviation value between the first camera and the second camera and then the second camera collects the image. According to the invention, the OIS motor pushes the image sensor to reduce the distance between the two cameras, so that smooth switching of the cameras can be realized.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a shooting method and related equipment. Background Art

[0002] With the development of shooting functions in electronic devices, camera applications are increasingly used in electronic devices. To obtain a better photo taking experience, current electronic devices are usually equipped with multiple cameras, each corresponding to a different focal length.

[0003] When shooting, electronic devices can switch between cameras with different focal lengths to perform zoom shooting in response to user input. They can also process the captured images using digital zoom to accommodate various high-magnification shooting scenarios. However, during zooming, due to the physical distance and viewing angle differences between cameras, image content and size can fluctuate between camera switches, preventing smooth camera switching.

[0004] Therefore, during the shooting process, how to smoothly switch cameras becomes a problem that needs to be solved urgently. Summary of the Invention

[0005] The present application provides a shooting method and related equipment, which reduces the distance between two cameras by pushing the image sensor through the OIS motor, thereby achieving smooth switching of the cameras.

[0006] In a first aspect, a shooting method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, and the method includes: opening a camera application; displaying a first image, where the first image is obtained by capturing an image with the first camera; receiving a first zoom operation; and displaying and saving a second image, where the second image is obtained by capturing an image with the second camera after the OIS motor of the second camera pushes the image sensor to reduce the offset value between the first camera and the second camera.

[0007] The offset value between the first camera and the second camera refers to the second Offset value in the specification.

[0008] In an embodiment of the present application, the image sensor is moved by the OIS motor included in the camera, so that the distance between the two cameras is shortened, thereby reducing the offset value between the two cameras, and further reducing the jump caused by the physical gap and perspective difference between multiple cameras, thereby achieving smooth zoom of multiple cameras and improving user experience.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the offset value between the first camera and the second camera after reduction is less than a preset tolerance value.

[0010] In an embodiment of the present application, the method can be applied to an electronic device including multiple cameras, and the OIS motor included in the camera can move the image sensor. Thus, when the offset value calculated between the two cameras is large, such as exceeding a preset margin, the image sensor is moved by the OIS motor to shorten the distance between the two cameras, thereby reducing the offset between the two cameras. After multiple detections and adjustments, when the offset value satisfies the reserved margin, smooth zoom of multiple cameras can be achieved, thereby improving the user experience.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: in response to a first zoom operation, acquiring images captured by the first camera and the second camera; performing feature point detection and registration on the images captured by the first camera and the second camera to obtain matching feature point pairs; determining an offset value between the first camera and the second camera based on the feature point pairs; determining whether the offset value exceeds the preset tolerance value; and if so, starting the OIS motor of the second camera to push the image sensor to reduce the offset value between the first camera and the second camera.

[0012] In an embodiment of the present application, the actual offset value is determined by matching feature point pairs between images captured by the first camera and the second camera, and then it is determined whether the actual offset value exceeds a preset tolerance value. If so, the offset value needs to be reduced.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: if not, displaying and saving a third image, wherein the third image is obtained by capturing an image by the second camera; looping the following steps: in response to a first zoom operation, acquiring images captured by the first camera and the second camera; performing feature point detection and registration on the images captured by the first camera and the second camera to obtain matching feature point pairs; determining an offset value between the first camera and the second camera based on the feature point pairs; judging whether the offset value exceeds the preset tolerance value; and if so, starting the OIS motor of the second camera to push the image sensor to reduce the offset value between the first camera and the second camera.

[0014] In an embodiment of the present application, the actual offset value is determined by matching feature point pairs between the images captured by the first camera and the second camera, and then it is determined whether the actual offset value exceeds the preset tolerance value. If so, the offset value needs to be reduced; if not, the above steps can be repeated for continuous detection and adjustment.

[0015] In combination with the first aspect, in some implementations of the first aspect, before determining the offset value between the first camera and the second camera based on the feature point pair, the method further includes: eliminating feature point pairs with matching errors.

[0016] In the embodiments of the present application, after testing, the number of correctly registered feature points in each scene is greater than the number of incorrectly registered feature points. Therefore, in order to improve processing efficiency and improve the effect of subsequent processing, the incorrectly registered feature points can be eliminated and only the correctly registered feature point pairs are retained.

[0017] In combination with the first aspect, in certain implementations of the first aspect, before determining whether the offset value exceeds the preset tolerance value, the method further includes: determining an initial offset value between the first camera and the second camera; and setting a preset tolerance value based on the initial offset value.

[0018] The initial offset value is the first Offset value in the specification.

[0019] In combination with the first aspect, in certain implementations of the first aspect, determining the initial offset value between the first camera and the second camera includes: obtaining calibration parameters of the first camera and the second camera; obtaining depth information of the photographed object; and determining the initial offset value based on the calibration parameters and the depth information.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: responding to the first zoom operation; determining that the zoom ratio of the electronic device satisfies the zoom ratio range when the first camera is running in the background or turned off, and the second camera is displayed in the foreground.

[0021] It should be understood that foreground display refers to the use of images captured by the camera for display; background operation refers to the use of images captured by the camera for image processing, not for display.

[0022] It should be understood that when it is determined that the zoom ratio of the electronic device meets the zoom ratio range of the first camera running in the foreground and the second camera being displayed in the background, the shooting method of the present application can also be executed, but the image displayed is the image sent by the first camera, and the second camera is only used for OIS to push the image sensor to reduce the offset value between the first camera and the second camera.

[0023] In combination with the first aspect, in certain implementations of the first aspect, the first zoom operation includes: a two-finger reverse sliding operation, a sliding operation on a displayed zoom control, a voice operation, or an air gesture operation.

[0024] In combination with the first aspect, in some implementations of the first aspect, the first camera includes a wide-angle camera, and / or the second camera includes a telephoto camera or an ultra-wide-angle camera.

[0025] In combination with the first aspect, in certain implementations of the first aspect, the OIS motor in the second camera pushes the image sensor at least along the x-axis and the y-axis, and the x-axis and the y-axis are perpendicular to each other and parallel to the plane where the image sensor is located.

[0026] In the embodiment of the present application, the OIS motor drives the image sensor to move to reduce the distance between the image sensors of the first camera and the second camera, which is equivalent to reducing the spacing between the first camera and the second camera.

[0027] Optionally, the OIS motor in the second camera can also push the image sensor in the z-axis, which is perpendicular to the x-axis and the y-axis.

[0028] According to a second aspect, an electronic device is provided, comprising: one or more processors, a memory, a first camera, and a second camera; the memory is coupled to the one or more processors, the memory being configured to store computer program code, the computer program code comprising computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to execute:

[0029] Open the camera application; display a first image, where the first image is obtained by capturing an image with the first camera; receive a first zoom operation; display and save a second image, where the second image is obtained by capturing an image with the second camera after the OIS motor of the second camera pushes the image sensor to reduce the offset value between the first camera and the second camera.

[0030] It should be understood that the expansion, limitation, explanation and description of the relevant content in the above-mentioned first aspect also apply to the same content in the second aspect.

[0031] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any one of the shooting methods in the first aspect.

[0032] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the shooting methods in the first aspect.

[0033] In a fifth aspect, a computer program product is provided, comprising: a computer program code, which, when executed by an electronic device, enables the electronic device to execute any one of the shooting methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0035] Figure 2 is a schematic diagram of a software system for an electronic device applicable to the present application;

[0036] Figure 3 This is a schematic diagram of an arrangement of multiple cameras on an electronic device provided by an embodiment of the present application;

[0037] Figure 4 This is a schematic diagram of zoom ratios corresponding to different types of cameras provided in an embodiment of the present application;

[0038] Figure 5 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0039] Figure 6 It is a schematic diagram of a set of image flows involved in the prior art;

[0040] Figure 7 This is a diagram of the offset and margin values ​​of the main camera and telephoto camera;

[0041] Figure 8 This is a schematic flow chart of a shooting method provided in an embodiment of the present application;

[0042] Figure 9 is a schematic flow chart of another shooting method provided in an embodiment of the present application;

[0043] Figure 10 is a schematic flow chart of another shooting method provided in an embodiment of the present application;

[0044] Figure 11 is a schematic diagram of a set of image flows involved in the embodiments of the present application;

[0045] Figure 12 is a schematic flow chart of another shooting method provided in an embodiment of the present application;

[0046] Figure 13 This is a schematic diagram of an application scenario of zoom switching provided by an embodiment of the present application;

[0047] Figure 14 This is a schematic diagram of an application scenario of zoom switching provided by an embodiment of the present application;

[0048] Figure 15 This is a diagram showing the correspondence between multiple cameras and zoom ratios provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In the embodiments of this application, the terms "first," "second," and the like are used for descriptive purposes only and should not be understood to indicate or imply relative importance or to implicitly indicate the number of the technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0050] First, some of the terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0051] 1. Field of view (FOV) in optical instruments is the angle formed by the two edges of the maximum range through which the image of the target object can pass through the lens, with the lens as the vertex. The size of the field of view angle determines the field of view of the optical instrument. The larger the field of view angle, the greater the field of view and the smaller the optical magnification. In other words, the target object will not be captured by the lens if it exceeds this angle. The shorter the focal length, the wider the horizontal field of view, and therefore the smaller the image. The horizontal field of view narrows as the focal length increases, and the object being photographed also increases in size.

[0052] 2. Registration refers to the matching of geographic coordinates of different images obtained by different imaging methods within the same area. This includes three aspects: geometric correction, projection transformation, and uniform scale.

[0053] 3. Zoom ratio: The zoom ratio is used to indicate the zoom size of an electronic device when shooting.

[0054] 4. Optical Image Stabilization (OIS), also known as optical image stabilization, refers to the detection of electronic device vibration by motion sensors (such as gyroscopes and accelerometers) during camera exposure. The OIS controller controls the OIS motor and moves the lens or image sensor based on the vibration data detected by the motion sensor, so that the optical path remains as stable as possible during the entire exposure period, thereby obtaining a clear exposed image.

[0055] In the embodiments of the present application, it mainly involves the OIS motor driving the image sensor.

[0056] 5. Optical axis is the direction in which light is transmitted by the optical system, with reference to the chief ray of the central field of view.

[0057] The above is a brief introduction to the nouns involved in the embodiments of this application, and no further details will be given below.

[0058] The hardware system, software system and application scenarios of the electronic device provided in the embodiments of the present application are described below with reference to the accompanying drawings.

[0059] Exemplarily, the electronic device 100 may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device 100.

[0060] See also Figure 1 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0061] It should be noted that Figure 1 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 1 More or fewer components than those shown, or the electronic device 100 may include Figure 1 Combinations of some of the components shown, or alternatively, the electronic device 100 may include Figure 1Subassemblies of some of the components shown. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0062] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices.

[0063] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0064] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0065] Exemplarily, the processor 110 can be used to execute the shooting method of the embodiment of the present application; for example, open a camera application; display a first image, where the first image is obtained by capturing an image by the first camera; receive a first zoom operation; display and save a second image, where the second image is obtained by capturing an image by the second camera after the OIS motor of the second camera pushes the image sensor and reduces the offset value between the first camera and the second camera.

[0066] Figure 1 The connection relationship between the modules shown is only for illustrative purposes and does not limit the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of the multiple connection modes in the above embodiments.

[0067] The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0068] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0069] Electronic device 100 can implement display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0070] Display screen 194 may be used to display images or videos.

[0071] For example, in an embodiment of the present application, the display screen 194 may be used to display the second image.

[0072] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0073] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0074] The camera 193 (also called the lens) is used to capture still images or videos. It can be triggered to turn on through application instructions to realize the photo function, such as capturing images of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. The light emitted or reflected by the object enters the imaging lens, passes through the filter, and finally converges on the image sensor. The imaging lens is mainly used to focus the light emitted or reflected by all objects in the photographic field of view (also called the scene to be photographed, the target scene, or the scene image that the user expects to capture) to form an image; the filter is mainly used to filter out excess light waves in the light (for example, light waves other than visible light, such as infrared); the image sensor can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is mainly used to perform photoelectric conversion on the received light signal, convert it into an electrical signal, and then transmit the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB, YUV, etc.

[0075] Exemplarily, the gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake. The gyroscope sensor 180B can also be used in scenarios such as navigation and somatosensory games.

[0076] In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0077] Among them, the camera 193 can be located in the front of the electronic device 100 or in the back of the electronic device 100. The specific number and arrangement of the cameras can be set according to needs, and this application does not impose any restrictions.

[0078] Exemplarily, the electronic device 100 includes a front camera and a rear camera. For example, the front camera or the rear camera may include one or more cameras. Taking the example of the electronic device 100 having four rear cameras, when the electronic device 100 activates the four rear cameras for shooting, the shooting method provided in the embodiments of the present application can be used.

[0079] Alternatively, the camera is provided on an external accessory of the electronic device 100, and the external accessory is rotatably connected to the frame of the mobile phone, and the angle formed between the external accessory and the display screen 194 of the electronic device 100 is any angle between 0 and 360 degrees. For example, when the electronic device 100 takes a selfie, the external accessory drives the camera to rotate to a position facing the user. Of course, when the mobile phone has multiple cameras, only some of the cameras can be provided on the external accessory, and the remaining cameras can be provided on the electronic device 100 body. This embodiment of the application does not impose any restrictions on this.

[0080] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0081] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0082] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used in scenarios such as navigation and somatosensory games.

[0083] Accelerometer 180E can detect the magnitude of the acceleration of electronic device 100 in various directions (generally the x-axis, y-axis, and z-axis). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. Accelerometer 180E can also be used to identify the posture of electronic device 100, which can serve as an input parameter for applications such as landscape / portrait switching and pedometers.

[0084] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, for example, in a photography scenario, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0085] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0086] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0087] The touch sensor 180K is also referred to as a touch-sensitive device. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a touch screen. The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor 180K can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100 and at a different location from the display screen 194.

[0088] The hardware system of the electronic device 100 is described in detail above. The software system of the electronic device 100 is introduced below.

[0089] Figure 2 Schematic diagram of the software system of the electronic device provided in an embodiment of the present application.

[0090] like Figure 2 As shown, the system architecture may include an application layer 210 , an application framework layer 220 , a hardware abstraction layer 230 , a driver layer 240 and a hardware layer 250 .

[0091] The application layer 210 may include applications such as camera application, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0092] The application framework layer 220 provides an application programming interface (API) and a programming framework for the application programs of the application layer; the application framework layer may include some predefined functions.

[0093] For example, the application framework layer 220 may include a camera access interface, which may include camera management and camera devices. The camera management may be used to provide an access interface for managing the camera, and the camera device may be used to provide an interface for accessing the camera.

[0094] The hardware abstraction layer 230 is used to abstract the hardware. For example, the hardware abstraction layer may include a camera abstraction layer and other hardware device abstraction layers; the camera hardware abstraction layer may call camera algorithms.

[0095] For example, the hardware abstraction layer 230 includes a camera hardware abstraction layer and a camera algorithm; the camera algorithm may include a software algorithm for image processing.

[0096] Exemplarily, the camera algorithm library may include algorithms corresponding to the shooting method provided in the embodiments of the present application.

[0097] For example, the algorithm in the camera algorithm may refer to a code that does not rely on specific hardware implementation; for example, a code that can generally be run in a CPU, etc.

[0098] The driver layer 240 is used to provide drivers for different hardware devices. For example, the driver layer may include a camera driver.

[0099] The hardware layer 250 is located at the bottom of the operating system; Figure 3 As shown, the hardware layer 250 may include camera 1, camera 2, camera 3, etc. Among them, camera 1, camera 2, camera 3 may correspond to multiple cameras on the electronic device.

[0100] For ease of understanding, the following takes the electronic device 100 as an example of a mobile phone having the above-mentioned software and hardware structure, and first describes in detail the camera on the electronic device 100 to which the method provided in the embodiment of the present application is applicable.

[0101] The electronic device to which the method provided in the embodiment of the present application is applicable has at least multiple cameras 193, for example, three types of cameras 193; the three types of cameras are a main camera (for example, a wide-angle camera), an ultra-wide-angle camera and a telephoto camera; the three cameras can be used to shoot the same scene to be shot.

[0102] Optionally, the electronic device 100 may also have other cameras 193. The type of cameras 193 and the number of each type of camera 193 may be set as needed, and the embodiment of the present application does not impose any restrictions on this.

[0103] For example, Figure 3 As shown, the electronic device 100 may have three cameras 193 as an example for explanation; the arrangement of the three cameras may be as follows Figure 3 As shown in (a) in , or as Figure 3 As shown in (b) in FIG; for example, the three cameras 193 can be a main camera 1931 (for example, a wide-angle camera), an ultra-wide-angle camera 1932 and a telephoto camera 1933.

[0104] It should be understood that the above are only examples of two arrangements, and other arrangements are also possible; the specific arrangement can be designed and changed as needed, and the embodiments of the present application do not impose any restrictions on this.

[0105] It should be noted that if Figure 3 As shown in (c) in the figure, when the above three cameras are shooting, the field of view angle range corresponding to the main camera 1931 is usually larger than the field of view angle range corresponding to the telephoto camera 1933; and the field of view angle range corresponding to the ultra-wide-angle camera 1932 is larger than the field of view angle range corresponding to the main camera 1931; there may be an overlap between the field of view angle of the ultra-wide-angle camera 1932 and the field of view angle of the main camera 1931; that is, the ultra-wide-angle camera 1932 can capture the scene content captured by the main camera 1931 and the scene content around it.

[0106] It should be understood that Figure 3 As shown in (c) in the figure, the field of view of the telephoto camera 1933 is smaller than that of the main camera 1931. The field of view of the main camera 1931 and the field of view of the telephoto camera 1933 may overlap; that is, the main camera 1931 can capture the scene content captured by the telephoto camera 1933 and the surrounding scene content. The field of view of the ultra-wide-angle camera 1932 and the field of view of the telephoto camera 1933 may overlap; that is, the ultra-wide-angle camera 1932 can capture the scene content captured by the telephoto camera 1933 and the surrounding scene content.

[0107] Among them, the ultra-wide-angle camera 1932 is suitable for shooting close-ups due to its smaller focusing distance; and the ultra-wide-angle camera 1932 is suitable for shooting scenes with a larger field of view; the main camera 1931 is more suitable for shooting portraits due to its higher clarity, while the telephoto camera 1933 is more suitable for shooting distant close-ups.

[0108] For example, Figure 4 As shown, the zoom ratio of the ultra-wide-angle camera can be less than M times the zoom ratio; the zoom ratio range of the wide-angle camera, i.e., the main camera, can be [M, N); the zoom ratio of the telephoto camera can be greater than or equal to N times the zoom ratio.

[0109] For example, M can be 1 and N can be 2.5; then the zoom magnification of the ultra-wide-angle camera is less than 1x zoom magnification (1×); the zoom magnification range of the wide-angle camera is 1x zoom magnification to 2.5x zoom magnification [1×~2.5×); the zoom magnification of the telephoto camera is greater than or equal to 2.5x zoom magnification.

[0110] It should be understood that during the shooting process of an electronic device, the greater the zoom ratio, the smaller the corresponding field of view angle.

[0111] The following combination Figure 5 The application scenarios of the shooting method provided in the embodiments of the present application are illustrated with examples.

[0112] The method in the embodiments of the present application can be applied to photo-taking scenarios, preview scenarios, video recording scenarios, or video call scenarios, etc.; through the method in the embodiments of the present application, smooth zoom and / or smooth switching of different cameras can be achieved in electronic devices, thereby improving the user's shooting experience and image quality.

[0113] For example, the preview scenarios include but are not limited to the following scenarios:

[0114] Photo preview, aperture preview, night scene preview, portrait preview, video preview or professional preview, etc.

[0115] It should be understood that the preview scene may refer to a scene in which the electronic device captures an image before a button indicating shooting is clicked in a certain shooting mode.

[0116] In one example, if Figure 5 As shown in (a) in FIG, after the electronic device enters the camera application, the default photo mode can be turned on; in the photo mode, the electronic device can enter the default shooting mode, which can refer to a shooting mode in which the wide-angle camera is used as the main camera and the zoom ratio is a single zoom ratio (1×), and the electronic device displays the image captured by the main camera; Figure 5 As shown in (b), in response to the user's operation, when the zoom ratio satisfies the zoom range corresponding to the telephoto camera, the electronic device can switch to display the image captured by the telephoto camera.

[0117] For example, the shooting method in the embodiment of the present application can also be applied to a video call scenario, wherein the video call scenario may include but is not limited to the following scenarios:

[0118] Video calls, video conferencing applications, long and short video applications, live video applications, online video courses, portrait intelligent mirroring application scenarios, system camera recording function video recording, video surveillance, or smart cat-eye and other portrait shooting scenarios, etc.

[0119] It should be understood that the above is an example of an application scenario and does not limit the application scenario of this application.

[0120] by Figure 3 Taking the electronic device shown in FIG. 1 as an example, the ultra-wide-angle camera 1932, the main camera 1931, and the telephoto camera 1933 are physically located at a certain distance from each other. Figure 3As shown in (b), it is assumed that the center of the main camera 1931 and the center of the ultra-wide-angle camera 1932 are located on the same horizontal line, and the vertical distance between the center of the telephoto camera 1933 and the horizontal line is 18.2 mm; the distance between the vertical line passing through the center of the main camera and the vertical line passing through the center of the telephoto camera 1933 is 10.1 mm; the distance between the vertical line passing through the center of the ultra-wide-angle camera 1932 and the vertical line passing through the center of the telephoto camera 1933 is 9.2 mm.

[0121] Currently, since the ultra-wide-angle camera 1932, the main camera 1931, and the telephoto camera 1933 are physically located at a certain distance from each other, the field of view angles corresponding to the different cameras are not only different in size, but also inevitably have deviations in the center. Figure 3 As shown in (c) in the figure, the field of view angle range corresponding to the telephoto camera 1933 is close to the upper edge of the field of view angle range corresponding to the main camera 1931, and the field of view angle range corresponding to the main camera 1931 is close to the left edge of the field of view angle range corresponding to the ultra-wide-angle camera 1932. Thus, in a preview scene or a recording scene, if the zoom factor is increased (e.g., from 1× to 10×) in response to a user operation, and the camera used to capture images is switched from the main camera 1931 to the telephoto camera 1933, since the two cameras have different corresponding field of view angle ranges and center points, the previous and next frames in the previewed or recorded video may include partially different imaging content, or the corresponding region of interest (ROI) may be disjointed, resulting in visual jumps.

[0122] It should be understood that the ROI may indicate the entire content or a portion of the content within the field of view when the camera is shooting.

[0123] For example, Figure 6 is a set of image streams involved in the prior art. Figure 6 As shown in (a) to (c) in the figure, assuming that the zoom ratio gradually increases during the zoom process, the main camera first captures the image and displays it. When the zoom switching point is reached, the telephoto camera is switched to capture the image and displays it. Before and after the switching, due to the different field of view angle ranges and centers of the main camera and the telephoto camera, and often due to insufficient tolerance (Margin), the images displayed before and after the switching will have an offset (Offset) jump, resulting in non-smooth multi-camera zoom switching, and thus leading to a poor user experience.

[0124] It should be noted that Offset refers to the offset value from the main camera to the telephoto camera, or from the main camera to the ultra-wide-angle camera; Margin refers to the tolerance reserved when the main camera and the telephoto camera, or the main camera and the ultra-wide-angle camera are aligned, which can include the offset value and the rotation amount.

[0125] For example, Figure 7 Schematic diagram of the field of view of the main camera 1931 and the telephoto camera. Figure 7 As shown, the offset value between the main camera 1931 and the telephoto camera 1933 is Offset1, and the tolerance reserved when the main camera 1931 and the telephoto camera 1933 are aligned is Margin1. Since Offset1 exceeds Margin1, when switching from the main camera 1931 to the telephoto camera 1933, the images displayed before and after the switch will have an offset jump.

[0126] In view of this, an embodiment of the present application provides a shooting method; in an embodiment of the present application, the method can be applied to an electronic device including multiple cameras, and the OIS motor included in the camera can move the image sensor. Thus, when the offset value calculated between the two cameras is large, such as exceeding the preset margin, the image sensor is moved by the OIS motor to shorten the distance between the two cameras, thereby reducing the offset between the two cameras; after multiple detections and adjustments, when the offset value meets the reserved margin, smooth zoom of multiple cameras can be achieved, thereby improving the user experience.

[0127] The following combination Figure 8 The schematic flowchart of the shooting method provided in the embodiment of the present application is described in detail.

[0128] Figure 8 is a schematic flow chart of the shooting method provided in the embodiment of the present application. Figure 1 The electronic device shown is executed; the method 300 includes steps S310 to S340, and steps S310 to S340 are described in detail below.

[0129] It should be understood that in the embodiments of the present application, the electronic device includes a first camera and a second camera; wherein the first camera and the second camera are cameras of different types.

[0130] S310. Start the camera application.

[0131] For example, the user may instruct the electronic device to start the camera application by clicking the icon of the “Camera” application.

[0132] For example, when the electronic device is in the lock screen state, the user can instruct the electronic device to launch the camera application by swiping right on the electronic device's display. Alternatively, when the electronic device is in the lock screen state and the lock screen interface includes a camera application icon, the user can instruct the electronic device to launch the camera application by clicking the camera application icon. Alternatively, when the electronic device is running another application, the application has permission to call the camera application; the user can instruct the electronic device to launch the camera application by clicking the corresponding control. For example, when the electronic device is running an instant messaging application, the user can instruct the electronic device to launch the camera application by selecting the camera function control.

[0133] It should be understood that the above is an example of the operation of opening the camera application; the camera application can also be opened by voice instructions or other operations to instruct the electronic device; this application does not impose any limitation on this.

[0134] It should also be understood that launching the camera application may refer to running the camera application.

[0135] S320: Display a first image, where the first image is obtained by capturing an image with a first camera.

[0136] For example, the first camera may be Figure 3 The wide-angle camera, main camera shown, or other cameras with a larger field of view than the telephoto camera.

[0137] Exemplarily, the first image may be an RGB image captured by the main camera, or in other words, the first image may be an RGB image obtained by processing a series of camera algorithms from a Raw image captured by the main camera.

[0138] For example, Figure 13 The preview image shown in (a) is the first image captured and displayed by the first camera. For example, Figure 14 The video image shown in (a) is the first image captured and displayed by the first camera.

[0139] Optionally, when displaying the first image captured by the first camera, the depth corresponding to the photographed object may also be captured.

[0140] Illustratively, before displaying the first image, the first camera may further perform disparity detection on the content in the first image to determine the depths corresponding to different photographed objects included in the first image.

[0141] For example, Figure 14As shown in (a), when the first camera sends the first image for display, it can perform parallax detection to determine the depth corresponding to different photographed objects. The depth information can be displayed on the first image or not, and this application does not limit this.

[0142] It should be understood that the first image may include one or more photographed objects, and when displayed, a detection frame may be displayed simultaneously for each photographed object determined based on AI detection.

[0143] It should also be understood that the type of subject can be preset as needed. For example, when the type is set to face, the target object can be used to indicate a face recognized in the first image. The type of subject can also include plants, animals, etc. This embodiment of the application does not impose any restrictions on this.

[0144] S330: Receive a first zoom operation.

[0145] Optionally, receiving the first zoom operation may include: receiving the first zoom operation for the first image.

[0146] It should be understood that the zoom operation on the first image may indicate an operation instruction of the user, or may indicate a zoom operation instruction automatically triggered by the electronic device based on AI detection. The target object is one of the shooting objects included in the first image.

[0147] Exemplarily, when the zoom operation indicates an operation instruction triggered by the user, the zoom operation may include a two-finger reverse sliding operation, a click operation, a voice operation, an air gesture operation, etc. on the first image, and the embodiments of the present application do not impose any restrictions on this.

[0148] For example, Figure 14 (a) and Figure 14 As shown in (b), when the electronic device displays one or more photographic objects and a zoom control, the zoom operation may include a click operation on the target object in the first image and a sliding operation on the zoom control; or, the zoom operation may also include a click operation on the target object in the first image and a two-finger reverse sliding operation. It should be understood that when the zoom operation includes two operations, when executing, the user needs to first perform the click operation, and then perform the sliding operation on the zoom control or the two-finger reverse sliding operation on the target object. The zoom operation can also be other operations, or the zoom operation can also include three or more sub-operations, and the embodiments of the present application do not impose any restrictions on this.

[0149] S340: Display and save the second image.

[0150] The second image is obtained by the second camera capturing an image after the OIS motor of the second camera pushes the image sensor to reduce the offset value between the first camera and the second camera.

[0151] Illustratively, the OIS motor of the second camera can push the image sensor to move along the x-axis and / or y-axis within the plane in which it is located, and the x-axis and y-axis are perpendicular to each other; of course, the OIS motor can also push the image sensor to move in a direction perpendicular to the plane (i.e., the z-axis), and the z-axis is perpendicular to both the x-axis and the y-axis. This is not limited to the embodiments of the present application.

[0152] For example, if the first camera is located to the lower left of the second camera, the OIS motor of the second camera can push the image sensor to the lower left, so that the offset value between the first camera and the second camera is reduced.

[0153] Exemplarily, the first image may be an RGB image, and the corresponding second image may be an RGB image.

[0154] For example, the first zoom operation is an operation of increasing the zoom ratio, or the first zoom operation is an operation of decreasing the zoom ratio. When the zoom ratio is increased, the size of the subject included in the second image is larger than the size of the subject included in the first image. When the zoom ratio is decreased, the size of the subject included in the second image is smaller than the size of the subject included in the first image.

[0155] Optionally, the target object in the second image is centered in the second image.

[0156] Optionally, the second image includes the entire content or a portion of the target object. Figure 14 As shown, assuming that the target object is the fourth subject, after magnification, the second image may include the entire fourth subject, or the second image may also include partial content of the fourth subject, such as the face of the fourth subject; when further magnified, the second image may also include a partial face of the fourth subject.

[0157] Optionally, in order to ensure that the target object displayed by the electronic device can be smoothly enlarged during the zoom process, the above S340 can be executed in a loop multiple times, and the target object in the second image obtained each time is larger than the target object obtained last time. That is, in response to the zoom operation, the electronic device drives the image sensor with the OIS motor, reduces the second camera after the offset value between the first camera and the second camera, and obtains and displays multiple second images, each of which includes the target object, and the target object in the multiple second images gradually increases from the original size. The number of loop executions and the magnitude of the size change can be set as needed, and this application does not limit this. Among them, the original size indicates the size of the target object in the first image.

[0158] It should be understood that S340 may be executed while S330 is being executed, or S340 may be executed after S330 is executed. This embodiment of the present application does not impose any limitation on this.

[0159] For example, when in the preview scene, the second image can be displayed. Wherein, the preview shooting mode includes but is not limited to: shooting modes such as night scene preview mode, video preview mode, photo preview mode, and portrait preview mode.

[0160] Exemplarily, when in a video recording scene, the second image may be displayed and saved.

[0161] Optionally, the method further includes:

[0162] In response to a first zoom operation;

[0163] Determine whether the zoom ratio of the electronic device meets the zoom ratio range of the first camera foreground display and the second camera background operation.

[0164] For example, in response to the first zoom operation, when the zoom magnification has reached the minimum zoom magnification of the second camera but has not reached the maximum zoom magnification of the first camera (for example, Figure 15 As shown, the minimum zoom ratio of the telephoto camera is 1.8×, and the maximum zoom ratio of the main camera is 2.5×). At this time, the first image captured by the first camera continues to be displayed, but using the method provided in the embodiment of the present application, the OIS motor in the second camera will push the image sensor to reduce the offset value between the first camera and the second camera. Continuing to respond to the first zoom operation, when the zoom ratio reaches the zoom switching point (such as a zoom ratio of 2.5×), the second camera switches from background operation to foreground display. At this time, the second image captured by the second camera can be displayed. The second image is obtained by the second camera capturing an image with the OIS motor included in the second camera pushing the image sensor to reduce the offset value between the first camera and the second camera.

[0165] An embodiment of the present application provides a shooting method and related equipment; in an embodiment of the present application, the image sensor is moved by the OIS motor included in the camera, so that the distance between the two cameras is shortened, thereby reducing the offset value between the two cameras, thereby reducing the jump caused by the physical gap and perspective difference between multiple cameras, thereby achieving smooth zoom of multiple cameras and improving user experience.

[0166] Optionally, when the offset value between the first camera and the second camera is reduced to a preset tolerance value, the content captured by the first camera and the second camera can be aligned. At this time, the first camera can be switched to the second camera, and multiple cameras can achieve smooth zoom.

[0167] Optionally, when the Offset value between the first camera and the second camera is reduced to less than a preset tolerance value, the content captured by the first camera and the second camera can be aligned. At this time, if the first camera has been switched to the second camera, the second camera with the reduced Offset value can continue to capture images for display.

[0168] For example, Figure 11 is a set of image streams involved in this application. Figure 11 (a), (b) to Figure 11 As shown in (c), during the zoom process, the OIS motor of the second camera pushes the image sensor, which can reduce the offset value between the first camera and the second camera. In this way, the content captured by the first camera and the second camera can be aligned, and the FOV of the image stream can be smoothly enlarged, and the camera can be switched smoothly during the zoom process.

[0169] Figure 9 A schematic flowchart exemplarily shows another shooting method of the electronic device 100 . Figure 10 A schematic flow chart of S401 is exemplarily shown.

[0170] Taking the first camera as a main camera and the second camera as a telephoto camera as an example, the method 400 includes S401 to S409; S401 to S409 are described in detail below.

[0171] S401: Determine a first Offset value.

[0172] Optionally, the above S401 may include S4011 to S4013.

[0173] S4011. Obtain calibration parameters of each camera.

[0174] It should be understood that calibration parameters can include camera intrinsic parameters and camera extrinsic parameters. Camera intrinsic parameters are used to convert 3D world coordinates into 3D coordinates in the camera coordinate system. Camera extrinsic parameters are used to describe the position and orientation of the camera in the world coordinate system. They consist of a rotation matrix and a translation vector and are used to convert points in 3D space from the world coordinate system to the camera coordinate system.

[0175] S4012: Obtain depth information of the object.

[0176] For example, the distance to an object can be determined by emitting energy and measuring the time or phase change of its reflection. For example, a TOF (Time of Light) camera and structured light technology can be used to actively measure the distance to obtain the depth information of the object.

[0177] S4013. Determine a first Offset value according to the calibration parameters and the depth information.

[0178] The process of determining the first Offset value according to the calibration parameters and the depth information may utilize an algorithm provided by the prior art, which is not limited in the embodiments of the present application.

[0179] S402: Set a Margin value according to the first Offset value.

[0180] A preset Margin value may be set according to the determined first Offset value, and the preset Margin value is a tolerance for subsequent camera switching determinations. Margin is a proportional value, such as 10% or 20%.

[0181] It should be understood that the preset Margin value should exceed the first Offset value to ensure that when the distance between the two cameras is the first Offset value, there will be no jump when switching cameras.

[0182] S403 : In response to the zoom operation, acquire images captured by the first camera and the second camera.

[0183] For example, the image captured by the first camera may be a RAW image, a YUV image, or an RGB image captured by the main camera; the image captured by the second camera may be a RAW image, a YUV image, or an RGB image captured by the telephoto camera.

[0184] S404 : Perform image feature point detection and registration on the images captured by the first camera and the second camera to obtain multiple pairs of feature point pairs.

[0185] Optionally, image feature point detection may adopt any existing image feature point detection algorithm, and this application does not impose any limitation on this.

[0186] Exemplarily, image feature point detection is performed on the first initial image and the second initial image, respectively, to extract M feature points from the first initial image and N feature points from the second initial image; then, the M feature points and the N feature points are registered, where both M and N are integers greater than 0.

[0187] It should be understood that each pair of feature points includes a feature point belonging to the image captured by the first camera and a feature point belonging to the image captured by the second camera, and the two feature points are used to indicate the same object. The feature points after registration have a one-to-one correspondence.

[0188] Optionally, after S403 , the method may further include: screening correctly registered feature point pairs.

[0189] Specifically, the correctly registered feature point pairs can be screened out by eliminating the incorrectly registered feature point pairs.

[0190] After testing, in each scene, there are more correctly registered feature points than incorrectly registered feature points. Therefore, in order to improve processing efficiency and the effect of subsequent processing, the incorrectly registered feature points can be eliminated and only the correctly registered feature point pairs are retained.

[0191] S405 : Determine a second offset value between the first camera and the second camera according to the feature point pair.

[0192] It should be understood that the second Offset value is equivalent to the offset between the same object photographed by the first camera and the second camera, so as to reflect the actual offset value between the first camera and the second camera.

[0193] S406: Determine whether the second Offset value exceeds the preset Margin value. If yes, execute S407; if not, execute S408.

[0194] It should be understood that if the second Offset value exceeds the preset Margin value, it means that the distance between the first camera and the second camera is large. Even if it is adjusted to the reserved tolerance limit, the images captured by the first camera and the second camera cannot be aligned. Therefore, the first camera and the second camera need to be adjusted; otherwise, no adjustment is required.

[0195] S407 , starting the OIS motor of the second camera to push the image sensor, so that the offset value between the first camera and the second camera is reduced.

[0196] It should be understood that after the OIS motor of the second camera pushes the image sensor, although the Offset value between the first camera and the second camera is reduced, the size between the Offset between the first camera and the second camera and the preset Margin value still needs to be determined again. Therefore, at this time, the above S403 to S406 can be executed in a loop until the result of the judgment is that the second Offset value does not exceed the set Margin value.

[0197] S408: After the second Offset value is adjusted to within the preset Margin range, when the zoom reaches the zoom switching point, the display is switched from the first camera to the second camera.

[0198] It should be understood that when the second Offset value is adjusted to within the preset Margin range, it means that the distance between the first camera and the second camera becomes smaller, the images captured by the first camera and the second camera can be aligned, and the camera can switch smoothly. Therefore, at this time, if the zoom is reached to the zoom switching point, the display is switched from the first camera to the second camera to achieve smooth switching.

[0199] Optionally, after the switch, the first camera can be turned off, or the first camera can continue to run in the background and turn off after reaching a certain zoom ratio.

[0200] For example, Figure 15 As shown, the zoom switching point is 2.5×. After zooming to 2.5×, you can switch from the first camera to the second camera for foreground display. At this time, continue to zoom, and the first camera can run in the background until the zoom reaches 2.7×, and then the first camera is turned off.

[0201] S409: After switching to the second camera for display, continue executing S403 to 408.

[0202] It should be understood that after switching to the second camera for display, in order to ensure that the zoom is stable, the offset between the first camera and the second camera can continue to be detected and adjusted in real time.

[0203] In an embodiment of the present application, feature point detection and alignment are performed on two frames of images captured by the first camera and the second camera, and then a second offset value between the first camera and the second camera is determined based on the feature point pair. It is then determined whether the second offset value exceeds a preset Margin value. For the first camera and the second camera, if the second offset value determined between the two exceeds the preset Margin value, the content captured by the first camera and the second camera cannot be aligned. Therefore, the OIS motor of the second camera can be used to push the image sensor to reduce the distance between the first camera and the second camera, that is, to reduce the Offset value between the first camera and the second camera. In this way, the content captured by the first camera and the second camera can be aligned, and the FOV of the image can be kept consistent during zooming and camera switching, thereby ensuring smooth transition and smooth switching of the cameras.

[0204] Figure 12 The following is a schematic flow chart showing another shooting method of the electronic device 100. The method 500 includes S501 to S509, and S501 to S509 are described in detail below.

[0205] S501: Acquire an image captured by a main camera.

[0206] Exemplarily, the image captured by the main camera may be a Raw image captured by the wide-angle camera; or, the image captured by the main camera may be a Raw image captured by the ultra-wide-angle camera.

[0207] S502: First front-end processing.

[0208] Optionally, the first front-end processing may include a related algorithm for converting the Raw image captured by the main camera into a YUV image; this application does not impose any limitation on the algorithm.

[0209] For example, the first front-end processing may refer to an image processing algorithm executed in the ISP to convert a raw image into a YUV image. The first front-end processing may also include subsequent operations such as cropping and magnification based on the zoom ratio in response to a zoom operation.

[0210] S503: Acquire an image captured by the telephoto camera.

[0211] Exemplarily, the image captured by the telephoto camera may be a Raw image captured by the telephoto camera.

[0212] Optionally, S504 and S501 may be executed simultaneously, or S504 and S501 may be executed sequentially.

[0213] S504: Second front-end processing.

[0214] Optionally, the second front-end processing may include a related algorithm for converting the Raw image captured by the telephoto camera into a YUV image; this application does not impose any limitation on the algorithm.

[0215] For example, the second front-end processing may refer to an image processing algorithm executed in the ISP to convert a raw image into a YUV image. The second front-end processing may also include subsequent operations such as cropping and magnification based on the zoom ratio in response to a zoom operation.

[0216] For example, when the zoom ratio is 1× to 1.8×, only S501 is executed. When the zoom ratio is 1.8× to 2.5×, S501 and S503 are executed simultaneously. When the zoom ratio is 2.5× or above, only S503 is executed.

[0217] S505: First backend processing.

[0218] Exemplarily, the first back-end processing includes but is not limited to: brightening processing, denoising processing, saturation adjustment processing, cropping processing or deformation processing.

[0219] S506: Smoothing process (or alignment process).

[0220] Exemplarily, smoothing is performed on the image stream captured by the main camera and the image stream captured by the telephoto camera. This can be understood as the input smoothing data being two image streams, while the smoothed image stream is a single image stream. For example, the image stream captured by the main camera and the image stream captured by the telephoto camera are corrected based on the image stream captured by the main camera. For example, this smoothing process can also include intra-frame smoothing and inter-frame smoothing.

[0221] In an embodiment of the present application, the smoothing process may also include the shooting method provided by the present application. When executing S506, image feature point detection and registration may be performed on the images captured by the main camera and the telephoto camera to obtain a matching feature point pair; based on the feature point pair, a second offset value between the first camera and the second camera is determined, and it is determined whether the second offset value exceeds a preset margin value. If not, the subsequent steps are continued; if it exceeds the preset margin value, the OIS motor of the second camera is activated to push the image sensor to reduce the offset value between the first camera and the second camera, until it is reduced to no more than the preset margin value, and then the subsequent steps are continued.

[0222] S507: Second backend processing.

[0223] Exemplarily, the second back-end processing includes anti-shake processing; for example, anti-shake processing may be performed on image frames in the image stream according to a jitter parameter to obtain a processed image.

[0224] S508: Display the processed image.

[0225] For example, when the zoom ratio increases and is between 1× and 1.8×, the processed image is: the processed image corresponding to the image captured by the main camera; when the zoom is to 1.8× to 2.5×, the processed image is: the image captured by the main camera, corrected based on the image captured by the telephoto camera; when the zoom is to 2.5× and above, the processed image is the processed image corresponding to the image captured by the telephoto camera.

[0226] S509: A zoom operation is detected.

[0227] For the zoom operation, reference may be made to the description in S330 above.

[0228] Optionally, the electronic device may display an image captured by the main camera, or an image captured by the telephoto camera, and the electronic device detects a user operation on the displayed image.

[0229] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or variations based on the above examples, and such modifications or variations also fall within the scope of the embodiments of the present application.

[0230] For example, Figure 13 A schematic diagram of an application scenario provided for an embodiment of the present application.

[0231] like Figure 13 As shown in (a) of FIG, in response to a user's operation on a camera application, the electronic device may display a preview interface 1301, which includes a preview window and a shooting control, wherein a preview image 1302 displayed in the preview window may include a first shooting object, a second shooting object, and a third shooting object. Here, when the camera application is opened, the zoom ratio defaults to 1×. Figure 13 As shown in (b), assuming that the user wants to zoom in on the third photographed object located at the lower left of the preview image 1302, the user can perform a two-finger reverse sliding operation on the screen; in response to the operation, for example, Figure 13 As shown in (c) in FIG. 13 , the zoom ratio can be increased to 10×, the imaging size corresponding to the third object changes from small to large, and the third object can be located in the middle of the preview image 1303 in response to the movement of the user's finger.

[0232] When the zoom ratio increases and the camera switches, it can adapt to Figure 8 、 Figure 9 、 Figure 10 The shooting method described.

[0233] It should be understood that the above is a preview scene in the photo mode, and the embodiment of the present application can also be applied to preview scenes in other shooting modes such as video. The preview scene can refer to the scene in which the electronic device captures an image before clicking the shooting control in a certain shooting mode.

[0234] For example, Figure 14 A schematic diagram of another application scenario provided for an embodiment of the present application.

[0235] After the camera application is opened, in response to the user's click operation on the recording control, the electronic device can start recording the video and display the video recording interface 1401, such as Figure 14 As shown in (a) of FIG. 1 , the video recording interface 1401 may include a recording window, a pause control, an end control, and a zoom control. The video image 1402 displayed in the recording window may include a first subject, a second subject, a third subject, and a fourth subject. When video recording begins, the zoom ratio defaults to 1×.

[0236] If the electronic device supports and turns on the AI ​​detection and parallax detection functions, the electronic device can detect the image content and depth when capturing the video image 1402, and display multiple detection frames when displaying the video image 1402, each detection frame is used to indicate the position information of a photographed object in the video image 1402. Figure 14 As shown in (a) in FIG. 1 , four detection frames are displayed in the video image 1402 , and the four detection frames respectively indicate the position information of the first to fourth shooting objects.

[0237] Optionally, if the AI ​​detection and disparity detection functions are only used for depth detection of faces, the electronic device may display only two detection frames when displaying the video image 1402, and the two detection frames respectively indicate the face of the second subject and the face of the fourth subject.

[0238] like Figure 14 As shown in (a) in FIG, during the recording process, assuming that the user wants to zoom in on the face of the fourth shooting subject located on the left side of the video image 1402, the user can click on the detection box corresponding to the fourth shooting subject on the screen. Figure 14As shown in (b) of FIG, in response to the user's click operation, the detection frame of the fourth shooting object included in the video image 1403 can be selected. For example, the selected detection frame can be in a different color and / or style than other detection frames. Then, the electronic device can receive the user's sliding operation on the zoom control, such as Figure 14 As shown in (c) in the figure, in response to a sliding operation on the zoom control (assuming sliding to 8×), the electronic device can display a video image 1404, in which the face of the fourth subject included in the video image 1404 becomes larger relative to the face of the fourth subject included in the video image 1402, and the face of the fourth subject can be located in the middle position of the video image 1404 in response to the movement of the user's finger.

[0239] It should be noted that if the electronic device detects the image content and only detects one subject or one face, the user does not need to click on the detection box corresponding to the subject or face, and the electronic device automatically selects it; then, in direct response to the user's sliding operation on the zoom control, the subject or face is enlarged and displayed in the center.

[0240] When the zoom ratio changes and the camera switches, it can adapt Figure 8 、 Figure 9 、 Figure 10 The shooting method described.

[0241] It should be understood that the above is a video recording scenario, and the embodiments of the present application can also be applied to scenarios such as video calls.

[0242] Exemplarily, video call scenarios may include but are not limited to the following scenarios: video calls, video conferencing applications, long and short video applications, live video applications, online video courses, portrait smart mirror application scenarios, system camera recording function video recording, video surveillance, or smart cat-eye and other shooting scenarios.

[0243] Combined with the above Figures 1 to 14 The shooting method provided in the embodiment of the present application is described in detail.

[0244] The present application also provides a computer program product, which, when executed by a processor, implements the method described in any method embodiment of the present application.

[0245] The computer program product may be stored in a memory, for example, a program, which is converted into an executable target file that can be executed by a processor after undergoing processes such as preprocessing, compilation, assembly, and linking.

[0246] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements the method described in any method embodiment of the present application. The computer program can be a high-level language program or an executable target program.

[0247] The computer-readable storage medium is, for example, a memory. The memory may be a volatile memory or a non-volatile memory, or the memory may include both volatile memory and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0248] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0249] In the several embodiments provided in this application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not executed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0250] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0251] Additionally, the terms "system" and "network" are often used interchangeably. The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0252] In short, the above description is only a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

Claims

1. A shooting method, characterized in that: Applied to an electronic device, the electronic device includes a first camera and a second camera, and the method includes: Open the Camera app; Display a first image, where the first image is acquired by the first camera; receiving a first zoom operation; Display and save the second image, where the second image is obtained by capturing an image with the second camera after the OIS motor of the second camera pushes the image sensor and reduces the offset value between the first camera and the second camera.

2. The shooting method according to claim 1, wherein: The reduced offset value between the first camera and the second camera is smaller than a preset tolerance value.

3. The shooting method according to claim 2, characterized in that: The method further comprises: In response to a first zoom operation, acquiring images captured by the first camera and the second camera; Performing feature point detection and registration on images captured by the first camera and the second camera to obtain matching feature point pairs; determining an offset value between the first camera and the second camera according to the feature point pair; Determining whether the offset value exceeds the preset tolerance value; If so, the OIS motor of the second camera is started to push the image sensor to reduce the offset value between the first camera and the second camera.

4. The shooting method according to claim 3, characterized in that: The method further comprises: If not, display and save a third image, where the third image is obtained by capturing an image with the second camera; The steps of claim 3 are executed in a loop.

5. The shooting method according to claim 3 or 4, characterized in that: Before determining the offset value between the first camera and the second camera according to the feature point pair, the method further includes: Eliminate incorrectly matched feature point pairs.

6. The shooting method according to any one of claims 3 to 5, characterized in that: Before determining whether the offset value exceeds the preset tolerance value, the method further includes: determining an initial offset value between the first camera and the second camera; A preset tolerance value is set according to the initial offset value.

7. The shooting method according to claim 6, characterized in that: Determining an initial offset value between the first camera and the second camera includes: Obtaining calibration parameters of the first camera and the second camera; Obtain depth information of the object; The initial offset value is determined according to the calibration parameter and the depth information.

8. The shooting method according to any one of claims 1 to 6, characterized in that: The method further includes: in response to the first zoom operation; Determine that the zoom ratio of the electronic device meets the zoom ratio range that the first camera runs in the background or is closed, and the second camera is displayed in the foreground.

9. The shooting method according to any one of claims 1 to 8, characterized in that: The first zoom operation includes: a two-finger reverse sliding operation, a sliding operation on a displayed zoom control, a voice operation, or an air gesture operation.

10. The shooting method according to any one of claims 1 to 9, characterized in that: The first camera includes a wide-angle camera, and / or the second camera includes a telephoto camera or an ultra-wide-angle camera.

11. The shooting method according to any one of claims 1 to 10, characterized in that: The OIS motor in the second camera pushes the image sensor at least along the x-axis and the y-axis, where the x-axis and the y-axis are perpendicular to each other and parallel to the plane where the image sensor is located.

12. An electronic device, characterized in that: The electronic device includes: one or more processors, a memory, a first camera and a second camera; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the shooting method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the photographing method according to any one of claims 1 to 11.

Images