Shooting methods and related equipment

Abstract

The present application relates to the field of image processing, and provides a shooting method and related equipment thereof, the shooting method is applied to an electronic device, the electronic device includes a first camera and a movable camera, the method includes: starting a camera application; displaying a first image captured by the first camera; in response to a zoom operation on the first image, processing the first image to obtain a second image; in response to the zoom operation, the movable camera rotates; obtaining a third image captured after the movable camera rotates; combining the second image, processing the third image to obtain a fourth image. Based on the present technical solution, directional zoom can be achieved in combination with the rotation of the movable camera.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a shooting method and related equipment. Background Art

[0002] With the development of shooting functions in electronic devices, camera applications are increasingly used in electronic devices. To obtain a better photo-taking experience, current electronic devices are usually equipped with multiple cameras, each with a different focal length. For example, an electronic device may include a wide-angle camera, a main camera, and a telephoto camera. The wide-angle camera has a focal length of 0x to 1x, the main camera has a focal length of 1x to 5x, and the telephoto camera has a focal length of 5x to 20x.

[0003] When shooting, in response to user operations, electronic devices can switch cameras with different focal lengths to perform zoom shooting, and can also process the captured images in combination with digital zoom to meet various high-magnification shooting scenarios. However, during the zooming process, due to the influence of physical position, there are jumps in imaging content and size between the switching of different cameras, and the camera cannot be switched smoothly; and digital zoom can only magnify and crop the imaging content in the center of the field of view, which is very limited.

[0004] Therefore, during the shooting process, how to smoothly switch the camera and zoom in on the imaging content at any position becomes an urgent problem to be solved. Summary of the invention

[0005] The present application provides a shooting method and related equipment, which can be combined with camera rotation to perform zoom shooting (directional zoom) for a target object at any position.

[0006] In a first aspect, a shooting method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a movable camera, and the method includes:

[0007] Open the Camera app;

[0008] Displaying a first image captured by the first camera;

[0009] In response to a zoom operation on the first image, processing the first image to acquire a second image;

[0010] In response to the zoom operation, rotating the movable camera;

[0011] Acquire a third image captured by the movable camera after rotation;

[0012] combining the second image, processing the third image to obtain a fourth image; the first image, the second image, the third image and the fourth image all include a target object;

[0013] The fourth image is displayed and saved.

[0014] In a possible implementation, in response to a zoom operation on the first image, the electronic device may perform AI detection on the first image, and the zoom operation may correspond to a zoom operation instruction automatically triggered after the AI ​​detection of the electronic device.

[0015] In an embodiment of the present application, the electronic device includes a movable camera. Through the rotation of the movable camera, the field of view angle of the telephoto camera can be overcome to a certain extent to meet the needs of different shooting scenes; when the camera in the electronic device is flexibly rotated, in the scheme of the present application, when the electronic device switches from the first camera to the telephoto camera, the jump in imaging content and size can be avoided, and smooth switching can be achieved; and, in response to user operation or in response to automatic recognition, combined with the rotation of the telephoto camera, the electronic device can zoom in on the imaging content at any position in the currently displayed image, or in other words, directional zoom can be achieved.

[0016] In combination with the first aspect, in some implementations of the first aspect, processing the first image to obtain the second image includes: performing at least non-center cropping and enlarging the first image to obtain the second image;

[0017] The size of the target object in the second image is larger than the size of the target object in the first image.

[0018] In this implementation, when the zoom ratio is high, the user needs to capture a subject by clicking on the main interface (first image) to zoom in on that position, and the center zoom is no longer used.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the zoom operation includes a two-finger reverse sliding operation, a sliding operation on a displayed zoom control, or includes a click operation on a target object in the first image and a sliding operation on the zoom control.

[0020] With reference to the first aspect, in some implementations of the first aspect, processing the third image in combination with the second image to obtain a fourth image includes:

[0021] Performing intra-frame compensation and image rotation correction processing on the third image to obtain an image rotation corrected image;

[0022] Performing anti-shake algorithm processing on the image rotation correction image to obtain an anti-shake image;

[0023] Combined with the second image, the anti-shake image is aligned to obtain the fourth image.

[0024] In this implementation, when the electronic device realizes flexible rotation, smooth switching and directional zooming of the camera, it can also be combined with image stabilization processing to improve the image stabilization effect.

[0025] In combination with the first aspect, in some implementations of the first aspect, processing the first image to obtain the second image includes: processing the first image multiple times to obtain multiple second images; or processing the first image multiple times within a preset time length to obtain multiple second images, wherein the preset time length is greater than the time length of the movable camera rotating;

[0026] The size of the target object in the second image acquired each time is larger than the size of the target object in the second image acquired last time.

[0027] Optionally, smoothing processing may be performed on the plurality of second images.

[0028] In this implementation, when the zoom ratio is small and before switching from the first camera to the movable camera, the first camera continues to output images for display, and ensures smooth changes in the target object and the zoom ratio.

[0029] With reference to the first aspect, in some implementations of the first aspect, performing alignment processing on the anti-shake image in combination with the second image to obtain the fourth image includes:

[0030] The anti-shake image is aligned with the second image that is acquired closest to the time of acquiring the third image, or with the second image that is acquired for the last time when a preset time length is reached, to obtain the fourth image.

[0031] In this implementation, when the zoom ratio is large and the switch is made from the first camera to the movable camera, since the image output of the first camera is aligned, the image output of the movable camera can change smoothly when displayed, thus avoiding abrupt changes or jumps in the image content.

[0032] In combination with the first aspect, in certain implementations of the first aspect, the field of view angle range of the first camera is greater than the field of view angle range of the movable camera, and the clarity of the first image is less than the clarity of the fourth image.

[0033] In this implementation, when switching from the first camera to the movable camera, the quality of the displayed image can be improved.

[0034] In combination with the first aspect, in certain implementations of the first aspect, the movable camera includes an optical lens and a motor, the motor includes a rotation motor and an anti-shake motor, the rotation motor rotates along an X-axis and a Y-axis to rotate the movable camera, and the anti-shake motor is used to perform translational motion along the X-axis and / or the translational motion along the Y-axis to perform optical image stabilization.

[0035] In combination with the first aspect, in some implementations of the first aspect, the first camera includes a wide-angle camera or an ultra-wide-angle camera, and the movable camera includes a movable telephoto camera.

[0036] In this implementation, the telephoto camera has a smaller field of view than the wide-angle and ultra-wide-angle cameras but has higher clarity.

[0037] In a second aspect, an electronic device is provided, the electronic device comprising: one or more processors, a memory, a first camera, and a movable camera; the memory is coupled to the one or more processors, the memory is used to store a computer program code, the computer program code comprises a computer instruction, and the one or more processors call the computer instruction to cause the electronic device to execute:

[0038] Open the Camera app;

[0039] Displaying a first image captured by the first camera;

[0040] In response to a zoom operation on the first image, processing the first image to acquire a second image;

[0041] In response to the zoom operation, the movable camera rotates;

[0042] Acquire a third image captured by the movable camera after rotation;

[0043] combining the second image, processing the third image to obtain a fourth image; the first image, the second image, the third image and the fourth image all include a target object;

[0044] The fourth image is displayed and saved.

[0045] It should be understood that the expansion, limitation, explanation and description of the relevant contents in the above-mentioned first aspect also apply to the same contents in the second aspect.

[0046] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes any one of the shooting methods in the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the shooting methods in the first aspect.

[0048] In a fifth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes any one of the shooting methods in the first aspect.

[0049] In an embodiment of the present application, the electronic device includes a movable camera. Through the rotation of the movable camera, the field of view angle of the telephoto camera can be overcome to a certain extent to meet the needs of different shooting scenes; when the camera in the electronic device is flexibly rotated, in the scheme of the present application, when the electronic device switches from the first camera to the telephoto camera, the jump in imaging content and size can be avoided, and smooth switching can be achieved; and, in response to user operation or in response to automatic recognition, combined with the rotation of the telephoto camera, the electronic device can zoom in on the imaging content at any position in the currently displayed image, or in other words, directional zoom can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0051] Figure 2 is a schematic diagram of the arrangement of cameras on an electronic device according to an embodiment of the present application;

[0052] Figure 3 It is a schematic diagram of an interface involved in a zooming method provided by the related art;

[0053] Figure 4 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0054] Figure 5 is a schematic diagram of another application scenario provided by an embodiment of the present application;

[0055] Figure 6 is a schematic diagram of a side view structure corresponding to a telephoto camera provided in an embodiment of the present application;

[0056] Figure 7 is a side view schematic diagram of an OIS controller controlling lens movement in a telephoto camera provided by an embodiment of the present application;

[0057] Figure 8 It is a schematic diagram of the structure of the rotary motor assembly provided in the embodiment of the present application controlling the movement of the prism around the x-axis;

[0058] Fig. 9 is a side view schematic diagram of a rotary motor assembly provided in an embodiment of the present application controlling the movement of a prism around an x-axis;

[0059] Fig.10 yes Figure 8 , Fig. 9 The corresponding optical path diagram;

[0060] Fig.11 It is a schematic diagram of the structure of the rotary motor assembly provided in the embodiment of the present application controlling the movement of the prism around the y-axis;

[0061] Fig.12 is a front view schematic diagram of a rotary motor assembly provided in an embodiment of the present application controlling the movement of a prism around the y-axis;

[0062] Fig.13 yes Fig.11 , Fig.12 The corresponding optical path diagram;

[0063] Fig.14 yes Fig.13 Schematic diagram of the position of the corresponding imaging points;

[0064] Fig.15 This is a schematic diagram of a change in the field of view angle range provided in an embodiment of the present application;

[0065] Fig.16 is a schematic diagram of two frames of images provided in an embodiment of the present application;

[0066] Fig.17 is a schematic flow chart of a shooting method provided in an embodiment of the present application;

[0067] Fig.18 is a schematic diagram of the movement of a movable camera provided in an embodiment of the present application;

[0068] Fig.19 It is a schematic diagram of the change of the field of view angle of a movable camera before and after movement provided by an embodiment of the present application;

[0069] Fig. 20 is a schematic flow chart of another shooting method provided in an embodiment of the present application;

[0070] Fig.21 It is a schematic diagram of a coordinate system mapping relationship provided in an embodiment of the present application;

[0071] Fig. 22 is a schematic diagram of a change in a viewing angle provided in an embodiment of the present application;

[0072] Fig.23 This is a schematic diagram of changes in an image captured by a movable camera after movement provided by an embodiment of the present application;

[0073] Fig.24 is a schematic diagram of calculating a new memory matrix provided by an embodiment of the present application;

[0074] Fig.25 is a schematic diagram of another shooting method provided in an embodiment of the present application;

[0075] Fig.26 is a schematic diagram of image transformation for zoom switching provided in an embodiment of the present application;

[0076] Fig. 27 is a schematic diagram of another shooting method provided in an embodiment of the present application;

[0077] Fig.28 is a schematic diagram of image division provided in an embodiment of the present application;

[0078] Fig.29 is a schematic diagram of an image stabilization method provided by an embodiment of the present application;

[0079] Fig.30 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0080] Fig.31 is a schematic diagram of an electronic device suitable for the present application. DETAILED DESCRIPTION

[0081] In the embodiments of the present application, the following terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present embodiment, unless otherwise specified, "plurality" means two or more.

[0082] First, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0083] 1. The movable camera may include an optical lens and a motor; the motor includes: a rotation motor and an anti-shake motor; the anti-shake motor may perform translational anti-shake on the x-axis; and at the same time, it may rotate around the x-axis to achieve anti-shake on the y-axis; the rotation motor may perform large rotations around the x-axis and the y-axis to achieve lens steering.

[0084] 2. Calibration. The process of solving parameters can be called calibration. For example, in the embodiment of the present application, optical image stabilization calibration can refer to converting the coordinates used in the motion correction of optical image stabilization into coordinates that can be recognized by the motion correction of electronic image stabilization.

[0085] For example, it may refer to converting the original code of optical image stabilization collected by the Hall sensor into data in pixel units.

[0086] 3. Optical Image Stabilization (OIS), which can also be called optical image stabilization, refers to the detection of the shake of electronic equipment by motion sensors (for example, gyroscopes, accelerometers) during camera exposure. The OIS controller controls the motor that drives the OIS and moves the lens or image sensor based on the shake data detected by the motion sensor, so that the optical path remains as stable as possible during the entire exposure period, thereby obtaining a clear exposed image.

[0087] 4. Electronic Image Stabilization (EIS), also known as electronic image stabilization, is an anti-shake process based on motion sensor data. The motion between image frames in the image sequence is calculated through the data collected by the motion sensor during the exposure of each frame of the image. The motion between image frames is corrected to generate a relatively stable image sequence.

[0088] 5. Optical axis is the direction in which light is transmitted by the optical system, referring to the chief ray of the central field of view.

[0089] 6. Field of view (FOV), in optical instruments, is the angle formed by the two edges of the maximum range of the image of the target object that can pass through the lens, with the lens of the optical instrument as the vertex. The size of the field of view angle determines the field of view of the optical instrument. The larger the field of view angle, the larger the field of view and the smaller the optical magnification, that is, the target object will not be captured by the lens if it exceeds this angle. The shorter the focal length, the wider the horizontal field of view, and the smaller the image. The horizontal field of view becomes narrower as the focal length increases, and the object being photographed increases accordingly.

[0090] 7. In this application, the image coordinate system can also be called the standard coordinate system.

[0091] 8. Baseline difference: The distance between the optical centers of the two cameras results in the distance between the center points of the images taken by the two cameras. The center of the lens included in the camera can be approximately regarded as the optical center.

[0092] 9. Focus: Parallel light converges into a point after passing through the lens, and this point is called the focus.

[0093] Focal Plane: The plane perpendicular to the optical axis of an imaging system and containing the focal point of the imaging system.

[0094] Image plane: The plane that is perpendicular to the optical axis of the imaging system and contains the image points of the imaging system.

[0095] Image distance: the distance from the optical center to the image plane.

[0096] Focal length: the distance from the optical center to the focal plane.

[0097] Object distance: the distance from the optical center to the object.

[0098] The above is a brief introduction to the terms involved in the embodiments of the present application, which will not be repeated below.

[0099] Below we first combine the attached Figure 1 The hardware system of the electronic device provided in the embodiment of the present application is described.

[0100] Exemplarily, the electronic device 100 may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiment of the present application does not impose any limitation on the specific type of the electronic device 100.

[0101] See also Figure 1 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0102] It should be noted that Figure 1The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 1 More or fewer components than those shown, or the electronic device 100 may include Figure 1 A combination of some of the components shown, or the electronic device 100 may include Figure 1 Subassemblies of some of the components shown. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0103] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0104] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0105] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0106] Figure 1 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of multiple connection modes in the above embodiments.

[0107] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0108] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0109] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0110] The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0111] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0112] The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0113] Exemplarily, the motor 191 can generate vibrations. The motor 191 can be used for incoming call reminders or for touch feedback. The motor 191 can generate different vibration feedback effects for touch operations acting on different applications. The motor 191 can also generate different vibration feedback effects for touch operations acting on different areas of the display screen 194. Different application scenarios (for example, time reminders, receiving messages, alarm clocks, and games) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0114] Display screen 194 may be used to display images or videos.

[0115] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0116] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0117] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0118] For example, in an embodiment of the present application, the camera 193 may include a movable camera.

[0119] For example, in an embodiment of the present application, the processor 110 may execute: start a camera application; the display screen 194 is used to display a first image captured by the first camera; the processor 110 may also execute: in response to a zoom operation on the first image, process the first image to obtain a second image; in response to the zoom operation, rotate the movable camera; obtain a third image captured after the movable camera rotates; combine the second image, process the third image to obtain a fourth image; the first image, the second image, the third image, and the fourth image all include a target object; save the fourth image. The display screen 194 is also used to display the fourth image.

[0120] Exemplarily, the gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used in scenes such as navigation and somatosensory games.

[0121] Exemplarily, in an embodiment of the present application, the gyro sensor 180B may be used to collect jitter information, and the jitter information may be used to indicate changes in the posture of the electronic device during shooting.

[0122] For example, the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally the x-axis, y-axis, and z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 as an input parameter for applications such as horizontal and vertical screen switching and pedometers.

[0123] Exemplarily, the distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, for example, in a shooting scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0124] Exemplarily, the ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0125] The hardware system of the electronic device 100 is described in detail above, and the software system of the electronic device 100 is introduced below. The software system can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the layered architecture as an example to exemplarily describe the software system of the electronic device 100.

[0126] Taking a mobile phone having the above hardware system as an example, the camera included therein is introduced in detail below.

[0127] For example, Figure 2 A schematic diagram showing the arrangement of cameras 193 on a mobile phone is shown.

[0128] The electronic device provided in the present application may include one or more cameras 193, which may be located on the front of the electronic device 100 or on the back of the electronic device 100. The camera 193 located on the front of the electronic device 100 may be referred to as a front camera, and the camera located on the back of the electronic device 100 may be referred to as a rear camera. In the present application, a camera may also be referred to as a camera module.

[0129] For example, in the embodiment of the present application, the electronic device 100 may include five cameras 193, including two front cameras and three rear cameras. Figure 2 (a) and Figure 2As shown in (b), the three rear cameras are arranged in a row from top to bottom on the back cover of the electronic device 100, and in the arrangement order, the three rear cameras are respectively a main camera 1931, a telephoto camera 1932 and a wide-angle camera 1933.

[0130] It should be understood that the above is only an example, and the electronic device 100 may also include other types of cameras, such as an ultra-wide-angle camera, a black and white camera, a multi-spectral camera, etc., which is not limited in this application.

[0131] Among them, combined Figure 2 , the wide-angle camera 1933 has a shorter focal length than the main camera 1931, and the telephoto camera 1932 has a longer focal length than the main camera 1931. Because the longer the focal length, the smaller the field of view angle, and the higher the clarity; therefore, Figure 2 As shown in (c) in FIG. 1 , the field of view corresponding to the wide-angle camera 1933 is larger than that corresponding to the main camera 1931, while the field of view corresponding to the telephoto camera 1932 is smaller than that corresponding to the main camera 1931. The clarity of the image captured by the wide-angle camera 1933 is relatively lower than that of the main camera 1931, while the clarity of the image captured by the telephoto camera 1932 is relatively higher than that of the main camera 1931.

[0132] Here, it should be noted that, since the wide-angle camera 1933, the main camera 1931 and the telephoto camera 1932 are physically located at a certain distance, the field of view angles corresponding to different cameras are not only different in size, but also inevitably have deviations in the center. Figure 2 As shown in (c) in FIG. 1 , the field of view angle range corresponding to the telephoto camera 1932 points to the edge of the field of view angle range corresponding to the main camera 1931. In this way, in the preview scene or the recording scene, if the zoom factor is increased (such as switching from 1x to 10x) in response to the user operation, when the camera used to collect images is switched from the main camera 1931 to the telephoto camera 1932, since the field of view angle ranges and center points of the two cameras are different, the previous and next frame images in the previewed or shot video may include partially different imaging contents, or the corresponding region of interest (ROI) area may be incoherent, thereby causing visual jumps.

[0133] It should be understood that the ROI may indicate the entire content or a portion of the content within the field of view when the camera is shooting.

[0134] For example, Figure 3 A schematic diagram of an interface involved in a zooming method provided in the related art.

[0135] like Figure 3 As shown, in the prior art, when the zoom factor is increased, the center zoom method is usually used. For example, the electronic device displays a preview interface 301, which may include a preview window and a zoom control. In the default mode, the zoom control is Figure 3 The preview interface 301 is displayed in the style 302 shown in (a) of FIG. 1 , which only includes three zoom factors: 0.6x, 1x, and 2x, for illustration; the image in the preview window defaults to the imaging content corresponding to 1x. In response to the user's sliding operation on the zoom control 302 (such as sliding to the left), the preview interface 301 can be switched to the following: Figure 3 In the display interface 303 shown in (b), the zoom control is switched from style 302 to style 304 for display; when the user slides the zoom factor from 1x to 5x, the image in the preview window is cropped and enlarged from the center based on the imaging content corresponding to 1x for display. Figure 3 That is to say, when switching from 1x to 5x, only the three flowers captured can be cropped and enlarged from the center point, and the enlarged content is the partial image at the center position before enlargement. In addition, it is not possible to zoom in on the partial content at any other position.

[0136] In order to overcome the above-mentioned problems, the present application improves the hardware and realizes flexible rotation of the camera, so that the field of view angle corresponding to the camera can be flexibly changed while keeping the clarity unchanged. Therefore, when switching the camera, the imaging content of different cameras can be made the same by controlling the rotation of the camera, thereby achieving smooth switching; at the same time, in response to user operations or in response to automatic recognition, combined with the flexible rotation of the camera, the imaging content at any position can be zoomed, and the center zoom is no longer used.

[0137] In one possible implementation, the present application improves the telephoto camera and can realize the rotation of the telephoto camera (the telephoto camera can also be called a movable camera), so that when the electronic device switches from the main camera to the telephoto camera, the jump in imaging content and size can be avoided and smooth switching can be achieved; and, in response to user operations or in response to automatic recognition, combined with the rotation of the telephoto camera, the electronic device can zoom in on the imaging content at any position in the currently displayed image, or in other words, directional zoom can be achieved.

[0138] On this basis, when the electronic device can achieve flexible rotation, smooth switching and directional zoom of the camera, it can also be combined with image stabilization processing to improve the image stabilization effect.

[0139] For example, Figure 4 A schematic diagram of an application scenario provided for an embodiment of the present application.

[0140] like Figure 4 As shown in (a) of FIG. 1 , in response to a user's operation on a camera application, the electronic device may display a preview interface 401, which includes a preview window and a shooting control, wherein a preview image 402 displayed in the preview window may include a first shooting object, a second shooting object, and a third shooting object. Here, when the camera application is opened, the zoom ratio defaults to 1x. Figure 4 As shown in (b) of FIG. 4 , assuming that the user wants to zoom in on the third photographed object located at the lower left of the preview image 402, the user can perform a two-finger reverse sliding operation on the screen; in response to the operation, for example, Figure 4 As shown in (c) in FIG. 4 , the zoom factor can be increased to 10x, the image size corresponding to the third photographed object changes from small to large, and the third photographed object can be located in the middle of the preview image 403 in response to the movement of the user's finger.

[0141] It should be understood that the above is a preview scene in the photo mode, and the embodiment of the present application can also be applied to preview scenes in other shooting modes such as video recording. The preview scene can refer to the scene in which the electronic device captures an image before clicking the shooting control in a certain shooting mode.

[0142] For example, Figure 5 A schematic diagram of another application scenario provided for an embodiment of the present application.

[0143] After the camera application is opened, in response to a user clicking operation on a recording control, the electronic device may start recording a video and display a video recording interface 501, such as Figure 5 As shown in (a) of FIG. 5 , the video recording interface 501 may include a recording window, a pause control, an end control, and a zoom control. The video image 502 displayed in the recording window may include a first shooting object, a second shooting object, a third shooting object, and a fourth shooting object. Here, when the video recording starts, the zoom multiple is 1x by default.

[0144] If the electronic device supports and turns on the AI ​​detection function, the electronic device can detect the image content when capturing the video image 502, and display multiple detection frames when displaying the video image 502, each detection frame is used to indicate the position information of a photographed object in the video image 502. Figure 5 As shown in (a) in FIG. 5 , four detection frames are displayed in the video image 502 , and the four detection frames respectively indicate the position information of the first to fourth shooting objects.

[0145] Optionally, if the AI ​​detection function is only used for face detection, the electronic device may display only two detection frames when displaying the video image 502, and the two detection frames respectively indicate the face of the second shooting subject and the face of the fourth shooting subject.

[0146] like Figure 5 As shown in (a) of FIG. 5 , during the recording process, assuming that the user wishes to zoom in on the face of the fourth shooting subject located on the left side of the video image 502, the user can click on the detection box corresponding to the fourth shooting subject on the screen. Figure 5 As shown in (b) of FIG. 5 , in response to a click operation by the user, the detection frame of the fourth shooting object included in the video image 503 may be in a selected state. For example, the selected detection frame may be in a different color and / or style than other detection frames. Then, the electronic device may receive a sliding operation of the user on the zoom control, such as Figure 5 As shown in (c) in the figure, in response to a sliding operation on the zoom control (assuming sliding to 8x), the electronic device can display a video image 504, in which the face of the fourth subject included in the video image 504 becomes larger relative to the face of the fourth subject included in the video image 502, and the face of the fourth subject can be located in the middle of the video image 504 in response to the movement of the user's finger.

[0147] It should be noted that if the electronic device detects the image content and only detects one subject or one face, the user does not need to click on the detection box corresponding to the subject or the face, the electronic device automatically selects it; then, in direct response to the user's sliding operation on the zoom control, the subject or the face is enlarged and displayed in the center.

[0148] It should be understood that the above is a video recording scenario, and the embodiments of the present application can also be applied to scenarios such as video calls.

[0149] Exemplarily, video call scenarios may include but are not limited to the following scenarios: video calls, video conferencing applications, long and short video applications, video live broadcast applications, video online course applications, portrait intelligent mirror application scenarios, system camera recording function video recording, video surveillance, or smart cat-eye and other shooting scenarios.

[0150] For example, Figure 6 A side view structural schematic diagram corresponding to a telephoto camera 1932 provided in the present application is shown.

[0151] like Figure 6As shown, the telephoto camera 1932 includes a lens, an OIS controller, a rotating motor assembly including a prism, and a photosensitive element, wherein the plane where the photosensitive element is located is perpendicular to the plane where the lens included in the lens is located; in addition, the prism in the rotating motor assembly is arranged in an inclined state, so that the light emitted from the lens can be refracted onto the photosensitive element; on this basis, the rotating motor in the rotating motor assembly can also control the rotation of the prism to cause the light path emitted from the lens to shift, thereby expanding the field of view angle range of the shooting. The rotating motor assembly can also include a motor, which is used to drive the rotating motor. Here, it should be noted that the rotating motor in the rotating motor assembly is not the same motor as the OIS motor in the OIS controller.

[0152] refer to Figure 7 As shown in (a) of FIG. 1 , the OIS motor in the OIS controller can drive the lens to move in the x-axis direction. Figure 4 The movement method shown is the same and will not be repeated here. Figure 7 As shown in (b) in FIG. 1 , the OIS motor in the OIS controller can also drive the lens to move in the y-axis direction. Figure 5 The movement method shown is the same and will not be repeated here.

[0153] In some embodiments, reference Figure 8 (a) and Figure 8 As shown in (b) of Figure 8 Taking the coordinate system shown in (a) in FIG. 1 as an example, the rotary motor in the rotary motor assembly can control the prism to rotate up and down around the x-axis, that is, nod, to expand the field of view in the z-axis direction. Specifically, Fig. 9 As shown in (a), the dotted line position indicates the initial position of the prism, and the rotating motor can rotate the prism around the x-axis and toward the positive direction of the z-axis to expand the field of view in the positive direction of the z-axis; or, Fig. 9 As shown in (b) in the figure, the rotating motor can also rotate the prism around the x-axis and toward the negative direction of the z-axis to expand the field of view in the negative direction of the z-axis.

[0154] See also Fig.10 , Fig.10 Shows Fig. 9 The schematic diagram of the optical path when the prism shown in (a) moves in the nodding direction. Fig.10 As shown in the figure, when the prism does not move, the two incident light rays are reflected by the prism and the image points on the photosensitive element are P and Q respectively. Fig.10 The nodding direction shown in Fig.10 When the prism is in the middle position, the same two incident light rays are reflected by the prism, and the imaging points on the photosensitive element are P' and Q' respectively. Before and after the movement, P' corresponds to P, and Q' corresponds to Q.

[0155] Here, it should be noted that since the nodding motion is in the plane of the optical path, when the prism is nodding, both P and Q will be displaced, and the displacements are almost equal. Therefore, the length of the line segment PQ before the nodding motion is basically equal to the length of the line segment P'Q' after the nodding motion. Therefore, when nodding, the image will not shake in the y-axis direction, that is, there will be no image rotation problem and no other distortion.

[0156] In other embodiments, reference Fig.11 (a) and Fig.11 As shown in (b) of Fig.11 Taking the coordinate system shown in (a) in FIG. 1 as an example, the rotary motor in the rotary motor assembly can control the prism to rotate left and right around the y-axis, that is, to shake the head, so as to expand the field of view in the x-axis direction. Specifically, Fig.12 As shown in (a), the dotted line position indicates the initial position of the prism, and the rotating motor can rotate the prism around the y-axis and in the positive direction of the x-axis to expand the field of view in the positive direction of the x-axis; or Fig.12 As shown in (b) in the figure, the rotating motor can also rotate the prism around the y-axis and toward the negative direction of the x-axis to expand the field of view in the negative direction of the y-axis.

[0157] See also Fig.13 , Fig.13 Shows Fig.12 The schematic diagram of the optical path when the prism shown in (a) moves in the shaking direction. Fig.13 As shown in the figure, the incident light and the reflected light are symmetrical about the optical axis. When the prism rotates, the optical axis will also rotate. When the prism shakes, such as shaking the θ angle, the horizontal coordinate of the incident point on the prism surface will also change slightly, resulting in a change in the optical path, causing the reflection angle to change. The change in the reflection angle will cause the image to rotate about θ angle in the x-axis direction and Δ angle in the y-axis direction (crosstalk).

[0158] like Fig.13 As shown in the figure, when the prism is not moving, an incident light ray is reflected by the prism and the imaging point on the photosensitive element is P2. Fig.13 The shaking direction shown moves to Fig.13 When the prism' is in the position shown, the same incident light is reflected by the prism' and the image point on the photosensitive element is P2'. Here, before and after the prism moves, P2 and P2' are coplanar, but the P2P2' line segment is not parallel to the x-axis or the y-axis. In other words, the image point moves obliquely in the xy plane before and after the movement.

[0159] Furthermore, before the prism rotates, if the incident point of one of the two incident light rays is O1 (assuming it is the center of the prism), the corresponding imaging point on the photosensitive element is P1; the incident point of the other light ray is O2 (assuming it is not the center of the prism), and the corresponding imaging point on the photosensitive element is P2. When the prism rotates at an angle θ, the imaging point of the incident point O1 on the photosensitive element is P1', and the imaging point of the incident point O2 on the photosensitive element is P2'. Due to the change in the optical path, the image rotates less at O1 and more at O2, which will cause the translation amount corresponding to the change from P1 to P1' to be different from the translation amount corresponding to the change from P2 to P2'. In this way, Fig.14 As shown in the figure, when the prism moves in the shaking direction, the change from P1P2 to P1'P2' is close to the double effect caused by translation + rotation, and image rotation will be produced visually after imaging, and the greater the angle of the prism shaking, the more serious the image rotation problem. Here, it should be noted that image rotation includes rotation in three axes: x-axis, y-axis, and z-axis, which can be decomposed into translation + perspective effect of x-axis and y-axis, and rotation effect of z-axis.

[0160] For example, using Figure 6 When shooting with the telephoto camera shown in the figure, Fig.15 As shown in (a), the initial field of view of the telephoto camera is FOV0, which is a rectangle; when the rotating motor assembly is turned on and the prism is controlled to nod and the movement amplitude is the largest, the field of view of the telephoto camera can be changed upward to FOV1 or downward to FOV2; when the rotating motor assembly is turned on and the prism is controlled to shake its head and the movement amplitude is the largest, the field of view of the telephoto camera can be changed to the left to FOV3 or to the right to FOV4. When the rotating motor assembly is turned on and the prism is controlled to nod and shake its head (the order is not limited), the field of view of the telephoto camera can be changed to FOV5 at the upper right corner, to FOV6 at the lower right corner, to FOV7 at the lower left corner, or to FOV8 at the upper left corner, etc.

[0161] If the angle of rotation of the motor assembly to control the prism to nod and shake is small, the FOV that the telephoto camera can achieve is within the field of view range of FOV3 to FOV8. In addition, by superimposing all FOVs, the maximum FOV that the telephoto camera can achieve can be obtained, such as Fig.15 The maximum FOV is much larger than the initial field of view FOV0, that is, the telephoto camera can achieve an expanded field of view when the motor assembly controls the rotation of the prism.

[0162] Here, for Fig.15In (a), it should also be noted that when the rotating motor assembly controls the prism to nod, there will be no image rotation problem in the y-axis direction, and there will be no other distortion. Therefore, the shapes of FOV1 and FOV2 remain basically unchanged relative to FOV0, and they are still horizontally and vertically straight rectangles. When the rotating motor assembly controls the prism to shake its head, image rotation problems will occur. Therefore, FOV3 to FOV8 have changed relative to FOV0, which can be regarded as translation + rotation. For this reason, when imaging, the images corresponding to FOV3 to FOV8 captured by the telephoto camera will all be images with image rotation problems.

[0163] For example, Fig.16 Figure 2 shows two frames of images taken using a telephoto camera. Fig.16 As shown in (a), it is a playing card image taken before the rotating motor in the telephoto camera rotates, and the characters and graphics in the image can be displayed normally; Fig.16 As shown in (b), it is a playing card image taken after the rotary motor in the telephoto camera rotates, and the characters and graphics in the image have image rotation problems.

[0164] From the above description, it can be seen that after adding a rotating motor assembly to the telephoto camera, the rotating motor controls the rotation of the prism, including nodding and / or shaking the head. Although the shooting field of view can be expanded, the field angle of the telephoto camera may change before and after the rotation and the image rotation problem may be introduced due to the rotation of the prism; in this case, if the existing anti-shake processing method is used, the final captured image will be jittery and the clarity will be poor.

[0165] In view of this, an embodiment of the present application provides a shooting method and related equipment; in an embodiment of the present application, after the camera application is started, in response to user operation or in response to AI detection, the electronic device can trigger the first camera currently displayed to perform directional zoom on the target object; while performing the directional zoom, the movable camera rotates and points to the target object, and then the movable camera can acquire an image of the target object and display it, thereby, the method can achieve directional zoom for a target object at any position, and the image switches smoothly and the clarity is improved during the zooming process.

[0166] In addition, due to the rotation of the movable camera, FOV offsets will occur in the X-axis and Y-axis directions, as well as perspective in the X and Y directions and image rotation in the Z direction. After correction by the corresponding software module, the perspective and image rotation are corrected, leaving only image offsets in the X and Y directions. For these offsets, the anti-shake algorithm is used to adapt the new intrinsic parameter matrix, so that a better image anti-shake effect can be achieved.

[0167] Combine the following Figures 17 to 21The schematic flowchart of the shooting method provided in the embodiment of the present application is described in detail.

[0168] Fig.17 A schematic flow chart of a shooting method of an electronic device 100 is exemplarily shown. The electronic device includes a main camera and a movable camera, wherein the main camera may include a wide-angle camera, and the movable camera may be a telephoto camera, such as Figure 6 As shown; the working principle of the telephoto camera is as follows Figures 5 to 16 As shown; the method 200 includes S210 to S270; S210 to S270 are described in detail below.

[0169] It should be understood that a movable camera means that after the camera moves, the center of the lens and the imaging center of the image sensor are offset, such as Fig.18 and Fig.19 shown.

[0170] S210: Detecting an operation of starting a camera application, and starting the camera application in response to the operation.

[0171] Exemplarily, a user can instruct an electronic device to open a camera application by clicking on the icon of the "Camera" application; or, when the electronic device is in a lock screen state, the user can instruct the electronic device to open the camera application by sliding to the right on the display screen of the electronic device. Alternatively, the electronic device is in a lock screen state, and the lock screen interface includes an icon of a camera application, and the user instructs the electronic device to open the camera application by clicking on the icon of the camera application. Alternatively, when the electronic device is running other applications, the application has the permission to call the camera application; the user can instruct the electronic device to open the camera application by clicking on the corresponding control. For example, when the electronic device is running an instant messaging application, the user can instruct the electronic device to open the camera application by selecting the control of the camera function.

[0172] It should be understood that the above is an example of the operation of opening the camera application; the camera application can also be opened by instructing the electronic device through voice instructions or other operations; this application does not impose any limitation on this.

[0173] S220: Display a first image captured by a first camera, where the first image includes a target object.

[0174] Exemplarily, the first camera may be Figure 2 The wide-angle camera, the main camera, or other cameras with a larger field of view than the telephoto camera can be shown, and the first camera is an immovable camera. In the embodiment of the present application, the first camera can indicate the immovable main camera 1931.

[0175] Exemplarily, the first image may be an RGB image captured by the main camera, or in other words, the first image may be an RGB image after a Raw image captured by the main camera is processed by a series of camera algorithms.

[0176] Exemplarily, before displaying the first image, the first camera may also perform AI detection on the content in the first image to determine the target object included in the first image.

[0177] For example, Figure 4 The preview image shown in (a) is the first image captured and displayed by the first camera, and the third photographed object is the target object included in the first image. For another example, Figure 5 The video image shown in (a) is the first image captured and displayed by the first camera, and the fourth photographed object is the target object included in the first image.

[0178] Optionally, when the first image captured by the first camera is displayed, a detection frame corresponding to the target object may also be displayed.

[0179] For example, Figure 5 As shown in (a), when the first camera sends the first image for display, the detection frame determined after AI detection can be displayed together.

[0180] It should be understood that the first image may include one or more photographed objects, and the target object is one of the photographed objects.

[0181] It should also be understood that the type of the photographed object can be preset as needed. For example, when the type is set to a face, the target object can be used to indicate a face identified in the first image. The type of the photographed object can also include plants, animals, etc. The embodiments of the present application do not impose any restrictions on this.

[0182] S230: In response to a zoom operation on the first image, process the first image to acquire a second image, where the second image includes the target object.

[0183] The size of the target object included in the second image is larger than the size of the target object included in the first image.

[0184] Optionally, the zoom operation on the first image may include: a zoom operation on a target object in the first image.

[0185] It should be understood that the zoom operation on the first image may indicate an operation instruction of the user, or may also indicate a zoom operation instruction automatically triggered by the electronic device based on AI detection. The automatic zoom function can be set in the camera application, and when the function is turned on, the function of automatically zooming in on the target object detected by AI can be realized.

[0186] Exemplarily, when the zoom operation indicates an operation instruction triggered by the user, the zoom operation may include a two-finger reverse sliding operation, a click operation, a voice operation, an air gesture operation, etc. on the first image, and the embodiments of the present application do not impose any restrictions on this.

[0187] For example, Figure 5 (a) and Figure 5 As shown in (b), when the electronic device displays one or more photographed objects and a zoom control, the zoom operation may include a click operation on the target object in the first image and a slide operation on the zoom control; or, the zoom operation may also include a click operation on the target object in the first image and a two-finger reverse slide operation. It should be understood that when the zoom operation includes two operations, when executing, the user needs to first perform the click operation, and then perform the slide operation on the zoom control or the two-finger reverse slide operation on the target object. The zoom operation may also be other operations, or the zoom operation may also include three or more sub-operations, and the embodiments of the present application do not impose any restrictions on this.

[0188] Exemplarily, the first image may be an RGB image, and the corresponding second image may be an RGB image.

[0189] It should be understood that since the target object is magnified, the size of the target object in the second image is larger than that of the target object in the first image.

[0190] Optionally, the target object in the second image is centered in the second image.

[0191] Optionally, enlarging the target object in the first image includes: cropping a region where the target object in the first image is located, and then enlarging the region.

[0192] It should be understood that since the user has specified the object to be zoomed, the cropping method should be non-center cropping. Non-center cropping should not be centered on the center of the first image, but should be centered on the target object, and the area where the target object is located should be cropped. For example, when the target object is located in the upper left corner of the first image, in response to the zoom operation, the electronic device can crop the upper left corner area where the target object is located in the first image, and enlarge it according to the zoom multiple indicated by the zoom operation or the zoom multiple range described by the first camera to achieve directional zoom.

[0193] Optionally, the second image includes the entire content or a partial content of the target object. Figure 5As shown, assuming that the target object is the fourth subject, after zooming in, the second image may include the entire fourth subject, or the second image may also include local content of the fourth subject, such as the face of the fourth subject; when further zoomed in, the second image may also include a part of the face of the fourth subject.

[0194] Optionally, in order to smoothly enlarge the target object displayed by the electronic device during the zooming process, the above S230 can be executed multiple times in a loop, and the target object in the second image obtained each time is larger than the target object obtained last time. That is, in response to the zooming operation, the electronic device can perform multiple cropping and enlargement on the area where the target object is located in the first image, and obtain and display multiple second images, each of which includes the target object, and the target objects in the multiple second images gradually increase from the original size. The number of loop executions and the magnitude of the size change can be set as needed, and this application does not limit this. Among them, the original size indicates the size of the target object in the first image.

[0195] S240: In response to the zoom operation on the first image in S230, rotating the movable camera.

[0196] The zoom operation on the first image is the zoom operation in the above S230 , which will not be described in detail here.

[0197] The movable camera can be rotated to realize the flexible rotation of the camera in the electronic device, so that the field of view of the electronic device can be more flexible. Here, exemplarily, in response to the user's zoom operation, the movable camera can be triggered to rotate and point to the target object indicated by the user or the target object identified by AI detection, so that the movable camera can capture a relatively high-definition image of the target object after rotation.

[0198] It should be understood that S240 may be executed while S230 is being executed, or S240 may be executed after S230 is executed, and the embodiments of the present application do not impose any limitation on this.

[0199] S250: Acquire a third image captured by the movable camera after rotation, where the third image includes the target object.

[0200] Exemplarily, the third image captured by the movable camera may be a Raw image, a YUV image or an RGB image captured by the movable telephoto camera.

[0201] For example, Fig.18 The schematic diagram exemplarily shows the positional relationship between the lens and the sensor before and after the movable camera rotates; for example, before the movable camera rotates, the optical path diagram of the incident light is as follows: Fig.18As shown in (a) in FIG. 1 ; after the rotatable camera rotates, the optical path diagram of the incident light is as shown in FIG. Fig.18 As shown in (b) of Fig.18 (a) and Fig.18 It can be seen from (b) that before the movable camera rotates, the center point of the incident light received by the sensor is point 1; after the movable camera rotates, the center point of the incident light received by the sensor is point 2; it can be understood that after the movable camera rotates, the center point of the image collected by the sensor moves from point 1 to point 2, resulting in an offset.

[0202] For example, Fig.19 As shown in (a), before the movable camera rotates, the optical axis is optical axis 1, the image center point of the captured image is O3, and the resolution of the image is w*h; before the movable camera rotates, the optical axis rotates to a certain extent, the optical axis is optical axis 2, and the image center point of the captured image is O4.

[0203] Combination Fig.18 and Fig.19 It can be seen that before and after the movable camera rotates, an offset is generated between the center of the lens and the imaging center point; therefore, if the internal parameters before the movable camera rotates are still used for image stabilization after the movable camera rotates, the image stabilization effect is poor; in an embodiment of the present application, when executing the subsequent S250, the new internal parameters of the movable camera after rotation are used for EIS processing, thereby improving the image stabilization effect.

[0204] It should be understood that S240 and S250 may be executed while S230 is being executed, or S240 and S250 may be executed after S230 is executed, and the embodiments of the present application do not impose any limitation on this.

[0205] S260: Process the third image in combination with the second image to obtain a fourth image.

[0206] Here, the third image is processed in combination with the second image mainly to eliminate the image rotation and jitter problems caused by the rotation of the movable camera, and the difference between the third image captured by the rotated movable camera and the second image captured by the first camera.

[0207] It should be noted that if the first camera captures multiple second images during the rotation of the movable camera and the target object in the second image gradually becomes larger based on the zoom factor, then when processing the third image, it can be processed in combination with the second image frame that is closest in time. In this way, the first camera and the rotatable camera can be regarded as producing images for the same target object, the same zoom factor, and the same time.

[0208] Optionally, the following combination Fig. 20The implementation method of S260 is described in detail. S260 may include the following S261 to S263.

[0209] S261 . Perform intra-frame compensation and image rotation correction processing on the third image to obtain an image rotation corrected image.

[0210] Exemplarily, intra-frame compensation includes Rolling Shutter correction and OIS correction; wherein, before performing Rolling Shutter correction, it is necessary to first perform coordinate mapping on the posture of the gyro sensor according to the image rotation angle; after the coordinate mapping, Rolling Shutter correction is performed, and the starting exposure time of any row of pixels in the captured Raw image is aligned through Rolling Shutter correction.

[0211] For example, Fig.21 (a) shows the image coordinate system before the movable camera rotates; Fig.21 (b) in FIG. 1 is the image coordinate system after the movable camera rotates; however, for the gyroscope sensor, the displacement of the gyroscope sensor does not change before and after the movable camera rotates; therefore, it is necessary to calculate the image coordinate system according to the image rotation angle (for example, the image rotation angle is (0, 0, θ z )) Coordinate mapping of the gyro sensor’s position, such as Fig.21 As shown in (c) in .

[0212] For example, when collecting the first row of pixels in the Raw image, the data of the gyroscope sensor is (α x ,α y ,α z ); coordinate mapping is performed on the gyro sensor's posture vector according to the image rotation angle, and the obtained gyro sensor data is (α' x ,α' y ,α' z ); where (α' x ,α' y ,α' z ) and (α x ,α y ,α z ) can be seen from the following equation:

[0213]

[0214] For example, the data of the gyroscope sensor is (α' x ,α' y ,α' z), perform Rolling Shutter correction on the third image; in addition, it is also necessary to perform OIS correction and image rotation correction on the third image to obtain an image rotation corrected image.

[0215] For example, after the coordinate mapping of the pose vector of the gyroscope sensor is performed, the Rolling shutter calibration can be performed according to the existing method; Rolling shutter calibration matrix H = K*R(α′-α0)*K -1 .

[0216] in, is the pose vector of any row in the image; α0 is the pose vector of the center row of the image; R represents the Rodlides matrix corresponding to the three-dimensional rotation vector; is the camera internal parameter matrix; w, h represent the width and height of the image; in addition, the original OIS coordinates Ox, Oy are obtained; wherein Ox represents the pixel offset on the X axis during OIS correction; Oy represents the pixel offset on the Y axis during OIS correction; the original OIS coordinates Ox, Oy are mapped to the image coordinate system, and the obtained OIS x ,ois y ;

[0217] The mapping relationship is:

[0218]

[0219] Optionally, the above-mentioned Rolling Shutter correction, OIS correction and image rotation correction processing may refer to any existing algorithm, and this application does not make any limitation thereto.

[0220] S262: Perform anti-shake algorithm processing on the image rotation corrected image to obtain a processed anti-shake image.

[0221] Exemplarily, after the movable camera rotates, EIS processing may be performed on the third image and the image rotation correction image, thereby avoiding obvious jumps in the image.

[0222] For example, Fig. 22 As shown, the field of view angle before the movable camera rotates can be A; the field of view angle after the movable camera rotates is B; the expected field of view angle after the movable camera rotates is C; after processing by S260 (S261 to S263), the field of view angle corresponding to the processed image is D.

[0223] For example, Fig.23 As shown in Figure 2, before the movable camera rotates, the captured image is as follows: Fig.23 As shown in (a) in FIG. 1 ; after the movable camera rotates, the collected image is as follows Fig.23As shown in (b) in the figure; the image rotation correction image after image rotation correction is shown in Fig.23 As shown in (c) in FIG. 1 ; the image after EIS processing of the image rotation correction image is shown in FIG. Fig.23 As shown in (d) in .

[0224] Optionally, the following combination Fig.24 The implementation method of S262 is described in detail. S262 may include the following S2621 and S2622.

[0225] S2621. Calculate a new intrinsic parameter matrix of the movable camera according to the rotation amount of the movable camera.

[0226] For example, Fig.19 (a) and Fig.19 As shown in (b), assuming that the movable camera rotates R degrees, before the movable camera rotates, the original internal parameter represents the distance from each pixel point (for example, pixel point a1) in the image captured by the movable camera to the center point (O3) of the image; after rotating the angle R and correcting the image rotation, the new internal parameter represents the distance from each pixel point (for example, pixel point a1) in the image captured by the movable camera to the center point (O4) of the image; for example, assuming that the resolution of the image captured before the rotation is w*h, the coordinates of the original center point are (u0, v0), u0 = -w / 2, v0 = -h / 2, then the original internal parameter matrix K0 is:

[0227]

[0228] Assume that the initial center of the camera can be moved to rotate by an angle R, and the coordinates of the center point after rotation are (Cx, Cy). Then the new internal parameter matrix K1 after rotation is:

[0229] Among them, u1=-(w / 2+Cx), v1=-(h / 2+Cy).

[0230] It can be understood that, Fig.19 As shown in (b), the original center point of the image captured by the movable camera before the rotation is O3; after the rotation, the new center point captured by the movable camera is O4; therefore, if the image frames before and after the rotation are to be stabilized, the distance between the pixel point (e.g., point a1) in the image captured after the rotation and the original center point (e.g., point O3) needs to be determined.

[0231] S2622: Expand the pixel coordinates in the image rotation correction image into three-dimensional coordinates P2 according to the new intrinsic parameter matrix; and perform mapping according to the anti-shake matrix to obtain the three-dimensional pixel coordinates P1 of the pixel in the anti-shake image.

[0232] For example, the mapping relationship is as follows:

[0233] P1 = H * P2;

[0234] H=KRK -1 ;

[0235]

[0236]

[0237] Wherein, H represents the image transformation matrix; R represents the Rodlides matrix corresponding to the three-dimensional rotation vector; K represents the camera intrinsic parameter matrix of the optical image stabilization controller.

[0238] It should be understood that, in the present application, the image transformation matrix may also be referred to as an anti-shake matrix; and R may be a matrix obtained according to the rotation parameters of the gyroscope.

[0239] For example, if the movable camera does not rotate, the original intrinsic parameter matrix K0 can be obtained according to the center point coordinates (u0, v0); assuming that the coordinates of the pixel point in the image are (x, y), the coordinates of the pixel point in the three-dimensional coordinate system after the mobile camera rotates are (x', y', 1); the mapped pixel point coordinates are:

[0240] Among them, u0=-w / 2, v0=-h / 2.

[0241] For example, if the movable camera rotates, the new intrinsic parameter matrix K1 can be determined according to the coordinates (Cx, Cy) of the center point after the rotation; assuming that the coordinates of the pixel point in the image are (x, y), the coordinates of the pixel point in the three-dimensional coordinate system after the mobile camera rotates are (x', y', 1); the coordinates of the pixel point after mapping are:

[0242] Where, u1 = -(w / 2 + Cx), v1 = -(h / 2 + Cy); Cx, Cy are the offsets of the image center after the lens is rotated;

[0243] For example, usually

[0244] Optionally, since the movable camera rotates, the path planning and boundary constraints need to be adapted according to the coordinates of the center point of the movable camera after rotation during EIS processing.

[0245] It should be understood that the electronic image stabilization path constraint is satisfied during EIS processing to ensure that the pixels after electronic image stabilization processing are within the effective range; the electronic image stabilization path constraint refers to the path planning of the camera based on the gyroscope data when the electronic image stabilization algorithm is performed, and the goal of the path planning is to find a sufficiently stable camera posture curve to process multiple images.

[0246] For example, the vertex coordinates can be calculated using the updated new intrinsic parameter matrix K, and then passed to the solver to obtain new boundary constraints.

[0247] Optionally, any existing path planning and boundary constraint algorithm can be used, and this application does not impose any limitations on this; it should be noted that if the camera intrinsic parameter matrix is ​​used in the path planning and boundary constraint algorithm, since the movable camera can be rotated, the standard intrinsic parameter matrix is ​​not used and the new intrinsic parameter matrix after rotation is used.

[0248] Optionally, a new anti-shake H matrix may be constructed according to the new intrinsic parameter matrix and the rotation matrix, and each pixel coordinate may be mapped to obtain a processed anti-shake image.

[0249] It should be noted that the calculation process of the anti-shake matrix can refer to any existing method; the new intrinsic parameter matrix in S2621 is used when using the camera intrinsic parameter matrix.

[0250] S263: performing alignment processing on the anti-shake image in combination with the second image to obtain a fourth image.

[0251] Exemplarily, the alignment process includes at least one of cropping, rotating, and deforming. The electronic device can perform cropping, rotating, and deforming processes on the anti-shake image based on the second image, so that the processed fourth image is consistent with the second image. This application does not impose any restrictions on the order of these processes.

[0252] It should be understood that since the fourth image is aligned based on the second image, there will be no jump in content when the subsequent image displayed is switched to the fourth image, that is, smooth switching can be achieved when switching from the first camera to the rotatable camera.

[0253] It should also be understood that, since the fourth image is obtained based on the third image captured by the rotatable camera, the clarity of the fourth image is higher than that of the third image.

[0254] S270: Display and save the fourth image.

[0255] Exemplarily, when in the preview scene, the fourth image may be displayed. The preview shooting mode includes but is not limited to: shooting modes such as night scene preview mode, video preview mode, photo preview mode, and portrait preview mode.

[0256] Exemplarily, when in a video recording scene, the fourth image may be displayed and saved.

[0257] Exemplarily, in response to a zoom operation, when the zoom ratio has not reached the maximum zoom ratio of the first camera (for example, the main camera is 5x), the second image acquired in S230 continues to be displayed, while the acquired fourth image is saved; when the zoom ratio is equal to the maximum zoom ratio of the first camera, the acquired second image or fourth image can be displayed, which can be set specifically as needed; and after the zoom ratio is greater than the maximum zoom ratio of the first camera, the second image can be saved, while the acquired fourth image is displayed.

[0258] An embodiment of the present application provides a shooting method and related equipment; in an embodiment of the present application, after the camera application is started, in response to user operation or in response to AI detection, the electronic device can trigger the first camera currently displaying to perform directional zoom on the target object; while performing the directional zoom, the movable camera rotates and points to the target object, and then the movable camera can acquire an image of the target object and display it, thereby, the method can achieve directional zoom for a target object at any position, and during the zooming process, the images captured by the two cameras can be switched smoothly and the clarity can be improved.

[0259] In addition, due to the rotation of the movable camera, FOV offsets will occur in the X-axis and Y-axis directions, as well as perspective in the X and Y directions and image rotation in the Z direction. After correction by the corresponding software module, the perspective and image rotation are corrected, leaving only image offsets in the X and Y directions. For these offsets, the anti-shake algorithm can continue to be used to adapt the new intrinsic parameter matrix, thereby achieving a better image anti-shake effect.

[0260] Fig.25 A schematic flow chart of another shooting method of an electronic device 100 is exemplified. The first camera in the electronic device 100 is a main camera, the main camera may include a wide-angle camera or an ultra-wide-angle camera, and the movable camera may be a telephoto camera. The method 300 includes S301 to S310; S301 to S310 are described in detail below.

[0261] S301, open the camera application.

[0262] S302: Obtain a first image captured by a main camera and send it for display. The first image includes a target object.

[0263] Exemplarily, the image captured by the main camera may be an RGB image captured by the wide-angle camera; or, the image captured by the main camera may be an RGB image captured by the ultra-wide-angle camera.

[0264] For example, the target object is Figure 5 The fourth photographed subject shown in (a).

[0265] S303: Receive a zoom operation for the first image.

[0266] Exemplarily, the zoom operation may be a two-finger reverse sliding operation of the user on the target object in the first image.

[0267] Exemplarily, the zoom operation may be a click operation of the user on the target object in the first image and a sliding operation on the zoom control.

[0268] S304: In response to the zoom operation on the first image in S303, trigger directional zoom.

[0269] At this time, the camera application may send a first instruction to the telephoto camera in the hardware layer, where the first instruction is used to instruct the telephoto camera to start directional zooming.

[0270] S305: Process the first image to obtain a second image. The second image includes a target object, and a size of the target object in the second image is larger than a size of the target object in the first image.

[0271] Optionally, non-center cropping and enlargement are performed on the target object in the first image to obtain the second image.

[0272] S306: After executing S304, an initial image captured by the telephoto camera is obtained.

[0273] The initial image is an image captured when the telephoto camera is at an initial position before it is rotated.

[0274] S307: Determine the rotation amount of the telephoto camera according to the initial image.

[0275] Optionally, the above S307 may include: based on the initial image, an iterative method may be used in combination with an image rotation parameter to determine the rotation amount of the telephoto camera.

[0276] In the embodiment of the present application, the rotation amount of the telephoto camera indicates the rotation amount of the rotation motor in the telephoto camera.

[0277] Optionally, the above S307 may include the following S3071 to S3079.

[0278] S3071. Taking the image center point corresponding to the initial image as the origin, the coordinate system expanded to the surrounding areas is recorded as the Scan canvas coordinate system.

[0279] Optionally, camera calibration parameters may be obtained, and the Scan canvas coordinate system may be obtained according to the camera calibration parameters.

[0280] It should be noted that the camera calibration parameters may also be referred to as dual-camera calibration parameters or SAT parameters. Specifically, when the electronic device leaves the factory, the manufacturer may store the camera calibration parameters in the electronic device.

[0281] S3072, assuming that the coordinates of the center point of the image captured by the telephoto camera after rotation in the Scan canvas coordinate system are the target coordinates.

[0282] S3073. Substitute the optical parameters to obtain the target position vector S corresponding to the rotation motor of the telephoto camera.

[0283] S3074. According to the target position vector S, obtain the image rotation parameter (ie, the rotation vector) and the corresponding rotation matrix R.

[0284] It should be understood that a position vector file may be stored in an electronic device. The position vector pre-stored in the position vector file may include a rotation vector and a translation vector. The rotation vector includes vector components in three axes: the x-axis, the y-axis, and the z-axis, wherein the vector component in the z-axis is generally referred to as the image rotation angle.

[0285] Therefore, based on the target position vector, a table lookup can be performed in the position vector table to find the rotation vector r and the translation vector corresponding to the target position vector S, wherein the rotation vector r can be determined as the image rotation parameter corresponding to the target position vector.

[0286] Then, based on the rotation vector r, the rotation matrix R(r) corresponding to the target position vector S can be determined.

[0287] S3075. Substitute the assumed target coordinates into the rotation matrix R to obtain new target coordinates.

[0288] S3076. Determine the difference between the target coordinates and the new target coordinates, that is, the coordinate offset Δp.

[0289] S3077. Convert the coordinate offset Δp into the corresponding position vector deviation Δs.

[0290] S3078. Superimpose the position vector deviation Δs onto the original target position vector S, and set Δs+S to be the target position vector corresponding to the rotation motor in the telephoto camera.

[0291] S3079, repeatedly executing the above S3074-S3078 to determine a new coordinate offset Δp, until the new coordinate offset Δp is less than a preset threshold.

[0292] Therefore, based on the initial image, using the above-mentioned iterative method, combined with the image rotation parameters, the corresponding relationship between the target position vector S of the rotary motor and the coordinate offset Δp of the center point of the image can be determined; based on this corresponding relationship, it can be deduced that if the telephoto camera wants to capture an image with the same center point coordinates as the second image captured by the main camera, the position vector that the rotary motor needs to reach can be deduced; according to the position vector that the rotary motor needs to reach, the rotation amount of the rotary motor can be obtained. It should be understood that there is a one-to-one correspondence between different position vectors of the rotary motor and the rotation amount (or the angle of the prism), therefore, based on this corresponding relationship, the rotation amount corresponding to different position vectors can be obtained.

[0293] S308: The telephoto camera rotates based on the rotation amount.

[0294] Here, when the telephoto camera rotates based on the rotation amount, the rotation amount can be synchronized to related processing modules such as intra-frame compensation, image rotation correction processing, and anti-shake algorithm processing to facilitate subsequent calculations.

[0295] It can be understood that the telephoto camera can point to the target object after rotating based on the rotation amount.

[0296] S309: Acquire a third image captured by the rotating telephoto camera, where the third image includes the target object.

[0297] Here, the rotation of the telephoto camera may be based on the stabilization of the rotation motor, that is, the acquisition time of the third image may be longer or later than the actual rotation time of the rotation motor.

[0298] S310, combining the second image, processing the third image to obtain a fourth image and sending it for display.

[0299] Here, for the description of S310 , reference may be made to the above description of S260 , which will not be repeated here.

[0300] Optionally, after S305, S305 may be repeatedly performed multiple times, or S305 may be performed for a preset time period. The size of the target object in the second image is relatively larger than the size of the target object in the second image acquired last time.

[0301] Here, the number of executions may be determined based on the duration of acquiring the third image. For example, when the third image captured by the rotated telephoto camera is acquired, the operation of acquiring the second image is stopped.

[0302] Exemplarily, when the third image captured by the rotated telephoto camera is acquired, the camera application may send a second instruction to the main camera of the hardware layer, where the first instruction is used to instruct the main camera to stop acquiring the second image.

[0303] Here, the execution time can be preset based on experience. For example, based on experience, it can be determined that the time it takes for the telephoto camera to rotate to stability is approximately within 150ms, so the time can be preset to 150ms. Thus, when the directional zoom is triggered, the main camera can repeat the execution of the second image, and after waiting for 150ms, it will automatically stop acquiring the second image.

[0304] Optionally, when the third image is processed in combination with the second image, the second image may be the second image captured by the main camera at the closest time to the third image or the anti-shake image, or the second image may be the second image captured by the main camera for the last time after a preset time has passed. The preset time is longer than the rotation time of the telephoto camera.

[0305] It should also be understood that before the fourth image is obtained, the second image obtained by the main camera is sent for display; after the fourth image is obtained, the fourth image obtained by the telephoto camera is switched for display.

[0306] For example, Fig.26 A schematic sequence diagram of sending and displaying is shown as an example. Fig.26 As shown in (a) of FIG. 1 , the electronic device may first display the first image sent by the main camera. For example, based on AI detection, if a face is detected in the first image, a detection frame corresponding to the face may be displayed, such as Fig.26 At this time, the electronic device can automatically trigger the zoom operation, and in response to the zoom operation, trigger the telephoto camera to rotate; before rotating, the telephoto camera can first obtain an initial image based on the original position, such as Fig.26 As shown in (e), since the size and center point of the field of view angle range corresponding to the telephoto camera and the main camera are different, the initial image obtained by the telephoto camera at this time may be quite different from the first image obtained by the main camera.

[0307] like Fig.26 As shown in (c), the main camera obtains the second image multiple times based on the first image and sends it for display, and the face size in the second image obtained each time is equivalent to the face size in the second image obtained last time, which is enlarged.

[0308] After the telephoto camera has finished rotating based on the determined rotation amount, the telephoto camera can obtain a third image and combine it with the second image of the n+sth frame most recently obtained by the main camera (such as Fig.26 Then, the fourth image is obtained and sent for display, as shown in (d) in FIG. Fig.26 Here, the second image of the n+sth frame most recently acquired by the main camera will no longer be sent for display.

[0309] The above is a shooting method provided by this application. The specific execution order can be adjusted and changed as needed, and this application does not limit this. During the zooming process, in response to the zoom operation of any position in the first image sent by the main camera, the telephoto camera can rotate and send a fourth image corresponding to the area to achieve directional zoom at any non-center position.

[0310] It should be noted that if after zooming in, in response to another zoom operation of the user on the fourth image (such as pinching with two fingers), the telephoto camera can be rotated back to the initial position during the zooming out process, and the image displayed can be switched from the telephoto camera to the main camera; in the process of continuing to zoom out, the main camera can perform a smooth ROI transition and obtain multiple second images for display. Here, the target object included in each second image will be smaller than the target object included in the previous second image, and the zoom factor is adjusted until it returns to the initial state of the main camera (for example, 1x, the target object is located at the center point of the image).

[0311] Fig. 27 The following is a schematic flow chart showing another shooting method of the electronic device 100. The method 400 includes S401 to S418; S401 to S418 are described in detail below.

[0312] S401: Acquire an image captured by a main camera.

[0313] Exemplarily, the image captured by the main camera may be a Raw image captured by the wide-angle camera; or, the image captured by the main camera may be a Raw image captured by the ultra-wide-angle camera.

[0314] S402: First front-end processing.

[0315] Optionally, the first front-end processing may include a related algorithm for converting a Raw image captured by the main camera into a YUV image; this application does not impose any limitation on the algorithm.

[0316] Exemplarily, the first front-end processing may refer to an image processing algorithm for converting a Raw image to a YUV image executed in the ISP. The first front-end processing may also include subsequent operations such as non-center cropping and magnification performed in response to a zoom operation.

[0317] S403: First backend processing.

[0318] Exemplarily, the first back-end processing includes, but is not limited to: brightening processing, denoising processing, saturation adjustment processing, cropping processing or deformation processing.

[0319] S404: Acquire images captured by the movable camera.

[0320] For example, the movable camera may be a movable telephoto camera; the structure of the telephoto camera may be referred to in Figure 6 For a description of the telephoto camera, see Figures 7 to 16 Related description.

[0321] For example, the image captured by the movable camera may be a Raw image captured by a movable telephoto camera.

[0322] Optionally, S404 and S401 may be executed simultaneously, or S404 and S401 may be executed successively; when S404 is executed, the movable camera is in an initial position; it can be understood that when S404 is executed, the movable camera does not rotate.

[0323] S405: Second front-end processing.

[0324] Optionally, the second front-end processing may include a related algorithm for converting a Raw image captured by the movable telephoto camera into a YUV image; this application does not impose any limitation on the algorithm.

[0325] Exemplarily, the second front-end processing may refer to an image processing algorithm for converting a Raw image into a YUV image executed in the ISP.

[0326] After executing S405, the processed image is subjected to a first back-end processing; the implementation method of the first back-end processing may refer to the relevant description of S403.

[0327] S406: Smoothing process (or alignment process).

[0328] Exemplarily, the image stream captured by the main camera and the image stream captured by the movable camera are smoothed; it can be understood that the input smoothing data is two image streams (for example, the image stream captured by the main camera and the data stream captured by the movable camera), and the image stream after smoothing is one image stream (for example, the image stream captured by the main camera, or the data stream captured by the movable camera).

[0329] Optionally, S406 may generate a correction matrix 4, where the correction matrix 4 is used to make image frames in one image stream transition smoothly after the camera is switched.

[0330] In the embodiments of the present application, smoothing processing can be used to avoid obvious jumps in the image when switching cameras.

[0331] S407: Image feature point detection.

[0332] Exemplarily, image feature point detection can be performed on image frames in an image stream output after smoothing to extract feature points in the image frames; by calculating the relative position difference between each feature point in the image stream, the overall displacement between image frames in the image stream is obtained, thereby cropping different areas of each frame of the image to achieve the effect of motion compensation.

[0333] Optionally, the image feature point detection may adopt any existing image feature point detection algorithm, and this application does not impose any limitation on this.

[0334] S408: Obtain rotation parameters of the gyro sensor and displacement parameters of the OIS.

[0335] Exemplarily, the rotation parameters of the gyroscope are used to characterize the movement of the electronic device in the captured image; the displacement parameters of the OIS can be used to represent the data of the OIS controller controlling the motor that drives the OIS and moving the lens or image sensor based on the jitter data detected by the motion sensor when the optical image stabilization is turned on during image capture.

[0336] S409, electronic image stabilization processing.

[0337] Exemplarily, electronic image stabilization processing is used to correct the motion between image frames in the image stream to avoid large image jumps between image frames and generate a stable image stream; during the electronic image stabilization processing, a correction matrix 2 can be generated based on the rotation parameters of the gyroscope and the displacement parameters of the OIS; the correction matrix 2 includes: a matrix for rolling shutter correction and a matrix for OIS correction.

[0338] Among them, the compensation processing includes compensation for the start time of exposure and / or OIS displacement compensation; wherein, the compensation for the start time of exposure is used to align the start exposure time of each row in a frame of image when capturing the frame of image; OIS displacement compensation is used to compensate for the OIS displacement in the optical image stabilization processing; optionally, the compensation for the start time of exposure can be achieved through Rolling Shutter correction, and the OIS displacement compensation can be achieved through OIS correction.

[0339] Optionally, after the correction matrix 2 is generated, OIS correction and Rolling shutter correction may be implemented based on the correction matrix 2 for one frame of image when S303 is executed.

[0340] Exemplarily, the electronic device generates a correction matrix 3 according to a result of detecting feature points in an image stream; the correction matrix 3 may be an EIS correction matrix for correcting motion between image frames.

[0341] S410: Second backend processing.

[0342] Exemplarily, the second back-end processing includes anti-shake processing; for example, anti-shake processing may be performed on image frames in the image stream according to the correction matrix 3 to obtain a processed image.

[0343] S411. Display the processed image.

[0344] Exemplarily, the processed image is an image after performing OIS correction, Rollingshutter correction and electronic image stabilization on the image with optical image stabilization turned on.

[0345] S412: third front-end processing.

[0346] Optionally, the relevant description of the third front-end processing can refer to the relevant description of S402, which will not be repeated here.

[0347] Optionally, the difference between S412, S405 and S402 lies in the different input images; wherein, the input image of S402 is a Raw image of the first resolution (for example, 1080P) captured by the main camera; the input image of S405 is a Raw image of the first resolution captured by the movable camera; the input image of S412 is a Raw image of the second resolution (for example, 12M) captured by the movable camera.

[0348] S413: First image processing.

[0349] Exemplarily, the first image processing includes brightening processing and denoising processing; the brightening processing and denoising processing may adopt any existing algorithm, and the present application does not impose any limitation on this.

[0350] S414: third backend processing.

[0351] Optionally, the relevant description of the third backend processing can refer to the relevant description of S403, which will not be repeated here.

[0352] It should be noted that, when executing S414, image processing may be performed according to correction matrix 1, correction matrix 2, and correction matrix 3.

[0353] S415: Store the processed image.

[0354] S416: A zoom operation is detected.

[0355] The zoom operation may refer to the description in the above S230.

[0356] Optionally, the electronic device can display an image captured by a main camera, or an image captured by a movable camera, and the electronic device detects a click operation on the displayed image.

[0357] S417, controlling the movable camera to rotate.

[0358] Exemplarily, the field of view of the rotated movable camera includes the photographic object clicked by the click operation included in the zoom operation or the photographic object recognized by AI detection.

[0359] Optionally, after the movable camera rotates, the coordinate system of the movable camera may be transformed from coordinate system 1 to coordinate system 2, and calibration parameters between coordinate system 1 and coordinate system 2 may be acquired in S409.

[0360] S418. Obtain an image rotation correction matrix according to the rotation amount of the movable camera.

[0361] Optionally, an image rotation correction matrix, ie, correction matrix 2, may be obtained according to the rotation amount of the movable camera and a pre-constructed position vector.

[0362] It can be understood that the movement of the prism is driven by the rotary motor, and there is a corresponding relationship between the position of the rotary motor and the angle of the prism movement. The rotary motor is driven by a motor, and when the motor drives the rotary motor to drive the prism to move, it will output a digital signal, that is, each digital signal can identify a position of the rotary motor. For the sake of convenience, this embodiment refers to the digital signal output by the motor as a scan code (using a New scan code to identify the current position of the rotary motor). Since there is a corresponding relationship between the position of the rotary motor and the angle of the prism movement, and the scan code has a corresponding relationship with the position of the rotary motor, there is a corresponding relationship between the scan code and the angle of the prism movement.

[0363] Accordingly, after determining the rotation vector and translation vector of each shooting point relative to the initial position, the scan code corresponding to each shooting point and the determined rotation vector and translation vector of the shooting point relative to the initial position can be saved, thereby obtaining the position vector table mentioned in each embodiment of the present application, and its form can be, for example, as shown in Table 1.

[0364] Table 1

[0365] Shooting point scan code Rotation Vector Translation Vector Point 1 Code1 R1 T1 Point2 Code2 R2 T2 Point3 Code3 R3 T3 Point 4 Code4 R4 T4 Point5 Code5 R5 T5 ... ... ... ... Point24 Code24 R24 T24

[0366] It should be understood that Table 1 is only an example listed for better understanding of the technical solution of this embodiment, and is not the only limitation to this embodiment.

[0367] It should be noted that after the movable camera rotates, the field of view angle of the image captured by the movable camera will change, and the correction matrix used to process the image captured by the movable camera will change due to the rotation of the movable camera; for example, after the movable camera rotates, part or all of correction matrix 1, correction matrix 2, correction matrix 3 and correction matrix 4 may change.

[0368] For example, Fig.28 As shown, three horizontal lines, such as h1, h2 and h3, and three vertical lines, such as v1, v2 and v3, evenly divide the movement range of the rotary motor into four grids and nine shooting points (such as Point 0 to Point 8).

[0369] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0370] Fig.29 It is a schematic flow chart of an image stabilization method provided in an embodiment of the present application; the method 500 includes S510 to S540; S510 to S540 are described in detail below.

[0371] S510: Acquire a third image and a first offset captured by a movable camera.

[0372] The first offset is used to represent the offset between the center of the optical lens of the movable camera and the center point of the image. Fig.18 The offset between point 1 and point 2 shown in (b).

[0373] S520: Obtain a first camera intrinsic parameter based on the first offset.

[0374] For example, the implementation of S520 can refer to Fig.24 The relevant description of S2621 will not be repeated here.

[0375] S530: Obtain a first correction matrix based on the first camera intrinsic parameter.

[0376] Optionally, in an implementation manner, obtaining a first correction matrix based on a first camera intrinsic parameter includes:

[0377] Based on the first camera intrinsic parameter, a camera intrinsic parameter matrix is ​​obtained;

[0378] A first correction matrix is ​​obtained based on the matrix corresponding to the camera intrinsic parameter matrix and the three-dimensional rotation vector.

[0379] For example, the implementation of S530 can refer to Fig.24 The relevant description of S2622 in ; wherein, the first correction matrix can be the image transformation matrix H in S2622; it will not be repeated here.

[0380] S540: Obtain an image after electronic image stabilization processing based on the first correction matrix and the third image.

[0381] Optionally, in one implementation, obtaining an image after electronic image stabilization processing based on the first correction matrix and the third image includes:

[0382] The third image is subjected to a first correction process, a second correction process and a third correction process to obtain a processed third image; wherein the first correction process is used to align the exposure moments of different rows of pixels for acquiring the third image, the second correction process is an optical image stabilization process, and the third correction is used to perform an image affine transformation process on the third image; and the pixels of the processed third image are multiplied based on the first correction matrix to obtain an image subjected to electronic image stabilization process.

[0383] Exemplarily, the coordinates of the pixel point in the processed third image are expanded into three-dimensional coordinates P2; and mapping is performed according to the anti-shake matrix to obtain the three-dimensional pixel coordinates P1 of the pixel point in the anti-shake image.

[0384] For example, the mapping relationship is as follows:

[0385] P1=HP2;

[0386] H=KRK -1 ;

[0387]

[0388]

[0389] Wherein, H represents the image transformation matrix; R represents the Rodlides matrix corresponding to the three-dimensional rotation vector; K represents the camera intrinsic parameter matrix of the optical image stabilization controller.

[0390] It should be understood that, in the present application, the image transformation matrix may also be referred to as an anti-shake matrix; and R may be a matrix obtained according to the rotation parameters of the gyroscope.

[0391] For example, the above implementation can be seen in Fig.13 The related description of S230 and S240 in the figure; wherein, the first correction processing may be the Rolling Shutter correction, the second correction processing may be the OIS correction, and the third correction processing may be the image rotation correction processing, which will not be repeated here.

[0392] Optionally, in an implementation manner, before performing the first correction process, the method further includes:

[0393] A first pose of the gyro sensor is obtained, where the first pose is the pose of the gyro sensor when the third image is collected; the first pose of the gyro sensor is mapped according to the image rotation angle of the third image to obtain a second pose of the gyro sensor.

[0394] For example, the above implementation can be seen in Fig. 20 The relevant description of S261 will not be repeated here.

[0395] Optionally, in one implementation, the method further includes:

[0396] According to the internal parameters of the first camera, the path planning and boundary constraint algorithm of the electronic image stabilization processing is determined.

[0397] Optionally, since the movable camera rotates, the path planning and boundary constraints need to be adapted according to the coordinates of the center point of the movable camera after rotation during EIS processing.

[0398] It should be understood that the electronic image stabilization path constraint is satisfied during EIS processing to ensure that the pixels after electronic image stabilization processing are within the effective range; the electronic image stabilization path constraint refers to the path planning of the camera based on the gyroscope data when the electronic image stabilization algorithm is performed, and the goal of the path planning is to find a sufficiently stable camera posture curve to process multiple images.

[0399] For example, the vertex coordinates can be calculated using the updated new intrinsic parameter matrix K1, and then passed to the solver to obtain new boundary constraints.

[0400] Optionally, any existing path planning and boundary constraint algorithm can be used, and this application does not impose any limitations on this; it should be noted that if the camera intrinsic parameter matrix is ​​used in the path planning and boundary constraint algorithm, since the movable camera can be rotated, the standard intrinsic parameter matrix is ​​not used and the new intrinsic parameter matrix after rotation is used.

[0401] Optionally, in an implementation, the first offset is related to the rotation of the movable camera, and further includes:

[0402] If the movable camera rotates, the first offset is updated and the updated first offset is obtained.

[0403] Optionally, in an implementation manner, obtaining a first correction matrix based on a first camera intrinsic parameter includes:

[0404] A camera intrinsic parameter matrix is ​​obtained based on the first camera intrinsic parameter; and a first correction matrix is ​​obtained based on a matrix corresponding to the camera intrinsic parameter matrix and the three-dimensional rotation vector.

[0405] Optionally, in one implementation, the movable camera includes an optical lens and a motor, the motor includes a rotating motor and an anti-shake motor, the rotating motor rotates on the X-axis and the Y-axis to rotate the movable camera, and the anti-shake motor is used to perform translational motion on the X-axis and / or translational motion on the Y-axis to perform optical image stabilization.

[0406] Optionally, in one implementation, the movable camera includes a movable telephoto camera.

[0407] In an embodiment of the present application, the electronic device includes a movable camera. By rotating the movable camera, the field of view angle of the telephoto camera can be overcome to a certain extent to meet the needs of different shooting scenes. When the camera in the electronic device is flexibly rotated, in the scheme of the present application, a first offset of the movable camera can be obtained; according to the first offset of the movable camera, a first camera parameter is obtained, thereby obtaining a first correction matrix for electronic image stabilization; because the movement of the movable camera cannot be detected by the gyroscope sensor when the movable camera moves, in the present application, the first offset can be obtained due to the movement of the movable camera; according to the first offset, a new intrinsic parameter of the camera can be calculated; and a correction matrix for electronic image stabilization is obtained through the new intrinsic parameter; therefore, through the scheme of the present application, the image anti-shake effect can be improved when the electronic device is flexibly rotated.

[0408] Combination of the above Figures 1 to 29 The shooting method provided by the embodiment of the present application is described in detail; Fig.30 and Fig.31 The device embodiments of the present application are described in detail. It should be understood that the device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0409] Fig.30 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 600 includes a processing module 610 and a display module 620; the electronic device 600 also includes a movable camera.

[0410] The processing module 610 is used to: start the camera application; the display module 620 is used to display the first image captured by the first camera; the processing module 610 is used to: in response to the zoom operation on the first image, process the first image to obtain the second image; in response to the zoom operation, rotate the movable camera; obtain the third image captured after the movable camera rotates; combine the second image, process the third image to obtain the fourth image; the first image, the second image, the third image and the fourth image all include the target object; save the fourth image. The display module 620 is also used to display the fourth image.

[0411] Optionally, as an embodiment, the processing module 610 is specifically used to: perform at least non-center cropping and enlargement on the first image to obtain a second image; the size of the target object in the second image is larger than the size of the target object in the first image.

[0412] Optionally, as an embodiment, the zoom operation includes a two-finger reverse sliding operation, a sliding operation on a displayed zoom control, or a click operation on a target object in the first image and a sliding operation on the zoom control.

[0413] Optionally, as an embodiment, the processing module 610 is also used to: perform intra-frame compensation and image rotation correction processing on the third image to obtain an image rotation corrected image; perform anti-shake algorithm processing on the image rotation corrected image to obtain an anti-shake image; and align the anti-shake image in combination with the second image to obtain a fourth image.

[0414] Optionally, as an embodiment, the processing module 610 is also used to: process the first image multiple times to obtain multiple second images; or, process the first image multiple times within a preset time length to obtain multiple second images, and the preset time length is greater than the rotation time length of the movable camera; wherein the size of the target object in each second image obtained is larger than the size of the target object in the second image obtained last time.

[0415] Optionally, as an embodiment, the processing module 610 is also used to: align the anti-shake image in combination with the second image acquired at the closest time to the time of acquiring the third image, or in combination with the second image acquired for the last time when a preset time length is reached, to obtain a fourth image.

[0416] Optionally, as an embodiment, the field of view angle range of the first camera is larger than the field of view angle range of the movable camera, and the clarity of the first image is smaller than the clarity of the fourth image.

[0417] Optionally, as an embodiment, the movable camera includes an optical lens and a motor, the motor includes a rotating motor and an anti-shake motor, the rotating motor rotates on the X-axis and the Y-axis to rotate the movable camera, and the anti-shake motor is used to perform translational motion on the X-axis and / or translational motion on the Y-axis to perform optical image stabilization.

[0418] Optionally, as an embodiment, the first camera includes a wide-angle camera or an ultra-wide-angle camera, and the movable camera includes a movable telephoto camera.

[0419] It should be noted that the electronic device 600 is implemented in the form of a functional unit. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0420] For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0421] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0422] Fig.31 A structural schematic diagram of an electronic device provided by the present application is shown. Fig.31 The dotted line in the figure indicates that the unit or the module is optional. The electronic device 700 can be used to implement the method described in the above method embodiment.

[0423] The electronic device 700 includes one or more processors 701, which can support the electronic device 700 to implement the shooting method in the method embodiment. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0424] The processor 701 may be used to control the electronic device 700, execute software programs, and process data of the software programs. The electronic device 700 may also include a communication unit 705 to implement input (reception) and output (transmission) of signals.

[0425] For example, the electronic device 700 may be a chip, the communication unit 705 may be an input and / or output circuit of the chip, or the communication unit 705 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0426] For another example, the electronic device 700 may be a terminal device, and the communication unit 705 may be a transceiver of the terminal device, or the communication unit 705 may be a transceiver circuit of the terminal device.

[0427] The electronic device 700 may include one or more memories 702 on which a program 704 is stored. The program 704 can be executed by the processor 701 to generate instructions 703, so that the processor 701 executes the method described in the above method embodiment according to the instructions 703.

[0428] Optionally, data may be stored in the memory 702. Optionally, the processor 701 may read data stored in the memory 702. The data may be stored at the same storage address as the program 704, or may be stored at a different storage address than the program 704.

[0429] The processor 701 and the memory 702 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0430] Exemplarily, the memory 702 can be used to store the related program 704 of the shooting method provided in the embodiment of the present application, and the processor 701 can be used to call the related program 704 of the shooting method stored in the memory 702 during video processing to execute the shooting method of the embodiment of the present application; for example, start the camera application; display the first image captured by the first camera; in response to the zoom operation on the first image, process the first image to obtain the second image; in response to the zoom operation, rotate the movable camera; obtain the third image captured after the movable camera is rotated; in combination with the second image, process the third image to obtain the fourth image; the first image, the second image, the third image and the fourth image all include the target object; display and save the fourth image.

[0431] The present application also provides a computer program product, which, when executed by the processor 701, implements the method described in any method embodiment of the present application.

[0432] The computer program product may be stored in the memory 702 , for example, a program 704 , which is converted into an executable target file that can be executed by the processor 701 after preprocessing, compiling, assembling, and linking.

[0433] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0434] The computer-readable storage medium is, for example, a memory 702. The memory 702 may be a volatile memory or a nonvolatile memory, or the memory 702 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0435] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0436] In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0437] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0438] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0439] In short, the above is only a preferred embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. A shooting method, characterized in that: Applied to an electronic device, the electronic device comprises a first camera and a movable camera, the movable camera comprises a rotation motor, a lens and a prism; the method comprises: Open the Camera app; Displaying a first image captured by the first camera; In response to a zoom operation on the first image, processing the first image to acquire a second image; In response to the zoom operation, the movable camera is rotated; the rotating the movable camera comprises: the rotating motor at least drives the prism to rotate with the main optical axis of the lens as the rotation axis; Acquire a third image captured by the movable camera after rotation; In combination with the second image, performing at least image rotation correction processing on the third image to obtain a fourth image; the first image, the second image, the third image and the fourth image all include a target object; The fourth image is displayed and saved.

2. The shooting method according to claim 1, characterized in that: Processing the first image to obtain a second image includes: performing at least non-center cropping and enlarging on the first image to obtain the second image; The size of the target object in the second image is larger than the size of the target object in the first image.

3. The shooting method according to claim 1 or 2, characterized in that: The zoom operation includes a two-finger reverse sliding operation, a sliding operation on a displayed zoom control, or includes a click operation on a target object in the first image and a sliding operation on the zoom control.

4. The shooting method according to claim 3, characterized in that: In combination with the second image, performing at least image rotation correction processing on the third image to obtain a fourth image includes: Performing intra-frame compensation and image rotation correction processing on the third image to obtain an image rotation corrected image; Performing anti-shake algorithm processing on the image rotation correction image to obtain an anti-shake image; Combined with the second image, the anti-shake image is aligned to obtain the fourth image.

5. The shooting method according to claim 4, characterized in that: Processing the first image to obtain a second image includes: processing the first image multiple times to obtain multiple second images; or processing the first image multiple times within a preset time length to obtain multiple second images, wherein the preset time length is greater than the time length of the movable camera rotating; The size of the target object in the second image acquired each time is larger than the size of the target object in the second image acquired last time.

6. The shooting method according to claim 5, characterized in that: Combining the second image, aligning the anti-shake image to obtain the fourth image, including: The anti-shake image is aligned with the second image that is acquired closest to the time of acquiring the third image, or with the second image that is acquired for the last time when a preset time length is reached, to obtain the fourth image.

7. The shooting method according to any one of claims 1, 2, 4 to 6, characterized in that: The field of view angle range of the first camera is greater than the field of view angle range of the movable camera, and the clarity of the first image is less than the clarity of the fourth image.

8. The shooting method according to claim 7, characterized in that: The movable camera also includes an anti-shake motor, which is used to drive the lens to perform translational movement in the X-axis and / or the Y-axis direction to perform optical image stabilization processing. The main optical axis of the lens is the Y-axis, and the X-axis and the Y-axis are perpendicular to each other.

9. The shooting method according to claim 8, characterized in that: The first camera includes a wide-angle camera or an ultra-wide-angle camera, and the movable camera includes a movable telephoto camera.

10. An electronic device, characterized in that: The electronic device includes: one or more processors, a memory, a first camera and a movable camera; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the shooting method as described in any one of claims 1 to 9.

11. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the shooting method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the photographing method according to any one of claims 1 to 9.

Images