Shooting method, electronic device and readable medium

Abstract

The present application provides a shooting method, an electronic device and a readable medium, the shooting method is applied to an electronic device, the electronic device includes a camera module with a rotatable reflective prism, the method includes: receiving a first operation of a user; in response to the first operation, starting a camera application and presenting a first interface, the first interface being a camera preview interface or a camera shooting interface, the first interface displaying a first object of a first size; receiving a second operation of the user on the first interface for the first object; in response to the second operation, determining a target position of the reflective prism; driving the reflective prism to rotate to the target position; after the reflective prism rotates to the target position, controlling the camera module to shoot an image or video including the first object; wherein the first object in the image shot by the camera module is of a second size, the second size being larger than the first size, so that the user can specify an object to be tracked and shot, and the electronic device automatically shoots a high-definition image or video.

Description

Technical Field

[0001] The present application relates to the field of photographing technology, and in particular to a photographing method, an electronic device, a computer program product, and a computer-readable storage medium. Background Art

[0002] When an electronic device tracks a subject, such as tracking a moving object at a long distance, the clarity of the captured image or video is usually not high, which affects the shooting effect. Summary of the invention

[0003] The present application provides a shooting method, an electronic device, a computer program product and a computer-readable storage medium, which can enable a user to specify an object to be tracked and photographed, and the electronic device to automatically shoot a high-definition image or video.

[0004] In order to achieve the above objectives, this application provides the following technical solutions:

[0005] In a first aspect, the present application provides a shooting method, which is applied to an electronic device, the electronic device includes a camera module, the camera module includes a rotatable reflective prism, and the shooting method includes: receiving a first operation of a user, responding to the first operation, starting a camera application and presenting a first interface, the first interface is a camera preview interface of a photo mode, the first interface displays a first object, and the first object is a first size; receiving a second operation of the user on the first interface for the first object; responding to the second operation, determining a target position of the reflective prism; after driving the reflective prism to rotate to the target position, controlling the camera module to shoot an image, the image shot by the camera module also displays the first object, the first object is a second size, and the second size is larger than the first size. In some embodiments, after driving the reflective prism to rotate to the target position, the user inputs a shooting operation on the first interface, and the electronic device responds to the user's shooting operation and controls the camera module to shoot an image.

[0006] It can be seen from the above content that: in the photo mode, the camera preview interface displayed by the electronic device shows the first object, the user performs the second operation on the first object, and the electronic device automatically responds to the second operation, determines the target position of the reflective prism, and drives the reflective prism to rotate to the target position, and then controls the camera module to capture an image, and the size of the first object in the image is larger than the size of the first object initially displayed on the camera preview interface. Since the image is captured by the camera module rather than cropped from other images, the clarity is high, so the user specifies the object to be tracked and captured, and the electronic device automatically captures a high-definition image.

[0007] Furthermore, the user only inputs the second operation on the first object, the first object is also displayed in the image captured by the camera module, and the size of the first object is larger than the size of the first object initially displayed on the camera preview interface. The user does not need to input other operations, and the problem of blurred images captured due to tracking of the user's mobile device can be avoided.

[0008] In a second aspect, the present application provides a shooting method, which is applied to an electronic device, the electronic device includes a camera module, the camera module includes a rotatable reflective prism, and the shooting method includes: receiving a first operation of a user, responding to the first operation, starting a camera application and presenting a first interface, the first interface is a camera preview interface or a camera shooting interface in a video recording mode, and the first interface displays a first object; receiving a second operation of the user on the first interface for the first object; responding to the second operation, determining a target position of the reflective prism; after driving the reflective prism to rotate to the target position, controlling the camera module to shoot a video. In some embodiments, the first interface is a camera preview interface in a video recording mode, after driving the reflective prism to rotate to the target position, the user inputs a shooting operation on the first interface, and the electronic device responds to the user's shooting operation and controls the camera module to shoot a video.

[0009] It can be seen from the above content that: in the video recording mode, the camera preview interface displayed by the electronic device shows a first object, and the user performs a second operation on the first object, and the electronic device automatically responds to the second operation, determines the target position of the reflective prism, and drives the reflective prism to rotate to the target position, and then controls the camera module to shoot video. The video shot by the camera module has a higher definition than the video obtained by cropping other images. Therefore, the user can specify the object to be tracked and shot, and the electronic device automatically shoots a high-definition video.

[0010] Furthermore, the user only needs to input the second operation on the first object, and the camera module can automatically shoot the video without the user having to move the handheld device to track and shoot, and the problem of blurred video caused by the user moving the device to track can also be avoided.

[0011] Based on the second aspect, in a possible implementation, after the reflective prism rotates to the target position, after the camera module is controlled to shoot a video including the first object, the method further includes: based on the video including the first object shot by the camera module, displaying a tracking screen window of the first object on the first interface. The size of the first object in the tracking screen window of the first object may be the same as the size of the first object displayed on the first interface, or may be larger than the size of the first object displayed on the first interface.

[0012] In the above possible implementation, in the video recording mode, after the reflective prism is driven to the target position, the camera module shoots a video including the first object, and a tracking screen window of the first object can also be displayed on the first interface, so that the user can know the object being tracked and photographed by the camera module through the tracking screen window. In addition, the video displayed in the tracking screen window is obtained by the camera module, and does not need to be obtained by cropping other images, and the clarity of the video displayed in the tracking screen window can also be improved.

[0013] Based on the first aspect or the second aspect, in a possible implementation, the camera module also includes a brake component, which is used to lock the position of the reflective prism, and before driving the reflective prism to rotate to the target position, it also includes: driving the brake component to unlock; after driving the reflective prism to rotate to the target position, it also includes: driving the brake component to lock.

[0014] In this possible implementation, the camera module includes a brake component. Before the reflective prism rotates, the brake component can be unlocked. After the reflective prism rotates, the brake component is locked, which can facilitate the reflective prism to quickly stabilize after reaching the position.

[0015] Based on the first aspect or the second aspect, in a possible implementation, the camera module further includes a focusing component, which is used to achieve focusing, and the method further includes: in the process of rotating the reflective prism to the target position, driving the focusing component to focus on the first object.

[0016] In this possible implementation, the camera module includes a focusing component. When the reflective prism rotates to the target position, the focusing component is driven to focus on the first object, thereby further ensuring the clarity of the first object in the image or video taken by the camera module.

[0017] Based on the first aspect or the second aspect, in a possible implementation, before driving the focusing component to perform focusing, it also includes: based on the target position of the reflection prism, compensating the focusing parameters; in the process of the reflection prism rotating to the target position, driving the focusing component to perform focusing, including: in the process of the reflection prism rotating to the target position, driving the focusing component to focus on the first object with the compensated focusing parameters.

[0018] Based on the first aspect or the second aspect, in a possible implementation, the camera module also includes an anti-shake component, which is used for optical image stabilization, wherein: during the process of the reflection prism rotating to the target position, the anti-shake component does not operate, and after the reflection prism rotates to the target position, the anti-shake component operates to perform optical image stabilization.

[0019] In this possible implementation, the anti-shake component does not operate during the process of the reflective prism rotating to the target position. After the reflective prism rotates to the target position, the anti-shake component operates to perform optical image stabilization. This can avoid the anti-shake component performing ineffective anti-shake during the process of the reflective prism rotating to the target position, thereby increasing power consumption.

[0020] Based on the first aspect or the second aspect, in a possible implementation manner, the component for driving the reflection prism to rotate and the anti-shake component are different components.

[0021] In the above possible implementations, the camera module of the electronic device can realize tracking objects with large rotation angles by controlling the rotation of the reflective prism, and can also support anti-shake for small rotation angles through the anti-shake component. The two do not interfere with each other, and can achieve high control accuracy and good control effect.

[0022] Based on the first aspect or the second aspect, in a possible implementation, the camera module includes a rotatable reflective prism, the reflective prism can rotate in a first plane and can rotate in a second plane, and the first plane and the second plane intersect.

[0023] Based on the first aspect or the second aspect, in a possible implementation, determining a target position of the reflecting prism in response to a first operation includes: determining the target position of the reflecting prism in response to a second operation based on position information of the second operation and position information of the reflecting prism.

[0024] Based on the first aspect or the second aspect, in a possible implementation, in response to a first operation, the target position of the reflecting prism is determined, including: in response to a second operation, based on the position information of the second operation, the position information of the reflecting prism, and the image data collected by the camera module at the first moment, determining the target position of the reflecting prism at a second moment, the second moment being the next moment of the first moment.

[0025] Based on the first aspect or the second aspect, in a possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for rotating the reflective prism, and the kernel layer of the operating system includes a driver for rotating the reflective prism; in response to a second operation, determining the target position of the reflective prism, including: in response to the second operation, the algorithm module determines the target position of the reflective prism; driving the reflective prism to rotate to the target position, including: the control module for rotating the reflective prism controls the driving operation of the reflective prism rotation to drive the reflective prism to rotate to the target position.

[0026] Based on the first aspect or the second aspect, in one possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: an anti-shake control module, and the kernel layer of the operating system includes an anti-shake driver; the algorithm module is also used to generate anti-shake parameters, and the electronic device controls the anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver.

[0027] In a third aspect, the present application provides an electronic device comprising: one or more processors, a memory, a camera module and a display screen; the camera module comprises a rotatable reflective prism; the memory, the camera module and the display screen are coupled to one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and when one or more processors execute the computer instructions, the electronic device executes the shooting method as described in any one of the first aspect or the second aspect.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the shooting method as described in any one of the first aspect or the second aspect.

[0029] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the shooting method as described in any one of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 An interface diagram for a user to perform tracking and zooming operations in a photo taking mode provided by an embodiment of the present application;

[0031] Figure 2 An interface diagram of a user performing a continuous tracking object operation in a protagonist mode provided by an embodiment of the present application;

[0032] Figure 3 A hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 5 A flowchart for initializing and powering on the module provided in the embodiment of the present application;

[0035] Figure 6 A flowchart for implementing the tracking object operation provided in an embodiment of the present application;

[0036] Figure 7 A flowchart for implementing a focusing operation provided in an embodiment of the present application;

[0037] Figure 8A flowchart for powering off a module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0039] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0040] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0041] Compared with professional cameras such as SLR cameras and mirrorless cameras, cameras in electronic devices have two shortcomings: 1. The lens photosensitive element (also called image sensor) of the camera module is relatively small in size, and its insufficient photosensitivity leads to low image clarity; 2. The lens of the camera module cannot be replaced. In response to the requirements of different scenes for shooting focal lengths, electronic equipment needs to be equipped with various forms of camera modules, such as wide-angle, ultra-wide-angle, telephoto and other forms of camera modules.

[0042] Electronic devices may refer to mobile phones, tablet computers, personal digital assistants (PDA), desktops, laptops, notebook computers, ultra-mobile personal computers (UMPC), handheld computers, netbooks, and wearable devices, etc.

[0043] Due to the limitations of the above two shortcomings, the commonly used shooting methods for electronic devices to shoot moving objects at medium and long distances are as follows:

[0044] Method 1: Use a wide-angle camera module to shoot images. The size of the objects in the images shot by the wide-angle camera module is small, so the images shot by the wide-angle camera module can be cropped and the moving objects can be captured through secondary composition. Due to the small size of the camera module sensor, the image of the secondary cropping composition has low image quality and low clarity, which is difficult to meet the shooting needs.

[0045] Method 2: Use a telephoto camera module to capture images, and the user holds the electronic device to follow the moving object to capture. Since the telephoto camera module has a small field of view (FOV), it is difficult for the user to manually compose the image, and it is easy to lose the target. When the user holds the electronic device and moves it, the anti-shake capability of the electronic device is required to be high. If the user's movements are too large, the anti-shake effect will be affected, and thus the image quality will be affected.

[0046] Therefore, for moving objects at medium and long distances, electronic devices cannot realize convenient user operation and capture high-quality images or videos. Based on this, the embodiment of the present application provides a shooting method, which can enable the user to only specify the object (human body or object) to be tracked, and the electronic device automatically tracks the user-specified object, and captures high-quality images or videos through composition, framing, imaging and other actions.

[0047] The shooting method provided in the embodiment of the present application can be applied to the camera's photo mode or video recording mode. The user can input a zoom operation on the shooting object, and the camera module of the electronic device can automatically zoom in or out on the object. It can also be applied to the tracking and shooting of moving objects in the camera's video recording mode. The user can also input a focus operation on the shooting object, and the camera module of the electronic device can automatically track and shoot the moving object selected by the user.

[0048] The following combination Figure 1 and Figure 2 , two application scenarios of the shooting method provided in the embodiments of the present application are introduced.

[0049] Figure 1 Taking a mobile phone as an example, the interface diagram of the camera shooting image scene is shown.

[0050] like Figure 1As shown in (a), the mobile phone starts the camera application's photo mode, and the mobile phone displays a camera preview interface 101. The camera preview interface 101 displays a picture captured by the mobile phone's camera module, and the person in the picture is relatively far away from the mobile phone. If the user wants to zoom in on the person 102, he can click (for example, single-click or double-click) the person 102 to input the tracking object and zoom operation. In response to the user's input operation, the mobile phone tracks the person 102 by rotating the reflective prism of the camera module, and obtains an enlarged image of the person 102, as shown in FIG. Figure 1 As shown in (b), in the camera preview interface 103 displayed by the mobile phone, the person 102 is magnified and displayed, and the other two persons are not displayed.

[0051] Figure 1 In the zoom scene shown, the phone zooms in on the object or area specified by the user, such as Figure 1 The object is tracked and zoomed in on the person 102 in (a). Figure 1 (a) Transition to Figure 1 In the process of (b), the character 102 is at the center of the picture.

[0052] In some embodiments, the camera can be configured with a switch for the object tracking function in the photo mode. Figure 1 As shown in (a), the person 102 is clicked to input the tracking object and zoom operation. The mobile phone responds to the user input operation by rotating the reflective prism of the camera module to track the person 102 and obtain an enlarged image of the person 102. The switch is in the off state, and the user Figure 1 As shown in (a), the person 102 is clicked to input the tracking object and zoom operation, but the mobile phone does not respond to the operation.

[0053] Figure 2 Taking a mobile phone as an example, the interface diagram of the camera shooting video scene is shown.

[0054] like Figure 2 As shown in (a), the mobile phone starts the main character mode in the video recording mode of the camera application, and the mobile phone displays a camera preview interface 201. The camera preview interface 201 displays a picture captured by the mobile phone camera module, and the picture includes three people. If the user wants to track and focus on the person 202, he can click on the person 202 (for example, single-click or double-click) to input a continuous tracking object operation (also called a tracking operation). The mobile phone responds to the operation input by the user, such as Figure 2 As shown in (b), the camera preview interface 203 displays the tracking image 204 of the person 202 in a small window. Figure 2As shown in (c), when the person 202 is moving, the mobile phone can continuously track the person 202 by rotating the reflective prism of the camera module. In the camera preview interface 205 displayed by the mobile phone, the tracking picture 204 of the person 202 follows the person 202.

[0055] In some embodiments, the tracking screen window in the camera preview interface 205 can be operated by the user. For example, the user can drag the tracking screen to another location in the camera preview interface.

[0056] In some embodiments, the user selects a tracking object in the protagonist mode of the camera recording mode, and the user activates the camera to start shooting. During the shooting process, the mobile phone can Figure 2 As shown in (b) and (c), the tracking object selected by the user is tracked and photographed, and the tracking image is displayed in a small window. The tracking object may include a moving person, animal or other object.

[0057] In some embodiments, when the camera is shooting a video in the protagonist mode, the user can also Figure 2 The tracking object is selected in the manner shown in (a), and the mobile phone then continuously tracks and shoots the object selected by the user.

[0058] In some embodiments, the camera is in other video recording modes, not the main mode. The display screen of the mobile phone displays the camera preview interface, which can also be Figure 2 As shown in (a), the person 202 is clicked to input the continuous tracking object operation. The mobile phone responds to the operation input by the user, such as Figure 2 As shown in (b), the camera preview interface 203 displays the tracking image 204 of the person 202 in a small window. Figure 2 As shown in (c), when the person 202 is moving, the mobile phone can continuously track the person 202 by rotating the reflective prism of the camera module.

[0059] When the camera is shooting videos in other video modes, users can also Figure 2 The tracking object is selected in the manner shown in (a), and the mobile phone then continuously tracks and shoots the object selected by the user.

[0060] In some embodiments, the camera can be configured with a switch for the object tracking function in the video recording mode. Figure 2 As shown in (a), the person 202 is clicked to input the continuous tracking object operation. The mobile phone responds to the operation input by the user, such as Figure 2 As shown in (b), the camera preview interface 203 displays the tracking image 204 of the person 202 in a small window. Figure 2As shown in (a), the person 102 is clicked to input a continuous tracking object operation, and the mobile phone does not respond to the operation.

[0061] In some embodiments, the switch for the tracking object function in the video recording mode and the switch for the tracking object function in the photo taking mode may be the same switch.

[0062] The shooting method provided in the embodiment of the present application can be applied to the electronic device mentioned in the aforementioned content.

[0063] Take mobile phones as an example. Figure 3 This is an example of the composition of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display screen 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, and a camera module 190, etc.

[0064] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0065] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0066] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0067] The internal memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0068] In some embodiments, the internal memory 120 stores instructions for executing the shooting method. The processor 110 can execute the instructions stored in the internal memory 120 to realize that in the camera's photo mode or video mode, in response to the user's input zoom operation on the shooting object, the camera module 190 is automatically controlled to zoom in or out with the object as the center, and can also realize the user's input focus tracking operation on the shooting object in the camera's video mode, and control the camera module 190 to automatically track and shoot the moving object selected by the user.

[0069] The electronic device can implement some shooting functions through the ISP, the camera 130, the video codec, the GPU, the display screen 140 and the application processor, etc. The shooting function can be understood as a conventional shooting function.

[0070] The ISP is used to process the data fed back by the camera 130. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.

[0071] The camera 130 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 130, where N is a positive integer greater than 1.

[0072] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy.

[0073] Video codecs are used to compress or decompress digital videos. Electronic devices can support one or more video codecs. In this way, electronic devices can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0074] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0075] The electronic device implements the display function through a GPU, a display screen 140, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 140 and the application processor. The GPU is used for image rendering by performing mathematical and geometric calculations. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0076] The display screen 140 is used to display images, video interfaces, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oled, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 140, where N is a positive integer greater than 1.

[0077] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0078] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network.

[0079] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 110 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0080] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0081] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D and the application processor.

[0082] The audio module 170 is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals.

[0083] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device can listen to music or listen to a hands-free call through the speaker 170A.

[0084] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0085] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and directional recording function, etc.

[0086] The earphone interface 170D is used to connect a wired earphone and can be a USB interface, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0087] In the sensor module 180, the pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 140, the electronic device detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0088] The touch sensor 180B is also called a "touch control device". The touch sensor 180B can be arranged on the display screen 140, and the touch sensor 180B and the display screen 140 form a touch screen, also called a "touch control screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 140. In other embodiments, the touch sensor 180B can also be arranged on the surface of the electronic device, which is different from the position of the display screen 140.

[0089] The acceleration sensor 180C can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity, and can also be used to identify the posture of the electronic device.

[0090] The gyro sensor 180D may be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180D.

[0091] In some embodiments, the electronic device implements other shooting functions through the ISP, the camera module 190, the video codec, the GPU, the display screen 140, and the application processor. Other shooting functions may refer to: in the camera's photo mode or video mode, in response to the user's input operation on the shooting object, the object is automatically enlarged or reduced with the object as the center, and in the camera's video mode, in response to the user's input operation on the shooting object, the moving object selected by the user can be automatically tracked and shot.

[0092] In some embodiments, the camera module 190 includes: a lens photosensitive element Sensor, an OIS IC and a Scan IC. The camera module 190 may also include an AF IC, which is not shown in the figure.

[0093] In some embodiments, the camera module 190 is a telephoto module.

[0094] In the processor 110, the ISP is also used to process the data fed back by the camera module 190. For example, when taking a photo, the shutter is opened, and the light is transmitted to the sensor through the lens. The light signal is converted into an electrical signal, and the sensor transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.

[0095] The lens of the camera module 190 includes a reflective prism, which is located in the incident light path of the sensor and is used to reflect light to the sensor. In some embodiments, the reflective prism supports rotational movement in the xz plane. Exemplarily, the range of rotational movement is +-24 degrees. It also supports rotational movement in the yz plane. Exemplarily, the range of rotational movement is +-7, and the xz plane and the yz plane are two intersecting planes. The rotation of the reflective prism can also be understood as the rotation of the camera module lens. The rotation of the camera module lens can achieve that the subject is in the center of the picture during the zooming process.

[0096] Scan IC can be understood as a controller that controls the rotation of the reflective prism. For example, Scan IC can be a chip with logic control capabilities. The Scan motor receives the control instructions of Scan IC and can drive the reflective prism to rotate. In some embodiments, the Scan motor includes two, one drives the reflective prism to rotate in the xz plane, and the other drives the reflective prism to rotate in the yz plane.

[0097] In some embodiments, the brake component can be used to lock and unlock the position of the camera module lens. Before the reflective prism rotates, the brake component can be unlocked, and after the reflective prism rotates, the brake component is locked, which can facilitate the camera module lens to quickly stabilize after reaching the position. In some embodiments, the brake component includes a brake IC and an actuator. The brake IC can receive control instructions from the Scan IC to control the actuator to lock or unlock. The brake IC can be understood as a controller, such as a chip with logic control capabilities.

[0098] In some embodiments, an OIS IC (Optical Image Stabilizer IC) is used for optical image stabilization. The OIS IC can control the OIS motor to perform image stabilization. In some embodiments, the OIS motor does not perform image stabilization during the rotation of the reflective prism driven by the Scan motor. Based on this, the Scan IC can control the OIS motor not to operate during the operation of the Scan motor and to operate after the Scan motor finishes operating. The OIS IC and the OIS motor can be collectively referred to as an image stabilization component.

[0099] In some embodiments, the AF motor is used for focusing, and the OIS IC can control the AF motor to focus after performing algorithm calculation compensation. The AF motor can also be called a focusing component.

[0100] Electronic equipment in Figure 3 The hardware components shown also have operating systems running on them. operating system, operating system, Operating system, etc. Applications such as camera applications can be installed and run on the operating system.

[0101] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0102] The layered architecture divides the operating system of the electronic device into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers, from top to bottom, namely the application (APP) layer, the application framework layer (abbreviated as FWK), the system library, the hardware abstraction layer (HAL) and the kernel layer.

[0103] The application layer can include a series of application packages. Figure 4 As shown, the application package may include applications such as camera and call.

[0104] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0105] like Figure 4 As shown, the application framework layer may include window management services, content providers, phone managers, view systems, resource managers, and camera services, etc.

[0106] The window management service is used to manage window programs. The window management service can implement window addition, deletion, display and hiding control, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data can include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0107] In some embodiments, the camera service may also be referred to as a camera framework, which is used to receive recording requests, image capture requests, and other requests from camera applications, while maintaining the business logic of the internal circulation of recording requests, image capture requests, and other requests, and sending the final result of the request to the camera application.

[0108] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. In some embodiments of the present application, the application cold start will run in the Android runtime, and the Android runtime will obtain the optimized file status parameters of the application. Then, the Android runtime can determine whether the optimized file is outdated due to system upgrades based on the optimized file status parameters, and return the judgment result to the application management module.

[0109] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0110] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0111] The system library may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0112] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0113] The hardware abstraction layer is a software located between the operating system kernel and the hardware circuit. It is usually used to abstract the hardware to realize the interaction between the operating system and the hardware circuit at the logical layer.

[0114] In some embodiments, the hardware abstraction layer includes a camera Hal, which includes an image link Pipeline and CamxHal3. The camera Hal is mainly responsible for the construction, linking, and interactive control of all modules in the image link. CamxHal3 is used to interact with the upper-level module to open or close the camera Hal, and to transmit data to the camera Hal.

[0115] In some embodiments, the image link Pipeline includes: a module control module, an ISP control module and an algorithm module; the module module is responsible for processing the camera module management, and the module module includes a sensor control module, a scan control module, an OIS control module, and a motor actuator control module.

[0116] In some embodiments, the sensor control module is used to connect to the sensor, and the sensor control module can control the operation of the sensor through the sensor driver and sensor firmware.

[0117] In some embodiments, the Scan control module may be referred to as a control module of a reflective prism, and is used to connect to a Scan motor to realize rotation of the reflective prism. The Scan control module may control the operation of the Scan motor through a Scan driver and Scan Firmware.

[0118] In some embodiments, the OIS control module can be called an anti-shake control module, which is used to implement anti-shake processing during the rotation of the camera module. The OIS control module can control the operation of the OIS motor through the OIS driver and OIS Firmware. In other embodiments, the Scan control module can control the operation of the OIS motor through the Scan driver, Scan Firmware and control OIS Firmware.

[0119] In some embodiments, the Actuator control module is used for AF focusing during the rotation of the camera module. The Actuator control module can control the operation of the AF motor through AF drive, OIS Firmware and AF Firmware.

[0120] The ISP control module is used to receive the image data sent by the sensor, process the image data and send it to the algorithm module. In some embodiments, the sensor refers to the sensor of the camera module 190.

[0121] The algorithm module is used to receive the image data sent by the ISP control module, analyze the image data, obtain the operation strategy of the lower module, and feed it back to the module in the module control module. In some embodiments, the algorithm module obtains the rotation strategy of the reflective prism and feeds it back to the Scan control module.

[0122] The kernel layer is the layer between hardware and software. The kernel layer receives commands from the control layer and supports the conversion of control commands into actual device operation instructions. Operation instructions include power-on and power-off control, device mode register instruction generation and issuance. The register control commands of the driver layer are passed to the firmware layer through the device bus. The kernel layer includes at least Sensor driver, Scan driver, OIS driver, and AF driver.

[0123] The firmware layer refers to the software system integrated into each component of the camera module, including Sensor Firmware, Scan Firmware, OIS Firmware, AF Firmware and Brake Firmware. In some embodiments, the firmware layer may not include Brake Firmware.

[0124] In some embodiments, Sensor Firmware is a software system running inside the Sensor, responsible for sensor exposure and image data output. Scan Firmware is a software system running inside the Scan IC, used to control the rotation of the camera module lens, and is responsible for converting the code sent down from the upper layer into a control signal, so as to drive the camera module lens to rotate and achieve the focus tracking and framing goals. Brake Firmware is a software system running inside the brake IC, used to lock the position of the camera module, so that the camera module can quickly stabilize after reaching the target position. OIS Firmware is a software system running inside the OIS IC, used for anti-shake. AF Firmware is a software system running inside the AF IC, used for focus tracking, responsible for converting the target position sent down from the upper layer into a motor current, and driving the AF motor to focus.

[0125] Under the above five-layer architecture, the electronic device is also provided with a hardware layer, which may include the above-mentioned electronic device hardware components. Figure 4 The display includes Sensor, Scan IC, OIS IC, AF motor and brake components.

[0126] It should be noted that although the embodiments of the present application are based on The system is used as an example, but the basic principles are also applicable to Electronic devices with other operating systems.

[0127] In the embodiment of the present application, the electronic device realizes the function of controlling the rotation of the reflective prism and the function of realizing the anti-shake function independently of each other. From the perspective of hardware structure: the function of controlling the rotation of the reflective prism is realized by the Scan motor, and the anti-shake function is realized by the OIS motor, and the two are independent of each other; from the perspective of software control: the Scan control module controls the operation of the Scan motor through the Scan driver and Scan Firmware, and the OIS control module controls the operation of the OIS motor through the OIS driver and OIS Firmware, and the two are also independent of each other.

[0128] This ensures that the camera module of the electronic device can track objects with large rotation angles by controlling the rotation of the reflective prism, and can also support anti-shake with small rotation angles through the operation of the OIS motor. The two do not interfere with each other, and can achieve high control accuracy and good control effect.

[0129] The electronic device is equipped with a brake component, which includes a brake IC and an actuator. The brake IC can receive a control instruction from the Scan IC to control the actuator to lock or unlock, so as to unlock the reflective prism before it rotates, and lock it after the reflective prism rotates into place to achieve rapid braking and stability.

[0130] The Actuator control module can control the AF motor operation through AF driver, OIS firmware and AF firmware. OIS firmware can realize the position dynamic compensation focus parameters of Scan motor.

[0131] Before the electronic device executes the shooting method provided in this embodiment, the electronic device may, according to the operation of opening the camera application by the user, Figure 4 The initialization of the modules associated with the shooting function is shown. The modules associated with the shooting function include: Figure 4 The displayed Sensor control module, Scan control module, OIS control module, Actuator control module, Sensor driver, Scan driver, OIS driver, AF driver, Sensor Firmware, Scan Firmware, OIS Firmware, AF Firmware, brake Firmware, Sensor, Scan IC, OIS IC, AF motor and brake components.

[0132] Among them, the Sensor control module, Sensor driver, Sensor Firmware and Sensor are used to obtain image data. After the module control module is created, the Sensor control module Sensor driver, Sensor Firmware and Sensor initialization and power-on process can be executed.

[0133] The Actuator control module, AF driver, AF Firmware and AF motor are used to perform focusing. After the module control module is created, the initialization and power-on process of the Actuator control module, AF driver, AF Firmware and AF motor can be executed.

[0134] The OIS control module, OIS driver, OIS Firmware and OIS IC are used to perform anti-shake. After the module control module is created, the initialization and power-on process of the OIS control module, OIS driver, OIS Firmware and OIS IC can be executed.

[0135] The Scan control module, Scan driver, Scan Firmware and Scan IC are used to realize the rotation of the reflective prism. After the module control module is created, the initialization and power-on process of the Scan control module, Scan driver, Scan Firmware and Scan IC can be executed.

[0136] In some embodiments, after the module control module is created, the initialization and power-on process can be performed in the order mentioned above, that is, the module for obtaining image data first, the module for executing focus, the module for executing anti-shake, and finally the module for realizing the rotation of the reflective prism. The initialization and power-on process of the module for obtaining image data, the module for executing focus, and the module for executing anti-shake can refer to conventional technology, and this embodiment will not be described in detail.

[0137] The following combination Figure 5 The initialization and power-on process of the module that realizes the rotation of the reflective prism is introduced.

[0138] like Figure 5 As shown, the initialization and power-on process of the module for realizing the rotation of the reflective prism provided in the embodiment of the present application includes:

[0139] S501: The user starts a camera application.

[0140] The user starts the camera application by inputting a camera start operation. The electronic device responds to the user's operation and starts the camera application. The camera application is started, the camera is controlled to collect images and provide them to the camera application, and the camera application can display the images collected by the camera on the display screen to form a camera preview interface. For example, Figure 1 (a) shows the camera preview interface when the camera application is turned on and in photo mode.

[0141] In some embodiments, the user's operation of opening the camera application may include: clicking the camera application icon on the display screen, pressing a specific physical button, inputting voice, and air gestures, etc.

[0142] S502: The camera application connects to the camera service.

[0143] The camera application is opened by the user, and the camera application is connected to the camera service. As described in the aforementioned electronic device software structure, the camera service is located in the application framework layer.

[0144] S503: The camera service turns on the camera Hal.

[0145] After the camera service is connected to the camera application, the camera Hal can be opened. As described in the aforementioned electronic device software structure, the camera Hal is located in the hardware abstraction layer.

[0146] S504, CamxHal3 creates Pipeline.

[0147] The camera Hal is opened, and CamxHal3 can create an image link Pipeline.

[0148] In some embodiments, the image link pipeline includes: a module control module, an ISP control module, and an algorithm module. Therefore, CamxHal3 creating a pipeline can be understood as: CamxHal3 creates the module control module, ISP control module, and algorithm module of the image link pipeline. In addition, the submodules in the module control module are also created.

[0149] S505, Pipeline initialization algorithm module.

[0150] After CamxHal3 creates an algorithm module through Pipeline, Pipeline can initialize the algorithm module. Pipeline initialization of the algorithm module can be understood as configuring the initialization parameters of the algorithm module.

[0151] S506, Pipeline creates a module control module.

[0152] The module control module includes: sensor control module, scan control module, OIS control module and actuator control module. Pipeline creation of module control module can also be understood as creating the sensor control module, scan control module, OIS control module and actuator control module of the module control module.

[0153] S507, the module control module creates a sub-module, and the sub-module includes a Scan control module.

[0154] After Pipeline creates the module control module, the module control module creates submodules, including: Sensor control module, Scan control module, OIS control module, Actuator control module. This embodiment introduces the initialization and power-on process of the module that realizes the rotation of the reflective prism, so this step only reflects the module control module creating the Scan control module.

[0155] S508. The Scan control module triggers the Scan Driver to power on.

[0156] After the Scan control module is created, it can trigger the Scan Driver to power on.

[0157] In some embodiments, the Scan control module executes AcquireDevice to trigger the Scan Driver to power on, and the ScanDriver is the Scan driver.

[0158] S509, Scan Driver triggers Scan IC to power on and initialize.

[0159] After Scan Driver is powered on, Scan Driver can control Scan IC to power on. After Scan IC is powered on, the link between ScanDriver and Scan IC is connected, such as I2C bus. Scan Driver can send initialization instructions to ScanIC through the link to control Scan IC to perform initialization operations, such as reset.

[0160] In some embodiments, after the Scan IC performs power-on and initialization, it can return its own execution results to the Scan Driver. The Scan Driver can return its own execution results to the Scan control module. The Scan control module can return its own execution results to the module control module. The module control module and the algorithm module can return their own execution results to the Pipeline. The Pipeline returns its own execution results to CamxHal3. CamxHal3 returns its own execution results to the camera service. The camera service returns its own execution results to the camera application.

[0161] After the initialization of the module associated with the shooting function is completed, the electronic device can execute the shooting method provided in the embodiment of the present application to shoot images or videos. Among them, the shooting method in the embodiment of the present application involves tracking object operation and focusing operation. Figure 6 Introduce the tracking process of electronic devices, combined with Figure 7 The focusing process of electronic equipment is introduced.

[0162] The object tracking process and the focusing process of the electronic device can be understood to be performed in parallel, so two embodiments are used for introduction below.

[0163] like Figure 6 As shown, the tracking object process of the electronic device provided in the embodiment of the present application includes:

[0164] S601: The camera application receives an input operation from a user.

[0165] The operations input by the user include: tracking an object and zooming in and out and continuously tracking an object.

[0166] In some embodiments, the user starts a camera application, and the camera application displays a camera preview interface on the display screen. The user can select a camera operating mode, input tracking object operations, etc. on the camera preview interface. Figure 1 It shows the user inputting tracking objects and zooming operations in the camera preview interface. Figure 2 Demonstrates user input into the camera preview interface to continuously track an object.

[0167] In some embodiments, the camera application starts shooting. During the video shooting process, the camera application displays a video shooting interface, and the user can also input tracking object operations in the video shooting interface.

[0168] In some embodiments, the camera application receives an input operation from a user and may also obtain an operation position of the user.

[0169] like Figure 2 As shown in (a), the user clicks on the object in the camera preview interface to input the tracking object and zoom operation, and the camera application receives the user's operation and obtains the user's operation position. Figure 3 As shown in (a), the user clicks on the object in the camera preview interface to input the continuous tracking object operation, and the camera application can also receive the user's operation and obtain the user's operation position. In some embodiments, the user's operation position can be understood as the coordinates of the operation position input by the user in the camera preview interface displaying the image.

[0170] S602: The camera application sends a request message to the camera service. The request carries coordinate information of the user's operation position.

[0171] After receiving the tracking object operation input by the user and obtaining the user's operation position, the camera application can send a request message to the camera service. The request message carries the location information of the user's operation position, that is, carries the coordinate information of the user's operation position. The request message sent by the camera application to the camera service is used to request the execution of the tracking object process.

[0172] S603. The camera service sends a request message to CamxHal3. The request carries the coordinate information of the user's operation position.

[0173] After receiving the request message sent by the camera application, the camera service sends the request message to CamxHal3.

[0174] S604. CamxHal3 sends a request message to Pipeline, and the request carries the coordinate information of the user's operation position.

[0175] After receiving the request message sent by the camera service, CamxHal3 sends the request message to Pipeline.

[0176] In some embodiments, the user inputs a tracking object and performs a zoom operation. After CamxHal3 receives a request message sent by the camera service, it sends the request message to the Pipeline corresponding to the camera module 190 .

[0177] It should be noted that when the user turns on the camera, the camera 130 of the electronic device (which can be understood as the main camera) runs to collect images, and the functional modules of the Pipeline corresponding to the camera 130 run to display the camera preview interface on the display screen, which displays the image taken by the camera 130.

[0178] The user inputs the tracking object and zoom operation in the camera preview interface, and the functional module of the Pipeline corresponding to the camera module 190 is driven to run, and the image captured by the camera module 190 can be displayed on the display screen to maintain the camera preview interface. The camera module 190 is a telephoto module, and the subject in the image captured by the camera module 190 is large, which can achieve the enlargement of the image of the photographed object. In some embodiments, the Pipeline corresponding to the camera 130 can run or stop running, without limitation.

[0179] In other embodiments, the user inputs a continuous tracking object operation, and after CamxHal3 receives the request message sent by the camera service, it sends the request to the Pipeline corresponding to the camera module 190.

[0180] Similar to the above content, the user turns on the camera, the camera 130 runs to collect images to obtain the camera preview interface, the user inputs the continuous tracking object operation, the functional module of the Pipeline corresponding to the camera module 190 is driven to run, and the image collected by the camera module 190 can be displayed in the tracking screen window of the camera preview interface.

[0181] S605: Pipeline calls the algorithm module to process the request message.

[0182] After receiving the request message, the Pipeline calls the algorithm module to process the request message. In some embodiments, the Pipeline may send the request message to the algorithm module to call the algorithm module to process the request message.

[0183] S607: The algorithm module obtains the target position of the reflecting prism based on the coordinates and the position information of the reflecting prism.

[0184] In some embodiments, the algorithm module stores data such as the position information of the reflective prism and the state of the brake component. In some embodiments, the position information of the reflective prism can be the detection value of the TMR sensor, referred to as TMR data.

[0185] In some embodiments, the user inputs a tracking object and zoom operation, and after receiving the request message, the algorithm module obtains the coordinate information and the position information of the reflective prism carried in the request message, and obtains the position reached by the reflective prism after rotation based on the coordinate information and the position information of the reflective prism carried in the request, that is, the target position. In some embodiments, the target position can be a code.

[0186] In some embodiments, the algorithm module may store calibration data, which includes the rotation angle of the reflective prism corresponding to the coordinates operated by the user. The algorithm module combines the position information and rotation angle of the reflective prism to obtain the target position of the reflective prism. The target position can be understood as the position to be reached by the reflective prism after rotation.

[0187] It should be noted that the algorithm module analyzes the coordinate information of the zoom operation and the position information of the reflective prism through step S607 to obtain the target position of the reflective prism. The target position can be understood as ensuring that the zoom operation object is located in the center area of ​​the screen after the zoom operation. Therefore, the Scan Firmware controls the Scan motor to rotate and drive the reflective prism to rotate until the reflective prism rotates to the target position. After the user performs a zoom operation on an object, the object is automatically located in the center of the screen in the screen displayed on the display screen, without user operation or image cropping. In addition, the user does not need to operate, and the problem of blurred images caused by the user moving the device to track can be avoided.

[0188] In other embodiments, the user inputs a continuous tracking object operation, and after the algorithm module receives the request message, it obtains the coordinate information and the position information of the reflecting prism carried in the request message, and also obtains the image data collected by the camera, and obtains the target position of the reflecting prism based on the image data, the coordinate information and the position information of the reflecting prism.

[0189] It should be noted that the user inputs a continuous tracking object operation, and the electronic device needs to continuously track and shoot the object specified by the operation. Therefore, the algorithm module can continuously obtain the target position of the reflective prism to drive the reflective prism to continuously rotate and track the object. Based on the coordinate information of the user's operation position, the algorithm module can determine the user's operation object and determine the moving direction of the object in the image data collected by the camera. The algorithm module combines the moving direction of the object and the current position information of the reflective prism to determine the target position of the next rotation of the reflective prism. The target position of each rotation of the reflective prism can be understood as a position that can ensure that the camera can capture the user's operation object. In this way, the electronic device can automatically follow the moving object for shooting. Since no user operation is required, the clarity of the captured video can also be guaranteed.

[0190] The camera for collecting images can be the main camera, that is, camera 130, or it can be the camera module 190; the camera can also be the main camera for the user input of the continuous tracking object operation moment, and after the algorithm module obtains the target position of the reflecting prism corresponding to the moment, the camera is the camera module 190.

[0191] In some embodiments, the target position of the reflective prism obtained by the algorithm module can be written into the data pool metadataPool so that other modules can monitor the changes of data in the data pool. In some embodiments, the metadata Pool includes multiple metadata, each metadata is used to store the value of a different function. The algorithm module writes the target position of the reflective prism into the metadata corresponding to the tracking object function.

[0192] In some embodiments, the algorithm module also stores the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake parameters of the OIS motor are the detection values ​​of the Hall sensor, referred to as HALL data. After receiving the request, the algorithm module can also obtain the indication value of the anti-shake strategy of the OIS motor according to the coordinates carried in the request and the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake strategy includes the operating mode and operating parameters of the OIS.

[0193] In some embodiments, the indicator value of the anti-shake strategy of the OIS motor obtained by the algorithm module (also called anti-shake parameter) can also be written into the data pool metadata Pool so that other modules can monitor the changes of data in the data pool. In some embodiments, the indicator value of the anti-shake strategy of the OIS motor obtained by the algorithm module is written into the metadata corresponding to the anti-shake function.

[0194] S608: The algorithm module sends the target position of the reflection prism to the Scan control module.

[0195] In some embodiments, after the algorithm module obtains the target position of the reflective prism, it can send it directly to the Scan control module. In other embodiments, the algorithm module writes the target position of the reflective prism into metadata, and the Scan control module can monitor the metadata to obtain the target position of the reflective prism written by the algorithm module.

[0196] S609. The Scan control module sends the target position of the reflective prism to the Scan driver.

[0197] In some embodiments, before the Scan control module sends the target position of the reflective prism to the Scan driver, it can verify whether the target position of the reflective prism is a legal position. If the verification passes, the target position of the reflective prism is sent to the Scan driver.

[0198] S610: The Scan driver sends a first control instruction to the Scan Firmware.

[0199] The first control instruction is used to control the Scan Firmware to execute brake unlocking, reflection prism rotation and brake locking.

[0200] After the Scan driver receives the target position of the reflective prism, it can control the Scan Firmware to execute brake unlocking, reflective prism rotation and brake locking.

[0201] In some embodiments, the Scan driver converts the target position of the reflective prism into a register instruction, and the register instruction is transmitted to the Scan Firmware via a bus such as I2C. The Scan Firmware executes the following steps S611 and S612 according to the register instruction.

[0202] S611. Scan Firmware sends a second control instruction to OIS Firmware.

[0203] The second control instruction is used for when the rotation starts, the Scan Firmware controls the OIS Firmware to enter the HOLDON state; after the rotation ends, the Scan Firmware controls the OIS Firmware to enter the OIS ON state.

[0204] During the rotation of the reflective prism, the anti-shake performed by the electronic device is invalid. Therefore, the electronic device does not need to perform anti-shake. When the rotation of the reflective prism ends, the electronic device performs anti-shake. Based on this, Scan Firmware controls OIS Firmware to enter HOLD ON at the beginning of the rotation of the reflective prism, and controls OIS Firmware to enter OIS ON at the end of the rotation of the reflective prism. Among them, HOLD ON can be understood as the anti-shake pause state, and OIS ON can be understood as the anti-shake on state.

[0205] S612, Scan Firmware executes the rotation of the reflection prism.

[0206] In some embodiments, the Scan Firmware first executes the brake unlocking before executing the rotation of the reflective prism, and then executes the brake unlocking after the rotation of the reflective prism is completed.

[0207] In some embodiments, the Scan Firmware controls the brake components to unlock and lock through the brake Firmware.

[0208] In some embodiments, the Scan Firmware controls the Scan motor to rotate and drive the reflective prism to rotate until the reflective prism rotates to a target position.

[0209] against Figure 1 The tracking object and zoom operation are shown. After the reflective prism moves to the target position, the display shows Figure 1 In the interface shown in (b), the user can view the zoomed image through the display screen.

[0210] against Figure 2 The continuous tracking operation of the electronic device is shown. The camera of the electronic device collects a frame of image. After the reflective prism rotates to the target position for the frame of image, the display screen displays the tracking screen of the frame of image. Figure 2 As shown in (b) or (c).

[0211] The execution order of step S611 and step S612 is not limited to Figure 6 As shown, in some embodiments, ScanFirmware may execute step S611 and step S612 in parallel.

[0212] S613, during the rotation, the OIS Firmware enters HOLD ON, and after the rotation is completed, the OIS Firmware enters OIS ON.

[0213] In some embodiments, during the rotation process, the OIS Firmware enters the HOLD ON state, and the control instructions issued by the OIS driver are only cached and not directly executed. After the reflective prism rotates to the target position, the Scan Firmware passes the TMR data and the OIS ON instruction to the OIS Firmware, and the OIS Firmware enters the OIS ON state, and updates the anti-shake parameters according to the TMR data, and controls the OIS motor to restore the state before the HOLD ON state or restore to the cached state.

[0214] In some embodiments, the Scan Firmware may periodically report TMR data to the Scan driver, and the Scan driver reports the TMR data to the algorithm module through the Scan control module, which is then updated and stored by the algorithm module.

[0215] In some embodiments, the OIS Firmware may also periodically report the HALL data to the OIS driver, and the OIS driver reports the HALL data to the algorithm module through the OIS control module, and the algorithm module updates and stores the data.

[0216] like Figure 7 As shown, the focusing process of the electronic device provided in the embodiment of the present application includes:

[0217] S701: The camera application receives an input operation from a user.

[0218] The specific implementation of step S701 can refer to the content of step S601 in the above embodiment, which will not be repeated here.

[0219] S702: The camera application sends a request message to the camera service. The request carries coordinate information of the user's operation location.

[0220] The specific implementation of step S702 can refer to the content of step S602 in the above embodiment, which will not be repeated here.

[0221] S703. The camera service sends a request message to CamxHal3, and the request carries the coordinate information of the user's operation position.

[0222] The specific implementation of step S703 can refer to the content of step S603 in the above embodiment, which will not be repeated here.

[0223] S704. CamxHal3 sends a request message to Pipeline, and the request carries the coordinate information of the user's operation position.

[0224] The specific implementation of step S704 can refer to the content of step S604 in the above embodiment, which will not be repeated here.

[0225] S705. Pipeline calls the algorithm module to process the request message.

[0226] The specific implementation of step S705 can refer to the content of step S605 in the above embodiment, which will not be repeated here.

[0227] S706: The algorithm module obtains focus parameters.

[0228] The focus parameters can be understood as the operating parameters of the AF motor. In some embodiments, the algorithm module can obtain the focus parameters based on the contrast focus algorithm or the phase focus algorithm. In other embodiments, the algorithm module obtains one focus parameter based on the contrast focus algorithm and another focus parameter based on the phase focus algorithm. The algorithm module combines the two focus parameters to obtain the focus parameters to be issued.

[0229] S707: The algorithm module sends the focus parameters to the Actuator control module.

[0230] In some embodiments, after obtaining the focus parameters, the algorithm module may directly send the focus parameters to the Actuator control module.

[0231] In some other embodiments, after the algorithm parameters obtain the focus parameters, the metadata corresponding to the focus function can be written into the metadata Pool. The Actuator control module monitors the data changes of the metadata and obtains the written focus parameters. In some embodiments, the focus parameters can be codes.

[0232] S708. The Actuator control module sends focus parameters to the AF driver.

[0233] S709 , when receiving the interrupt signal, the AF driver controls the OIS Firmware to perform focusing according to the focus parameter, and the interrupt signal indicates that the image is framed.

[0234] After receiving the focus parameters, the AF driver will not immediately control the AF firmware to perform focus through the OIS firmware, but will wait for the interrupt signal corresponding to the image frame output. After the AF driver receives the interrupt signal, it can perform focus according to the focus parameters.

[0235] S710, OIS Firmware compensates the focus parameters according to the target position of the reflective prism.

[0236] In some embodiments, the OIS Firmware stores the correspondence between the target positions of multiple reflective prisms and the compensation values ​​of the focus parameters. The OIS Firmware can determine the compensation value corresponding to the target position of the reflective prism based on the correspondence, and compensate the focus parameter based on the compensation value.

[0237] In some embodiments, the AF motor includes two, and the OIS Firmware compensates the focus parameters by the compensation values ​​of the two AF motors according to the target position of the reflective prism to obtain the compensated focus parameters of the two AF motors.

[0238] In some embodiments, the Scan Firmware periodically sends the position information of the reflective prism to the OIS Firmware, and the OIS Firmware may use the position information of the reflective prism sent by the Scan Firmware at the latest moment as the target position of the reflective prism.

[0239] S711. The OIS Firmware controls the AF Firmware to perform focusing according to the compensated focus parameters.

[0240] In some embodiments, when the OIS Firmware receives the focus parameters, step S710 may not be executed, and the focus parameters may be transparently transmitted to the AF Firmware, and the AF Firmware performs focusing according to the focus parameters.

[0241] S712, AF Firmware performs focusing according to the compensated focus parameters.

[0242] The AF Firmware controls the AF motor to operate and achieve focus according to the compensated focus parameters. In some embodiments, there are two AF motors, and the AF Firmware controls the two AF motors to operate and achieve focus according to the compensated focus parameters.

[0243] After the camera application is closed or a working module of the camera application is exited, the module associated with the shooting function mentioned above can be exited. Figure 8 Taking the camera application being closed as an example, the exit process of the module controlling the rotation of the reflective prism is introduced. After the camera is closed or the working mode of the camera application is exited, other modules associated with the shooting function will also execute the exit process. The exit process can be referred to conventional technology, and this embodiment will not introduce it in detail.

[0244] It should be noted that the module for realizing the rotation of the reflective prism will be started and run in the zoom working mode and the focus tracking working mode of the camera application. Therefore, after the zoom working mode and the focus tracking working mode of the camera application are exited, the module for realizing the rotation of the reflective prism can be controlled to exit operation.

[0245] like Figure 8 As shown, the exit process of the module for realizing the rotation of the reflecting prism includes:

[0246] S801. The user closes the camera application.

[0247] The user closes the camera application by inputting an operation to close the camera application. In some embodiments, the camera application closing operation may include performing a return operation in the camera preview interface of the camera application, closing the camera application in the multitasking management interface, closing the electronic device, etc.

[0248] S802: The camera application turns off the camera service.

[0249] After the user inputs an operation to close the camera application, the camera application may close the camera service before closing.

[0250] S803. The camera service turns off the camera Hal.

[0251] The camera service is closed under the control of the camera application. The camera service can close the camera Hal before closing.

[0252] S804, CamxHal3 destroys Pipeline.

[0253] When the camera Hal is closed by the camera service, the Pipeline can be destroyed by CamxHal3. The Pipeline is destroyed, which can be understood as executing the following steps S805 and S806 before the Pipeline is destroyed.

[0254] It should be noted that the image link pipeline includes: module control module, ISP control module and algorithm module. Before the pipeline is destroyed, the ISP control module can also be controlled to exit.

[0255] S805: The Pipeline control algorithm module exits.

[0256] S806, the Pipeline control module exits.

[0257] S807. The module control module sends a Scan exit command to the Scan control module.

[0258] Before exiting, the module control module sends a Scan exit command to the Scan control module to control the Scan control module to execute the power-off and ejection process for the modules of the driver layer and the components of the hardware layer.

[0259] S808. The Scan control module sends a Scan exit command to the Scan Driver and triggers power off.

[0260] After receiving the exit command, the Scan control module may send a Scan exit command to the Scan Driver to control the ScanDriver to exit.

[0261] S809, Scan Driver controls Scan IC to exit and power off.

[0262] Before the Scan Driver exits, the Scan IC can be controlled to exit and power off. In some embodiments, the Scan IC exiting and powering off includes: the Scan motor corresponding to the Scan IC is centered, the brake actuator performs brake locking, and the Scan IC is powered off, wherein the Scan motor corresponding to the Scan IC is centered and can be understood as restoring to an initial position.

[0263] In some embodiments, after the Scan IC completes the execution and exits and is powered off, it may return its own execution result to the Scan Driver. The Scan Driver may return its own execution result to the Scan control module. The Scan control module may return its own execution result to the module control module. The module control module and the algorithm module may return their own execution results to the Pipeline. The Pipeline returns its own execution result to CamxHal3. CamxHal3 returns its own execution result to the camera service. The camera service returns its own execution result to the camera application.

[0264] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0265] The computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0266] Another embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

Claims

1. A shooting method, characterized in that: Applied to an electronic device, the electronic device includes a camera module, the camera module includes a rotatable reflective prism, and the shooting method includes: Receiving a first operation from a user; In response to the first operation, a camera application is started and a first interface is presented, wherein the first interface is a camera preview interface or a camera shooting interface, and the first interface displays a first object of a first size; receiving a second operation of the user on the first interface with respect to the first object; In response to the second operation, obtaining the position information of the second operation through the algorithm module, determining the rotation angle of the reflective prism corresponding to the position information of the second operation based on calibration data, and determining the target position of the reflective prism based on the position information of the reflective prism and the rotation angle of the reflective prism; the calibration data includes the rotation angle of the reflective prism corresponding to the coordinates of the user operation; Verifying the target position of the reflecting prism by a control module of the reflecting prism to verify whether the target position of the reflecting prism is a legal position; When the target position of the reflective prism is verified, driving the reflective prism to rotate to the target position; After the reflective prism rotates to the target position, the camera module is controlled to capture an image or video including the first object; wherein the first object in the image captured by the camera module is of a second size, and the second size is larger than the first size.

2. The shooting method according to claim 1, characterized in that: After the reflective prism rotates to the target position, the camera module is controlled to shoot a video including the first object, and the method further includes: Based on the video including the first object shot by the camera module, a tracking screen window of the first object is displayed on the first interface.

3. The shooting method according to claim 1 or 2, characterized in that: The camera module also includes a brake component. Before driving the reflective prism to rotate to the target position, the method further includes: driving the brake component to unlock; After driving the reflective prism to rotate to the target position, the method further includes: driving the brake component to lock.

4. The shooting method according to claim 1 or 2, characterized in that: The camera module further includes a focusing component, and the method further includes: When the reflection prism rotates to the target position, the focusing component is driven to focus on the first object.

5. The shooting method according to claim 4, characterized in that: Before driving the focusing component to perform focusing, the method further includes: Compensating focus parameters based on a target position of the reflective prism; In the process of rotating the reflective prism to the target position, driving the focusing component to perform focusing includes: When the reflective prism rotates to the target position, the focusing component is driven to focus on the first object with the compensated focusing parameters.

6. The shooting method according to claim 1 or 2, characterized in that: The camera module also includes an anti-shake component, wherein: during the process of the reflection prism rotating to the target position, the anti-shake component does not operate, and after the reflection prism rotates to the target position, the anti-shake component operates to perform optical image stabilization.

7. The shooting method according to claim 6, characterized in that: The component driving the reflection prism to rotate and the anti-shake component are different components.

8. The shooting method according to claim 1 or 2, characterized in that: The reflecting prism is rotatable in a first plane and in a second plane, and the first plane and the second plane intersect.

9. The shooting method according to claim 1 or 2, characterized in that: Before determining the rotation angle of the reflective prism corresponding to the position information of the second operation based on the calibration data, and determining the target position of the reflective prism based on the position information of the reflective prism and the rotation angle of the reflective prism, the method further includes: Determining, by the algorithm module, a moving direction of the first object based on the position information of the second operation and the image data collected by the camera module at the first moment; Wherein, determining the rotation angle of the reflective prism corresponding to the position information of the second operation based on the calibration data, and determining the target position of the reflective prism based on the position information of the reflective prism and the rotation angle of the reflective prism, includes: Based on the calibration data, the rotation angle of the reflecting prism corresponding to the position information of the second operation along the moving direction of the first object is determined, and based on the position information of the reflecting prism and the rotation angle of the reflecting prism, the target position of the reflecting prism at a second moment is determined, and the second moment is the next moment of the first moment.

10. The shooting method according to claim 1 or 2, characterized in that: The hardware abstraction layer of the operating system of the electronic device includes: the control module for the rotation of the reflective prism and the algorithm module, and the kernel layer of the operating system includes a driver for the rotation of the reflective prism; The driving the reflective prism to rotate to the target position includes: the control module for the rotation of the reflective prism controls the driving operation of the rotation of the reflective prism to drive the reflective prism to rotate to the target position.

11. The shooting method according to claim 10, characterized in that: The hardware abstraction layer of the operating system of the electronic device includes: an anti-shake control module, and the kernel layer of the operating system includes an anti-shake driver; The algorithm module is also used to generate anti-shake parameters, and the electronic device controls the anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver.

12. An electronic device, characterized in that: include: One or more processors, memories, camera modules and display screens; the camera modules include a rotatable reflective prism; The memory, the camera module and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the shooting method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the shooting method according to any one of claims 1 to 11.

Images