Shooting method, electronic equipment and readable medium

Abstract

The invention provides a shooting method, electronic equipment and a readable medium, the shooting method is applied to the electronic equipment, the electronic equipment comprises a camera module with a rotatable reflecting prism, and the method comprises the following steps: receiving a first operation of a user; in response to the first operation, starting a camera application, and presenting a first interface, the first interface being a camera preview interface or a camera shooting interface, and the first interface displaying a first object with a first size; receiving a second operation of the user on the first interface for the first object; in response to the second operation, determining a target position of the reflecting prism; driving the reflecting prism to rotate to a target position; after the reflecting prism rotates to the target position, the camera module is controlled to shoot an image or a video comprising the first object; wherein the first object in the image shot by the camera module is of the second size, the second size is larger than the first size, a user can specify the object needing to be tracked and shot, and the electronic equipment can automatically shoot an image or a video with high definition.

Description

[0001] This application is a divisional application of the invention patent application with the application date of August 11, 2023, the application number of 202311019446.4, and the invention title of "Shooting Method, Electronic Device and Readable Medium". Technical Field

[0002] This application relates to the field of shooting technology, and in particular, to a shooting method, an electronic device, a computer program product, and a computer-readable storage medium. Background Art

[0003] When an electronic device tracks a shooting object, for example, tracks a moving object at a medium or long distance, the clarity of the captured image or video is usually not high, which affects the shooting effect. Summary of the Invention

[0004] This application provides a shooting method, an electronic device, a computer program product, and a computer-readable storage medium, which can enable a user to specify an object to be tracked and shot, and the electronic device automatically shoots an image or video with high clarity.

[0005] To achieve the above object, this application provides the following technical solutions:

[0006] In a first aspect, this application provides a shooting method, which is applied to an electronic device. The electronic device includes a camera module, and the camera module includes a rotatable reflecting prism. The shooting method includes: receiving a first operation of a user, and in response to the first operation, starting a camera application and presenting a first interface, where the first interface is a camera preview interface in a photo-taking mode, and a first object is displayed on the first interface, and the first object is of a first size; receiving a second operation of the user on the first object in the first interface; in response to the second operation, determining a target position of the reflecting prism; after driving the reflecting prism to rotate to the target position, controlling the camera module to capture an image, and the first object is also displayed in the image captured by the camera module, and the first object is of a second size, and the second size is larger than the first size. In some embodiments, after driving the reflecting prism to rotate to the target position, the user inputs a shooting operation on the first interface, and the electronic device, in response to the shooting operation of the user, controls the camera module to capture an image.

[0007] It can be seen from the above content that: in the photo-taking mode, a first object is displayed on the camera preview interface displayed by the electronic device. When the user performs a second operation on the first object, the electronic device automatically responds to the second operation, determines the target position of the reflecting prism, drives the reflecting prism to rotate to the target position, and then controls the camera module to capture an image, and the size of the first object in the image is larger than the size of the first object initially displayed on the camera preview interface. Since the image is captured by the camera module rather than obtained by cropping other images, the clarity is relatively high. Therefore, it is possible to enable a user to specify an object to be tracked and shot, and the electronic device automatically shoots an image with high clarity.

[0008] Moreover, the user only inputs a second operation on the first object, the first object is also shown in the image captured by the camera module, and the size of the first object is larger than the size of the first object when the camera preview interface starts to be displayed. Without the user inputting other operations, it can also avoid the problem that the captured image is blurred due to the user moving the device for tracking.

[0009] In a second aspect, the present application provides a shooting method applied to an electronic device. The electronic device includes a camera module, and the camera module includes a rotatable reflecting prism. The shooting method includes: receiving a first operation of the user, in response to the first operation, starting a camera application and presenting a first interface, the first interface being a camera preview interface or a camera shooting interface in a video recording mode, and the first interface shows a first object; receiving a second operation of the user on the first object in the first interface; in response to the second operation, determining a target position of the reflecting prism; after driving the reflecting prism to rotate to the target position, controlling the camera module to shoot a video. In some embodiments, the first interface is a camera preview interface in a video recording mode. After driving the reflecting prism to rotate to the target position, the user inputs a shooting operation on the first interface, and the electronic device, in response to the user's shooting operation, controls the camera module to shoot a video.

[0010] It can be seen from the above content that: in the video recording mode, the camera preview interface displayed by the electronic device shows a first object. The user performs a second operation on the first object, and the electronic device automatically responds to the second operation, determines the target position of the reflecting prism, and after driving the reflecting prism to rotate to the target position, controls the camera module to shoot a video. The video captured by the camera module is of higher clarity compared to the video obtained by cropping other images. Therefore, the object specified by the user for tracking shooting can be realized, and the electronic device automatically shoots a video with high clarity.

[0011] Moreover, the user only inputs a second operation on the first object, and the camera module can automatically shoot a video without the user holding the device to move for tracking shooting, and it can also avoid the problem that the captured video is blurred due to the user moving the device for tracking.

[0012] Based on the second aspect, in a possible implementation manner, after driving the reflecting prism to rotate to the target position and controlling the camera module to shoot a video including the first object, it further includes: based on the camera module shooting a video including the first object, displaying a tracking screen window of the first object on the first interface. The size of the first object in the tracking screen window of the first object may be the same as the size of the first object displayed on the first interface, or may be larger than the size of the first object displayed on the first interface.

[0013] In the above possible implementation, in the video recording mode, after the reflecting prism is driven to the target position, the camera module captures a video including a first object. A tracking screen window of the first object can also be displayed on the first interface, so that the user can know the object being tracked and captured by the camera module through the tracking screen window. Moreover, the video displayed in the tracking screen window is captured by the camera module, without the need to crop other images, and the clarity of the video displayed in the tracking screen window can also be improved.

[0014] Based on the first aspect or the second aspect, in a possible implementation, the camera module further includes a braking component, which is used to lock the position of the reflecting prism. Before driving the reflecting prism to rotate to the target position, it further includes: driving the braking component to unlock; after driving the reflecting prism to rotate to the target position, it further includes: driving the braking component to lock.

[0015] In this possible implementation, the camera module includes a braking component. Before the reflecting prism rotates, the braking component can be unlocked, and after the reflecting prism rotates, the braking component locks, which can facilitate the reflecting prism to be quickly stabilized after reaching the position.

[0016] Based on the first aspect or the second aspect, in a possible implementation, the camera module further includes a focusing component, which is used to achieve focusing. The method further includes: during the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing on the first object.

[0017] In this possible implementation, the camera module includes a focusing component. During the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing on the first object can further ensure the clarity of the first object in the image or video captured by the camera module.

[0018] Based on the first aspect or the second aspect, in a possible implementation, before driving the focusing component to perform focusing, it further includes: compensating the focusing parameters based on the target position of the reflecting prism; during the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing includes: during the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing on the first object with the compensated focusing parameters.

[0019] Based on the first aspect or the second aspect, in a possible implementation, the camera module further includes an anti-shake component, which is used for optical anti-shake. Among them: during the process of the reflecting prism rotating to the target position, the anti-shake component does not operate; after the reflecting prism rotates to the target position, the anti-shake component operates to perform optical anti-shake.

[0020] In a possible implementation, during the process of the reflecting prism rotating to the target position, the anti-shake component does not operate. After the reflecting prism rotates to the target position, the anti-shake component operates for optical image stabilization, which can avoid the anti-shake component performing ineffective anti-shake during the process of the reflecting prism rotating to the target position and increase power consumption.

[0021] Based on the first aspect or the second aspect, in a possible implementation, the component that drives the reflecting prism to rotate and the anti-shake component are different components.

[0022] In the above possible implementation, the camera module of the electronic device can control the rotation of the reflecting prism to track the object with a large-angle rotation, and can also support the anti-shake with a small-angle rotation through the operation of the anti-shake component. The two do not interfere with each other, and high control accuracy and good control effect can be achieved.

[0023] Based on the first aspect or the second aspect, in a possible implementation, the camera module includes a rotatable reflecting prism, and the reflecting prism can rotate in a first plane and can also rotate in a second plane, and the first plane and the second plane intersect.

[0024] Based on the first aspect or the second aspect, in a possible implementation, in response to the first operation, determining the target position of the reflecting prism includes: in response to the second operation, determining the target position of the reflecting prism based on the position information of the second operation and the position information of the reflecting prism.

[0025] Based on the first aspect or the second aspect, in a possible implementation, in response to the first operation, determining the target position of the reflecting prism includes: in response to the second operation, determining the target position of the reflecting prism at the second moment based on the position information of the second operation, the position information of the reflecting prism, and the image data collected by the camera module at the first moment, and the second moment is the next moment of the first moment.

[0026] Based on the first aspect or the second aspect, in a possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for the rotation of the reflecting prism, and the kernel layer of the operating system includes a driver for the rotation of the reflecting prism; in response to the second operation, determining the target position of the reflecting prism includes: in response to the second operation, the algorithm module determines the target position of the reflecting prism; driving the reflecting prism to rotate to the target position includes: the control module for the rotation of the reflecting prism controls the operation of the driver for the rotation of the reflecting prism to drive the reflecting prism to rotate to the target position.

[0027] Based on the first aspect or the second aspect, in a possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: an anti-shake control module, and the kernel layer of the operating system includes an anti-shake driver; the algorithm module is further configured to generate anti-shake parameters, and the electronic device controls the anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver.

[0028] In a third aspect, the present application provides an electronic device, including: one or more processors, a memory, a camera module, and a display screen; the camera module includes a rotatable reflecting prism; the memory, the camera module, and the display screen are coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the shooting method described in any one of the first aspect or the second aspect.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which is specifically configured to implement the shooting method described in any one of the first aspect or the second aspect when the computer program is executed.

[0030] In a fifth aspect, the present application provides a computer program product, which causes a computer to execute the shooting method described in any one of the first aspect or the second aspect when the computer program product runs on the computer. Description of the Drawings

[0031] Figure 1 It is an interface diagram provided by an embodiment of the present application for a user to perform tracking an object and zooming operation in a shooting mode;

[0032] Figure 2 It is an interface diagram provided by an embodiment of the present application for a user to perform continuous tracking of an object operation in the main character mode;

[0033] Figure 3 It is a hardware structure diagram of the electronic device provided by an embodiment of the present application;

[0034] Figure 4 It is a software structure schematic diagram of the electronic device provided by an embodiment of the present application;

[0035] Figure 5 It is a flowchart for initializing and powering on the module provided by an embodiment of the present application;

[0036] Figure 6 It is a flowchart for implementing the operation of tracking an object provided by an embodiment of the present application;

[0037] Figure 7 It is a flowchart for implementing the focusing operation provided by an embodiment of the present application;

[0038] Figure 8Flowchart for powering down the module provided by the embodiments of this application. Detailed implementation manners

[0039] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification and appended claims of this application, the singular forms "a", "an", "the", "above", "said", "this" are also intended to include the expression form such as "one or more", unless clearly indicated to the contrary in the context. It should also be understood that in the embodiments of this application, "one or more" means one, two or more than two; "and / or" describes the association relationship of associated objects and means that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0040] Referring to "one embodiment" or "some embodiments" described in this specification means that specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] The "multiple" involved in the embodiments of this application means greater than or equal to two. It should be noted that in the description of the embodiments of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0042] Compared with professional cameras such as single-lens reflex cameras and mirrorless cameras, the cameras of electronic devices have two deficiencies: 1. The size of the lens photosensitive element (also called image sensor Sensor) of the camera module is relatively small, and the insufficient photosensitive ability results in low image clarity; 2. The lens of the camera module cannot be replaced. For the requirements of different shooting focal lengths in different scenarios, electronic devices need to be equipped with camera modules in various forms, such as: camera modules in various forms such as wide-angle, ultra-wide-angle, and telephoto.

[0043] An electronic device can refer to a mobile phone, a tablet computer, a personal digital assistant (PDA), a desktop, laptop, notebook computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, and a wearable device, etc.

[0044] Due to the limitations of the above two deficiencies, when an electronic device shoots a moving object at a medium or long distance, the common shooting methods are as follows:

[0045] Method 1: Use a wide-angle camera module to shoot an image. The size of the object in the image captured by the wide-angle camera module is small. Therefore, the image captured by the wide-angle camera module can be cropped, and the moving object can be captured by means of secondary composition. Since the sensor size of the camera module is small, the image quality of the secondarily cropped and composed image is low and the clarity is not high, making it difficult to meet the shooting requirements.

[0046] Method 2: Use a telephoto camera module to shoot an image, and the user holds the electronic device to follow the moving object for shooting. Since the field of view (FOV) of the telephoto camera module is small, it is difficult for the user to manually compose the picture and the target is easily lost. When the user holds the electronic device and moves, the requirement for the anti-shake ability of the electronic device is relatively high. A large movement amplitude of the user will affect the anti-shake effect and thus affect the image quality.

[0047] Therefore, for a moving object at a medium or long distance, an electronic device cannot achieve convenient user operation and capture high-quality images or videos. Based on this, the embodiments of the present application provide a shooting method, which can enable the user to only specify the object (human body or object) to be tracked, and the electronic device automatically realizes the tracking of the user-specified object, and captures high-quality images or videos through actions such as composition, framing, and imaging.

[0048] The shooting method provided by the embodiments of the present application can be applied to the photo-taking mode or video-recording mode of a camera. The user can input a zoom operation on the shooting object, and the camera module of the electronic device can automatically zoom in or out on the object. It can also be applied to the tracking shooting of a moving object in the video-recording mode of a camera. The user can also input a focus tracking operation on the shooting object, and the camera module of the electronic device can automatically track and shoot the moving object selected by the user.

[0049] The following combines Figure 1 and Figure 2 , and introduces two application scenarios of the shooting method provided by the embodiments of the present application.

[0050] Figure 1 Taking a mobile phone as an example, an interface diagram of the camera shooting image scenario is shown.

[0051] As Figure 1As shown in (a), when the mobile phone starts the camera application in the photo-taking mode, the mobile phone displays the camera preview interface 101. The camera preview interface 101 shows a picture captured by the mobile phone's camera module, and the person in the picture is relatively far from the mobile phone. If the user wants to zoom in and display the person 102, the user can click (such as single-clicking or double-clicking) on the person 102 to input the tracking object and perform a zoom operation. In response to the operation input by the user, the mobile phone rotates the reflecting prism of the camera module to achieve tracking of the person 102 and obtain an enlarged image of the person 102, as Figure 1 shown in (b). In the camera preview interface 103 displayed on the mobile phone, the person 102 is enlarged and displayed, and the other two people are not shown.

[0052] Figure 1 In the zooming scenario shown, the mobile phone performs tracking object and zooming operations centered on the object or area specified by the user, such as Figure 1 the person 102 in (a). That is, during the transition of the camera preview interface from Figure 1 (a) to Figure 1 (b), the person 102 is at the center of the picture.

[0053] In some embodiments, the camera can be configured with a switch for the function of tracking objects in the photo-taking mode. When the switch is in the on state, as shown in (a), if the user clicks on the person 102 to input the tracking object and perform a zoom operation, the mobile phone rotates the reflecting prism of the camera module in response to the operation input by the user to achieve tracking of the person 102 and obtain an enlarged image of the person 102. When the switch is in the off state, as shown in (a), if the user clicks on the person 102 to input the tracking object and perform a zoom operation, the mobile phone does not respond to this operation. Figure 1 Figure 1

[0054] Figure 2 Taking the mobile phone as an example, the interface diagram of the camera shooting a video scene is shown.

[0055] Figure 2 Figure 2 As Figure 2 shown in (a), when the mobile phone starts the protagonist mode in the video recording mode of the camera application, the mobile phone displays the camera preview interface 201. The camera preview interface 201 shows a picture captured by the mobile phone's camera module, and the picture includes three people. If the user wants to track and focus on shooting the person 202, the user can click (such as single-clicking or double-clicking) on the person 202 to input a continuous tracking object operation (also known as a focus tracking operation). In response to the operation input by the user, as Figure 2 shown in (b), the tracking picture 204 of the person 202 is displayed in a small window on the camera preview interface 203. As Figure 2As shown in (c), during the movement of the person 202, the mobile phone can continuously track the person 202 by rotating the reflecting prism of the camera module. In the camera preview interface 205 displayed on the mobile phone, the tracking screen 204 of the person 202 follows the person 202.

[0056] In some embodiments, the tracking screen window in the camera preview interface 205 can be operated by the user. Exemplarily, the user can drag the tracking screen to other positions in the camera preview interface.

[0057] In some embodiments, when the user selects a tracking object in the main character mode of the camera video recording mode, the user starts the camera to start shooting. During the shooting process, the mobile phone can Figure 2 as shown in (b) and (c), track and shoot the tracking object selected by the user, and display the tracking screen in the form of a small window. The tracking object can include a moving person, animal, or other object, etc.

[0058] In some embodiments, during the process of the camera shooting a video in the main character mode, the user can also Figure 2 select a tracking object in the manner shown in (a). The mobile phone then continuously tracks and shoots the object selected by the user.

[0059] In some embodiments, the camera is in other video recording modes, not the main character mode. The display screen of the mobile phone displays the camera preview interface, and can also Figure 2 as shown in (a), click on the person 202 to input an operation for continuously tracking the object. In response to the operation input by the user, Figure 2 as shown in (b), display the tracking screen 204 of the person 202 in the camera preview interface 203 in the form of a small window. As Figure 2 shown in (c), during the movement of the person 202, the mobile phone can continuously track the person 202 by rotating the reflecting prism of the camera module.

[0060] During the process of the camera shooting a video in other video recording modes, the user can also Figure 2 select a tracking object in the manner shown in (a). The mobile phone then continuously tracks and shoots the object selected by the user.

[0061] In some embodiments, the camera can be configured with a switch for the function of tracking objects in the video recording mode. When the switch is in the on state, the user Figure 2 as shown in (a), clicks on the person 202 to input an operation for continuously tracking the object. In response to the operation input by the user, Figure 2 as shown in (b), displays the tracking screen 204 of the person 202 in the camera preview interface 203 in the form of a small window. When the switch is in the off state, the user Figure 2As shown in (a), click on the person 102 to input an operation for continuous tracking of the object, and the mobile phone does not respond to this operation.

[0062] In some embodiments, the switch for the object tracking function in the video recording mode and the switch for the object tracking function in the photo taking mode can also be the same switch.

[0063] The shooting method provided by the embodiments of this application can be applied to the electronic device proposed in the foregoing content.

[0064] Taking a mobile phone as an example, Figure 3 This is a component example of an electronic device provided by the embodiments of this application. As Figure 3 shown, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display screen 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, and a camera module 190, etc.

[0065] It can be understood that the structure schematically shown in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0066] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0067] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0068] The internal memory 120 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0069] In some embodiments, the instructions stored in the internal memory 120 are for executing a shooting method. The processor 110 can, by executing the instructions stored in the internal memory 120, implement in the camera's photo-taking mode or video-recording mode, for a zoom operation input by the user on the shooting object, control the imaging module 190 to automatically zoom in or out on the object centered on the object, and can also implement in the camera's video-recording mode for a focus-tracking operation input by the user on the shooting object, control the imaging module 190 to automatically track and shoot the moving object selected by the user.

[0070] The electronic device can implement some shooting functions through an ISP, the camera 130, a video codec, a GPU, the display screen 140, an application processor, etc. This shooting function can be understood as a conventional shooting function.

[0071] The ISP is used to process the data fed back by the camera 130. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element, the optical signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene.

[0072] The camera 130 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device may include one or N cameras 130, where N is a positive integer greater than 1.

[0073] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0074] The video codec is used to compress or decompress digital videos. The electronic device can support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0075] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0076] The electronic device realizes the display function through the GPU, the display screen 140, and the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 140 and the application processor. The GPU is used for image rendering by performing mathematical and geometric calculations. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0077] The display screen 140 is used to display images, video interfaces, etc. The display screen 140 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oled, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N display screens 140, where N is a positive integer greater than 1.

[0078] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0079] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: The antenna 1 can be multiplexed as the diversity antenna of the wireless local area network.

[0080] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 110 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 and radiate it out.

[0081] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 may also receive the signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0082] The electronic device may implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.

[0083] The audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 170 may also be used for encoding and decoding audio signals.

[0084] The speaker 170A, also known as the "loudspeaker", is used to convert audio electrical signals into sound signals. The electronic device may listen to music or hands-free calls through the speaker 170A.

[0085] The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device answers a call or voice message, the receiver 170B may be placed close to the user's ear to listen to the voice.

[0086] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C with their mouth to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In some other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device can also be provided with three, four or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0087] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB jack or a 3.5mm open mobile terminal platform (OMTP) standard jack, or a cellular telecommunications industry association of the USA (CTIA) standard jack.

[0088] In the sensor module 180, the pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 140, the electronic device detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0089] The touch sensor 180B, also known as the "touch control device". The touch sensor 180B can be disposed on the display screen 140, and the touch sensor 180B and the display screen 140 form a touch screen, also known as the "touch control screen". The touch sensor 180B is used to detect touch operations acting on it or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 140. In some other embodiments, the touch sensor 180B can also be disposed on the surface of the electronic device, at a different position from the display screen 140.

[0090] The acceleration sensor 180C can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity and can also be used to identify the attitude of the electronic device.

[0091] The gyroscope sensor 180D can be used to determine the motion attitude of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180D.

[0092] In some embodiments, the electronic device implements some other shooting functions through the ISP, the camera module 190, the video codec, the GPU, the display screen 140, and the application processor, etc. Some other shooting functions may refer to: in the photo-taking mode or video-recording mode of the camera, automatically zooming in or out on the object centered on the object in response to an operation input by the user on the shooting object, or in the video-recording mode of the camera, automatically tracking and shooting a moving object selected by the user in response to an operation input by the user on the shooting object.

[0093] In some embodiments, the camera module 190 includes: a lens photosensitive element Sensor, an OIS IC, and a Scan IC. The camera module 190 may also include an AF IC, which is not shown in the figure.

[0094] In some embodiments, the camera module 190 is a telephoto module.

[0095] In the processor 110, the ISP is also used to process the data fed back by the camera module 190. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the Sensor, where the optical signal is converted into an electrical signal. The Sensor transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene.

[0096] The lens of the camera module 190 includes a reflecting prism, which is located in the incident light path of the Sensor and is used to reflect light to the Sensor. In some embodiments, the reflecting prism supports rotational movement in the xz plane. Exemplarily, the range of rotational movement is +-24 degrees, and it also supports rotational movement in the yz plane. Exemplarily, the range of rotational movement is +-7. The xz plane and the yz plane are two intersecting planes. The rotation of the reflecting prism can also be understood as the rotation of the lens of the camera module. The rotation of the lens of the camera module can enable the shooting object to be centered in the frame during the zooming process.

[0097] The Scan IC can be understood as a controller that controls the rotation of the reflecting prism. Exemplarily, the Scan IC can be a chip with logical control capabilities. The Scan motor receives control instructions from the Scan IC and can drive the reflecting prism to rotate. In some embodiments, there are two Scan motors, one driving the rotation of the reflecting prism in the xz plane and the other driving the rotation of the reflecting prism in the yz plane.

[0098] In some embodiments, the braking component can be used to lock and unlock the position of the camera module lens. Before the reflecting prism rotates, the braking component can be unlocked. After the reflecting prism rotation ends, the braking component locks, which can facilitate the camera module lens to quickly stabilize after reaching the position. In some embodiments, the braking component includes a braking IC and an actuator. The braking IC can receive control instructions from the Scan IC and control the actuator to lock or unlock. The braking IC can be understood as a controller, such as a chip with logical control capabilities.

[0099] In some embodiments, the OIS IC (Optical Image Stabilizer IC) is used for optical image stabilization. The OIS IC can control the OIS motor for image stabilization. In some embodiments, during the rotation of the reflecting prism driven by the Scan motor, the OIS motor does not perform image stabilization. Based on this, the Scan IC can control the OIS motor not to operate during the operation of the Scan motor and to operate after the Scan motor ends its operation. The OIS IC and the OIS motor can be collectively referred to as the image stabilization components.

[0100] In some embodiments, the AF motor is used for focusing. The OIS IC can perform algorithmic operation compensation and then control the AF motor for focusing. The AF motor can also be referred to as the focusing component.

[0101] The electronic device Figure 3 above the hardware components shown also runs an operating system. For example operating system, operating system, etc. Application programs such as camera applications can be installed and run on the operating system.

[0102] Figure 4 This is a schematic diagram of the software structure of the electronic device provided by the embodiments of the present application.

[0103] The layered architecture divides the operating system of an electronic device into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers, from top to bottom: the application layer (APP), the application framework layer (abbreviated as FWK), the system library, the Hardware Abstraction Layer (HAL), and the kernel layer.

[0104] The application layer may include a series of application packages. For example, Figure 4 as shown, the application packages may include applications such as the camera and the call.

[0105] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0106] For example, Figure 4 as shown, the application framework layer may include the window management service, the content provider, the phone manager, the view system, the resource manager, and the camera service, etc.

[0107] The window management service is used to manage window programs. The window management service can implement controls such as adding, deleting, displaying, and hiding windows. The content provider is used to store and obtain data, and make this data accessible to applications. The data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The phone manager is used to provide the communication functions of the electronic device. For example, the management of call states (including answering, hanging up, etc.). The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and so on.

[0108] In some embodiments, the camera service, also known as the camera framework, is used to receive requests such as video recording requests and image capture requests from the camera application, and at the same time maintain the business logic of the internal flow of video recording requests, image capture requests, etc., and send the final result of the request to the camera application.

[0109] The Android Runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. In some embodiments of the present application, the cold start of an application runs in the Android runtime, and the Android runtime thus obtains the optimized file status parameters of the application. Furthermore, the Android runtime can determine whether the optimized file is outdated due to system upgrade based on the optimized file status parameters and return the judgment result to the application management and control module.

[0110] The core libraries consist of two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android.

[0111] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0112] The system libraries can include multiple functional modules. For example: the surface manager, Media Libraries, 3D graphics processing libraries (such as OpenGL ES), and 2D graphics engines (such as SGL), etc.

[0113] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The Media Libraries support the playback and recording of various common audio and video formats, as well as static image files, etc. The Media Libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0114] The hardware abstraction layer is a software located between the operating system kernel and the hardware circuit, usually used to abstract the hardware to achieve the interaction between the operating system and the hardware circuit at the logical layer.

[0115] In some embodiments, the hardware abstraction layer includes the Camera Hal, and the Camera Hal includes an image link Pipeline and CamxHal3. The Camera Hal is mainly responsible for the construction, connection, and interaction control of all modules in the image link. CamxHal3 is used to interact with the upper-layer module to open or close the Camera Hal and is used to transmit data to the Camera Hal.

[0116] In some embodiments, the image link pipeline includes: a module control module, an ISP control module, and an algorithm module; the module module is responsible for processing camera module management, and the module module includes a Sensor control module, a Scan control module, an OIS control module, and a motor Actuator control module.

[0117] In some embodiments, the Sensor control module is used to interface with the Sensor, and the Sensor control module can control the operation of the Sensor through Sensor drive and Sensor Firmware.

[0118] In some embodiments, the Scan control module can be referred to as the control module of the reflecting prism, and is used to interface with the Scan motor to realize the rotation of the reflecting prism. The Scan control module can control the operation of the Scan motor through Scan drive and Scan Firmware.

[0119] In some embodiments, the OIS control module can be referred to as the anti-shake control module, and is used to implement anti-shake processing during the rotation of the camera module. The OIS control module can control the operation of the OIS motor through OIS drive and OIS Firmware. In some other embodiments, the Scan control module can control the operation of the OIS motor through Scan drive, Scan Firmware, and controlling OIS Firmware.

[0120] In some embodiments, the Actuator control module is used for AF focusing during the rotation of the camera module. The Actuator control module can control the operation of the AF motor through AF drive, OIS Firmware, and AF Firmware.

[0121] The ISP control module is used to receive the image data sent by the Sensor, process the image data, and then send it to the algorithm module. In some embodiments, the Sensor refers to the Sensor of the camera module 190.

[0122] The algorithm module is used to receive the image data sent by the ISP control module, analyze the image data, obtain the operation strategy of the lower-level module, and feedback it to the modules in the module control module. In some embodiments, the algorithm module obtains the rotation strategy of the reflecting prism and feedbacks it to the Scan control module.

[0123] The kernel layer is the layer between the hardware and the software. The kernel layer undertakes the commands of the control layer, supports the conversion of control commands into actual device operation instructions, and the operation instructions include power-on and power-off control, device mode register instruction generation and distribution. The register control commands of the driver layer will be transmitted to the firmware layer through the device bus. The kernel layer at least includes Sensor drive, Scan drive, OIS drive, and AF drive, etc.

[0124] The firmware layer refers to the software systems integrated inside the components of the camera module, including Sensor Firmware, Scan Firmware, OIS Firmware, AF Firmware, and Brake Firmware. In some embodiments, the firmware layer may not include Brake Firmware.

[0125] In some embodiments, Sensor Firmware is a software system running inside the Sensor, responsible for sensor exposure and image data output. Scan Firmware is a software system running inside the Scan IC, used to control the rotation of the camera module lens, responsible for converting the codes sent from the upper layer into control signals, thereby driving the rotation of the camera module lens to achieve the focus tracking and viewfinder aiming goals. Brake Firmware is a software system running inside the Brake IC, used for locking the position of the camera module to facilitate quick stabilization after the camera module reaches the target position. OIS Firmware is a software system running inside the OIS IC, used for anti-shake. AF Firmware is a software system running inside the AF IC, used for focus tracking, responsible for converting the target position sent from the upper layer into motor current to drive the AF motor for focusing.

[0126] Under the above five-layer architecture, the electronic device is further provided with a hardware layer, which may include the electronic device hardware components proposed above. Exemplarily, Figure 4 shows a Sensor, Scan IC, OIS IC, AF motor, and brake components.

[0127] It should be noted that although the embodiments of this application are described by taking the system as an example, its basic principle is equally applicable to electronic devices based on operating systems such as

[0128] In the embodiments of this application, the functions of controlling the rotation of the reflecting prism and implementing the anti-shake function of the electronic device are independent of each other. From the perspective of the hardware structure: the function of controlling the rotation of the reflecting prism is realized by the Scan motor, and the anti-shake function is realized by the OIS motor, and the two are independent of each other; from the perspective of software control: the Scan control module controls the operation of the Scan motor through Scan drive and Scan Firmware, and the OIS control module controls the operation of the OIS motor through OIS drive and OIS Firmware, and the two are also independent of each other.

[0129] In this way, it can be ensured that the camera module of the electronic device can track an object with a large-angle rotation by controlling the rotation of the reflecting prism, and can also support anti-shake with a small-angle rotation through the operation of the OIS motor. The two do not interfere with each other, and high control precision and good control effect can be achieved.

[0130] The electronic device is configured with a braking component, which includes a brake IC and an actuator. The brake IC can receive the control instruction of the Scan IC to control the actuator to lock or unlock, so as to unlock before the reflecting prism rotates and lock after the reflecting prism rotates in place to achieve rapid braking and stability.

[0131] The Actuator control module can control the operation of the AF motor through AF drive, OIS Firmware and AF Firmware. The OIS Firmware can realize the dynamic compensation of the focus parameters for the position of the Scan motor.

[0132] Before the electronic device executes the shooting method provided in this embodiment, it can, according to the operation of the user to open the camera application, perform Figure 4 the initialization of the module that displays the associated shooting function. The module that displays the associated shooting function includes: Figure 4 the displayed Sensor control module, Scan control module, OIS control module, Actuator control module, Sensor drive, Scan drive, OIS drive, AF drive, Sensor Firmware, Scan Firmware, OIS Firmware, AF Firmware, brake Firmware, Sensor, Scan IC, OIS IC, AF motor, and braking component.

[0133] Among them, the Sensor control module, Sensor drive, Sensor Firmware, and Sensor are used to obtain image data. After the module control module is created, it can execute the initialization and power-on processes of the Sensor control module, Sensor drive, Sensor Firmware, and Sensor.

[0134] The Actuator control module, AF drive, AF Firmware, and AF motor are used to perform focusing. After the module control module is created, it can execute the initialization and power-on processes of the Actuator control module, AF drive, AF Firmware, and AF motor.

[0135] The OIS control module, OIS driver, OIS Firmware, and OIS IC are used to perform anti-shake. After the module control module is created, the initialization and power-on processes of the OIS control module, OIS driver, OIS Firmware, and OIS IC can be executed.

[0136] The Scan control module, Scan driver, Scan Firmware, and Scan IC are used to implement the rotation of the reflecting prism. After the module control module is created, the initialization and power-on processes of the Scan control module, Scan driver, Scan Firmware, and Scan IC can be executed.

[0137] In some embodiments, after the module control module is created, the initialization and power-on processes can be executed in the order proposed above, that is, the module for obtaining image data first, then the module for performing focusing, then the module for performing anti-shake, and finally the module for implementing the rotation of the reflecting prism. The initialization and power-on processes of the module for obtaining image data, the module for performing focusing, and the module for performing anti-shake can refer to the conventional technology, and this embodiment will not introduce them in detail.

[0138] The following combines Figure 5 to introduce the initialization and power-on processes of the module for implementing the rotation of the reflecting prism.

[0139] As Figure 5 shown, the initialization and power-on processes of the module for implementing the rotation of the reflecting prism provided by the embodiments of the present application include:

[0140] S501. The user starts the camera application.

[0141] The user starts the camera application by inputting a camera start operation. The electronic device responds to the user's operation and starts the camera application. When the camera application is started, the camera is controlled to collect images and provide them to the camera application, and the camera application can display the images collected by the camera on the display screen to form a camera preview interface. Exemplarily, Figure 1 in (a) shows the camera preview interface when the camera application is turned on and in the photo-taking mode.

[0142] In some embodiments, the operations for the user to turn on the camera application may include: clicking the camera application icon on the display screen, pressing a specific physical button, inputting voice, and air gestures, etc.

[0143] S502. The camera application connects to the camera service.

[0144] When the camera application is turned on by the user, the camera application connects to the camera service. As described in the content of the software structure of the electronic device above, the camera service is located in the application framework layer.

[0145] S503. The camera service opens the camera Hal.

[0146] After the camera service is connected to the camera application, the camera Hal can be opened. As described in the content of the software structure of the foregoing electronic device, the camera Hal is located in the hardware abstraction layer.

[0147] S504. CamxHal3 creates a Pipeline.

[0148] After the camera Hal is opened, CamxHal3 can create an image link Pipeline.

[0149] In some embodiments, the image link Pipeline includes: a module control module, an ISP control module, and an algorithm module. Therefore, CamxHal3 creating a Pipeline can be understood as: CamxHal3 creates the module control module, ISP control module, and algorithm module of the image link Pipeline. And the sub-modules in the module control module are also created.

[0150] S505. The Pipeline initializes the algorithm module.

[0151] After CamxHal3 creates the algorithm module through the Pipeline, the Pipeline can initialize the algorithm module. The Pipeline initializing the algorithm module can be understood as configuring the initialization parameters of the algorithm module.

[0152] S506. The Pipeline creates the module control module.

[0153] The module control module includes: a Sensor control module, a Scan control module, an OIS control module, and an Actuator control module. The Pipeline creating the module control module can also be understood as creating the Sensor control module, Scan control module, OIS control module, and Actuator control module of the module control module.

[0154] S507. The module control module creates sub-modules, and the sub-modules include the Scan control module.

[0155] After the Pipeline creates the module control module, the module control module creates sub-modules, and the sub-modules include: a Sensor control module, a Scan control module, an OIS control module, and an Actuator control module. This embodiment introduces the initialization and power-on process of the module for realizing the rotation of the reflecting prism. Therefore, this step only reflects that the module control module creates the Scan control module.

[0156] S508. The Scan control module triggers the power-on of the Scan Driver.

[0157] After the Scan control module is created, it can trigger the power-on of the Scan Driver.

[0158] In some embodiments, the Scan control module executes Acquire Device to trigger the power-on of the Scan Driver, and the ScanDriver is the Scan driver.

[0159] S509. The Scan Driver triggers the power-on and initialization of the Scan IC.

[0160] After the Scan Driver is powered on, the Scan Driver can control the power-on of the Scan IC. After the Scan IC is powered on, the link between the ScanDriver and the Scan IC is connected, such as the I2C bus. The Scan Driver can send initialization instructions to the Scan IC through the link to control the Scan IC to perform initialization operations, such as reset.

[0161] In some embodiments, after the Scan IC completes power-on and initialization, it can return its own execution result to the Scan Driver. The Scan Driver can return its own execution result to the Scan control module. The Scan control module can return its own execution result to the module control module. The module control module and the algorithm module can return their own execution results to the Pipeline. The Pipeline returns its own execution result to CamxHal3. CamxHal3 returns its own execution result to the camera service. The camera service returns its own execution result to the camera application.

[0162] After the initialization of the modules associated with the associated shooting function is completed, the electronic device can execute the shooting method provided by the embodiments of the present application to shoot images or videos. Among them, the shooting method of the embodiments of the present application involves tracking object operations and focusing operations. The following combines Figure 6 to introduce the tracking object process of the electronic device, and combines Figure 7 to introduce the focusing process of the electronic device.

[0163] The tracking object process and the focusing process of the electronic device can be understood as being carried out in parallel. Therefore, the following uses two embodiments for introduction.

[0164] As Figure 6 shown, the tracking object process of the electronic device provided by the embodiments of the present application includes:

[0165] S601. The camera application receives the user's input operation.

[0166] The operations input by the user include: tracking object and zooming operations and continuous tracking object operations.

[0167] In some embodiments, the user launches the camera application, and the camera application displays a camera preview interface on the display screen. The user can select the working mode of the camera and input operations such as tracking object operations in the camera preview interface. Figure 1 It shows that the user inputs a tracking object and a zoom operation in the camera preview interface. Figure 2 It shows that the user inputs a continuous tracking object operation in the camera preview interface.

[0168] In some embodiments, when the camera application starts shooting, during the video shooting process, the camera application displays a video shooting interface, and the user can also input tracking object operations in the video shooting interface.

[0169] In some embodiments, when the camera application receives the user's input operation, it can also obtain the user's operation position.

[0170] As Figure 1 shown in (a) below, the user clicks on an object in the camera preview interface to input a tracking object and a zoom operation. The camera application receives the user's operation and obtains the user's operation position. As Figure 2 shown in (a) below, the user clicks on an object in the camera preview interface to input a continuous tracking object operation. The camera application can also receive the user's operation and obtain the user's operation position. In some embodiments, the user's operation position can be understood as the coordinates of the operation position where the user inputs an image in the camera preview interface.

[0171] S602. The camera application sends a request message to the camera service, and this request carries the coordinate information of the user's operation position.

[0172] After the camera application receives the tracking object operation input by the user and obtains the user's operation position, it can send a request message to the camera service. This request message carries the position information of the user's operation position, that is, it carries the coordinate information of the user's operation position. The request message sent by the camera application to the camera service is used to request the execution of the tracking object process.

[0173] S603. The camera service sends a request message to CamxHal3, and this request carries the coordinate information of the user's operation position.

[0174] After the camera service receives the request message sent by the camera application, it sends this request message to CamxHal3.

[0175] S604. CamxHal3 sends a request message to the Pipeline, and this request carries the coordinate information of the user's operation position.

[0176] After CamxHal3 receives the request message sent by the camera service, it sends this request message to the Pipeline.

[0177] In some embodiments, after the user inputs a tracking object and a zoom operation, and CamxHal3 receives a request message sent by the camera service, it sends the request message to the Pipeline corresponding to the imaging module 190.

[0178] It should be noted that when the user turns on the camera, the camera 130 of the electronic device (which can be understood as the main camera) runs to collect images, and the functional modules of the Pipeline corresponding to the camera 130 run to enable the display screen to display the camera preview interface, and the camera preview interface displays the images captured by the camera 130.

[0179] When the user inputs a tracking object and a zoom operation in the camera preview interface, the functional modules of the Pipeline corresponding to the imaging module 190 are driven to run, and the images captured by the imaging module 190 can be displayed on the display screen to maintain the camera preview interface. The imaging module 190 is a telephoto module, and the main body in the images captured by the imaging module 190 is large, and the image of the shooting object can be enlarged. In some embodiments, the Pipeline corresponding to the camera 130 may run or stop running, which is not limited.

[0180] In other embodiments, after the user inputs a continuous tracking object operation, and CamxHal3 receives a request message sent by the camera service, it sends the request to the Pipeline corresponding to the imaging module 190.

[0181] The same as the foregoing content, when the user turns on the camera, the camera 130 runs to collect images to obtain the camera preview interface. When the user inputs a continuous tracking object operation, the functional modules of the Pipeline corresponding to the imaging module 190 are driven to run, and the images captured by the imaging module 190 can be displayed in the tracking screen window of the camera preview interface.

[0182] S605. The Pipeline calls the algorithm module to process the request message.

[0183] After the Pipeline receives the request message, it calls the algorithm module to process the request message. In some embodiments, the Pipeline may send the request message to the algorithm module to implement calling the algorithm module to process the request message.

[0184] S607. The algorithm module obtains the target position of the reflecting prism based on the coordinates and the position information of the reflecting prism.

[0185] In some embodiments, the algorithm module stores data such as the position information of the reflecting prism and the state of the braking component. In some embodiments, the position information of the reflecting prism may be the detection value of the TMR sensor, simply referred to as TMR data.

[0186] In some embodiments, the user inputs a tracking object and a zoom operation. After the algorithm module receives the request message, it obtains the coordinate information carried in the request message and the position information of the reflecting prism. Based on the coordinate information carried in the request and the position information of the reflecting prism, it obtains the position reached after the reflecting prism rotates, that is, the target position. In some embodiments, the target position may be code.

[0187] In some embodiments, the algorithm module may store calibration data, which includes the rotation angle of the reflecting prism corresponding to the coordinates of the user operation. The algorithm module combines the position information and the rotation angle of the reflecting prism to obtain the target position of the reflecting prism, which can be understood as the position that the reflecting prism needs to reach after rotation.

[0188] It should be noted that the algorithm module analyzes the coordinate information of the zoom operation and the position information of the reflecting prism through step S607 to obtain the target position of the reflecting prism. This target position can be understood as ensuring that after the object of the zoom operation performs the zoom operation, it is located in the central area of the screen. Therefore, the Scan Firmware controls the Scan motor to rotate to drive the reflecting prism to rotate until the reflecting prism rotates to the target position, so that after the user performs a zoom operation on an object, in the image displayed on the display screen, the object automatically locates in the center of the screen without the need for user operation or image cropping. Moreover, without user operation, it can also avoid the problem of blurred images caused by the user moving the device for tracking.

[0189] In other embodiments, the user inputs a continuous tracking object operation. After the algorithm module receives the request message, it obtains the coordinate information and the position information of the reflecting prism carried in the request message, and also obtains the image data collected by the camera. Based on the image data, the coordinate information, and the position information of the reflecting prism, it obtains the target position of the reflecting prism.

[0190] It should be noted that when the user inputs a continuous tracking object operation, the electronic device needs to continuously track and photograph the object specified by the operation. Therefore, the algorithm module can continuously obtain the target position of the reflecting prism to drive the reflecting prism to continuously rotate to track the object. The algorithm module can determine the user's operation object based on the coordinate information of the user operation position, and determine the moving direction of the object in the image data collected by the camera. The algorithm module combines the moving direction of the object and the current position information of the reflecting prism to determine the target position of the next rotation of the reflecting prism. The target position of each rotation of the reflecting prism can be understood as the position that can ensure that the camera captures the user's operation object, so that the electronic device can automatically follow the moving object for shooting, and without the need for user operation, it can also ensure the clarity of the captured video.

[0191] The camera for capturing images can be the main camera, i.e., camera 130, or the camera module 190; alternatively, for the moment when the user inputs an operation for continuously tracking an object, the camera can be the main camera, and after the algorithm module obtains the target position of the reflecting prism corresponding to this moment, the camera becomes the camera module 190.

[0192] In some embodiments, the target position of the reflecting prism obtained by the algorithm module can be written into the data pool metadataPool for other modules to monitor changes in the data in the data pool. In some embodiments, the metadata Pool includes multiple metadata, and each metadata is used to store numerical values for different functions. The algorithm module writes the target position of the reflecting prism into the metadata corresponding to the tracking object function.

[0193] In some embodiments, the algorithm module also stores the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake parameters of the OIS motor are the detection values of the HALL sensor, abbreviated as HALL data. After the algorithm module receives a request, it can also obtain the indication value of the anti-shake strategy of the OIS motor according to the coordinates carried in the request and the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake strategy includes the operating mode and operating parameters of the OIS, etc.

[0194] In some embodiments, the indication value (also referred to as the anti-shake parameter) of the anti-shake strategy of the OIS motor obtained by the algorithm module can also be written into the data pool metadata Pool for other modules to monitor changes in the data in the data pool. In some embodiments, the indication value of the anti-shake strategy of the OIS motor obtained by the algorithm module is written into the metadata corresponding to the anti-shake function.

[0195] S608. The algorithm module sends the target position of the reflecting prism to the Scan control module.

[0196] In some embodiments, after the algorithm module obtains the target position of the reflecting prism, it can directly send it to the Scan control module. In some other embodiments, the algorithm module writes the target position of the reflecting prism into the metadata, and the Scan control module can monitor this metadata to obtain the target position of the reflecting prism written by the algorithm module.

[0197] S609. The Scan control module sends the target position of the reflecting prism to the Scan driver.

[0198] In some embodiments, before the Scan control module sends the target position of the reflecting prism to the Scan driver, it can verify the target position of the reflecting prism to check whether the target position of the reflecting prism is a legal position. If the verification passes, it sends the target position of the reflecting prism to the Scan driver.

[0199] S610. The Scan driver sends a first control instruction to the Scan Firmware.

[0200] This first control instruction is used to control the Scan Firmware to perform brake unlocking, reflecting prism rotation, and brake locking.

[0201] After the Scan driver receives the target position of the reflecting prism, it can control the Scan Firmware to perform brake unlocking, reflecting prism rotation, and brake locking.

[0202] In some embodiments, the Scan driver converts the target position of the reflecting prism into a register instruction, and the register instruction is transmitted to the Scan Firmware through a bus such as I2C. The Scan Firmware executes the following steps S611 and S612 according to the register instruction.

[0203] S611. The Scan Firmware sends a second control instruction to the OIS Firmware.

[0204] This second control instruction is used to make the Scan Firmware control the OIS Firmware to enter HOLD ON at the start of rotation; after the rotation ends, the Scan Firmware controls the OIS Firmware to enter OIS ON.

[0205] During the rotation of the reflecting prism, the anti-shake execution of the electronic device is ineffective anti-shake. Therefore, the electronic device may not perform anti-shake. After the rotation of the reflecting prism ends, the electronic device performs anti-shake. Based on this, the Scan Firmware controls the OIS Firmware to enter HOLD ON at the start moment of the rotation of the reflecting prism until it controls the OIS Firmware to enter OIS ON at the end moment of the rotation of the reflecting prism. Among them, HOLD ON can be understood as an anti-shake pause state, and OIS ON can be understood as an anti-shake enabled state.

[0206] S612. The Scan Firmware performs the rotation of the reflecting prism.

[0207] In some embodiments, before the Scan Firmware performs the rotation of the reflecting prism, it first performs brake unlocking, and after the rotation of the reflecting prism ends, it performs brake locking again.

[0208] In some embodiments, the Scan Firmware controls the brake component to unlock and lock through the brake Firmware.

[0209] In some embodiments, the Scan Firmware controls the rotation of the Scan motor to drive the rotation of the reflecting prism until the reflecting prism rotates to the target position.

[0210] For Figure 1 the tracking object shown and the zoom operation, after the reflecting prism rotates to the target position, the display screen displays Figure 1 the interface shown in (b) of

[0211] For Figure 2 the continuous tracking object operation shown, the camera of the electronic device captures a frame of image. After the reflecting prism rotates to the target position for this frame of image, the display screen displays the tracking screen of this frame of image, as shown in Figure 2 (b) or (c) of

[0212] The execution order of step S611 and step S612 is not limited to Figure 6 shown. In some embodiments, the ScanFirmware can execute step S611 and step S612 in parallel.

[0213] S613. During the rotation, the OIS Firmware enters HOLD ON. After the rotation ends, the OIS Firmware enters OIS ON.

[0214] In some embodiments, during the rotation, the OIS Firmware enters the HOLD ON state. For the control instructions issued by the OIS driver, only cache them and do not execute them directly. After the reflecting prism rotates to the target position, the Scan Firmware transfers the TMR data and the OIS ON instruction to the OIS Firmware. The OIS Firmware enters the OIS ON state and updates the anti-shake parameters according to the TMR data, controlling the OIS motor to restore the state before the HOLD ON state or restore to the cached state.

[0215] In some embodiments, the Scan Firmware can periodically report the TMR data to the Scan driver. The Scan driver reports the TMR data to the algorithm module through the Scan control module, and the algorithm module updates the storage.

[0216] In some embodiments, the OIS Firmware can also periodically report the HALL data to the OIS driver. The OIS driver reports the HALL data to the algorithm module through the OIS control module, and the algorithm module updates the storage.

[0217] As Figure 7 shown, the focusing process of the electronic device provided by the embodiments of the present application includes:

[0218] S701. The camera application receives the user's input operation.

[0219] For the specific implementation of step S701, refer to the content of step S601 in the foregoing embodiment, which will not be elaborated here.

[0220] S702. The camera application sends a request message to the camera service, and the request carries the coordinate information of the user operation position.

[0221] For the specific implementation of step S702, refer to the content of step S602 in the foregoing embodiment, which will not be elaborated here.

[0222] S703. The camera service sends a request message to CamxHal3, and the request carries the coordinate information of the user operation position.

[0223] For the specific implementation of step S703, refer to the content of step S603 in the foregoing embodiment, which will not be elaborated here.

[0224] S704. CamxHal3 sends a request message to the Pipeline, and the request carries the coordinate information of the user operation position.

[0225] For the specific implementation of step S704, refer to the content of step S604 in the foregoing embodiment, which will not be elaborated here.

[0226] S705. The Pipeline calls the algorithm module to process the request message.

[0227] For the specific implementation of step S705, refer to the content of step S605 in the foregoing embodiment, which will not be elaborated here.

[0228] S706. The algorithm module obtains the focusing parameters.

[0229] The focusing parameters can be understood as the operating parameters of the AF motor. In some embodiments, the algorithm module can obtain the focusing parameters based on the contrast focusing algorithm or the phase focusing algorithm. In other embodiments, the algorithm module obtains one set of focusing parameters based on the contrast focusing algorithm and another set of focusing parameters based on the phase focusing algorithm, and the algorithm module combines the two sets of focusing parameters to obtain the focusing parameters to be sent.

[0230] S707. The algorithm module sends the focusing parameters to the Actuator control module.

[0231] In some embodiments, after the algorithm module obtains the focusing parameters, it can directly send the focusing parameters to the Actuator control module.

[0232] In some other embodiments, after the algorithm parameters obtain the focusing parameters, the focusing parameters can be written into the metadata corresponding to the focusing function in the metadata Pool. The Actuator control module monitors the data change of the metadata and obtains the written focusing parameters. In some embodiments, the focusing parameters can be codes.

[0233] S708. The Actuator control module sends the focusing parameters to the AF driver.

[0234] S709. When the AF driver receives the interrupt signal, it controls the OIS Firmware to perform focusing according to the focusing parameters, and the interrupt signal indicates that the image exits the frame.

[0235] After the AF driver receives the focusing parameters, it does not immediately control the AF Firmware to perform focusing through the OIS Firmware. It can wait for the interrupt signal corresponding to the image exiting the frame. After the AF driver receives this interrupt signal, it can perform focusing according to the focusing parameters.

[0236] S710. The OIS Firmware compensates the focusing parameters according to the target position of the reflecting prism.

[0237] In some embodiments, the OIS Firmware stores the corresponding relationship between the target positions of multiple reflecting prisms and the compensation values of the focusing parameters. The OIS Firmware can determine the compensation value corresponding to the target position of the reflecting prism based on this corresponding relationship, and compensate the focusing parameters based on this compensation value.

[0238] In some embodiments, there are two AF motors. The OIS Firmware compensates the focusing parameters with the compensation values of the two AF motors according to the target position of the reflecting prism, and obtains the compensated focusing parameters of the two AF motors.

[0239] In some embodiments, the Scan Firmware periodically sends the position information of the reflecting prism to the OIS Firmware, and the OIS Firmware can use the position information of the reflecting prism sent by the Scan Firmware at the latest moment as the target position of the reflecting prism.

[0240] S711. The OIS Firmware controls the AF Firmware to perform focusing according to the compensated focusing parameters.

[0241] In some embodiments, when the OIS Firmware receives the focusing parameters, it may not perform step S710, but transmit the focusing parameters transparently to the AF Firmware, and the AF Firmware performs focusing according to the focusing parameters.

[0242] S712. The AF Firmware performs autofocus according to the compensated autofocus parameters.

[0243] The AF Firmware controls the operation of the AF motor to achieve autofocus according to the compensated autofocus parameters. In some embodiments, there are two AF motors, and the AF Firmware controls the operation of the two AF motors to achieve autofocus according to the compensated autofocus parameters.

[0244] After the camera application is closed or a working module of the camera application is exited, the aforementioned module for the associated shooting function can be exited. The following embodiments of the present application will be combined Figure 8 , taking the closing of the camera application as an example, to introduce the exit process of the module for controlling the rotation of the reflecting prism. After the camera is closed or the working mode of the camera application is exited, other modules of the associated shooting function will also execute the exit process, and this exit process can refer to the conventional technology, and this embodiment will not introduce it in detail either.

[0245] It should be noted that the module for realizing the rotation of the reflecting prism will be started and run in the zoom working mode and the autofocus tracking working mode of the camera application. Therefore, after the zoom working mode and the autofocus tracking working mode of the camera application are exited, the module for realizing the rotation of the reflecting prism can be controlled to exit the operation.

[0246] As Figure 8 shown, the exit process of the module for realizing the rotation of the reflecting prism includes:

[0247] S801. The user closes the camera application.

[0248] The user closes the camera application by inputting an operation to close the camera application. In some embodiments, the operation to close the camera application may include performing a return operation on the camera preview interface of the camera application, closing the camera application in the multitasking management interface, turning off the electronic device, etc.

[0249] S802. The camera application closes the camera service.

[0250] After the user inputs an operation to close the camera application, the camera application may first close the camera service before closing.

[0251] S803. The camera service closes the camera Hal.

[0252] The camera service is closed under the control of the camera application, and the camera service may first close the camera Hal before closing.

[0253] S804. CamxHal3 destroys the Pipeline.

[0254] If the camera HAL is input with a shutdown command by the camera service, the pipeline can be destroyed through CamxHal3. The destruction of the pipeline can be understood as performing the following steps S805 and S806 before the pipeline is destroyed.

[0255] It should be noted that the image link pipeline includes: a module control module, an ISP control module, and an algorithm module. Before the pipeline is destroyed, the ISP control module can also be controlled to exit.

[0256] S805. Control the algorithm module of the pipeline to exit.

[0257] S806. Control the module control module of the pipeline to exit.

[0258] S807. The module control module sends a Scan exit command to the Scan control module.

[0259] Before the module control module exits, it sends a Scan exit command to the Scan control module to control the Scan control module to power down and initiate the process for the modules in the driver layer and the components in the hardware layer.

[0260] S808. The Scan control module sends a Scan exit command to the Scan Driver and triggers power down.

[0261] After receiving the exit command, the Scan control module can send a Scan exit command to the Scan Driver to control the Scan Driver to exit.

[0262] S809. The Scan Driver controls the Scan IC to exit and power down.

[0263] Before the Scan Driver exits, it can control the Scan IC to exit and power down. In some embodiments, the Scan IC exiting and powering down includes: centering the Scan motor corresponding to the Scan IC, the brake actuator performing brake locking, and powering down the Scan IC. Among them, centering the Scan motor corresponding to the Scan IC can be understood as restoring to the initial position.

[0264] In some embodiments, after the Scan IC finishes execution and powers down, it can return its execution result to the Scan Driver. The Scan Driver can return its execution result to the Scan control module. The Scan control module can return its execution result to the module control module. The module control module and the algorithm module can return their execution results to the Pipeline. The Pipeline returns its execution result to CamxHal3. CamxHal3 returns its execution result to the camera service. The camera service returns its execution result to the camera application.

[0265] Another embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are run on a computer or a processor, the computer or the processor is caused to execute one or more steps in any of the above methods.

[0266] The computer-readable storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0267] Another embodiment of the present application also provides a computer program product containing instructions. When the computer program product runs on a computer or a processor, the computer or the processor is caused to execute one or more steps in any of the above methods.

Claims

1. A shooting method, characterized in that, Applied to an electronic device, the electronic device includes a camera module, the camera module includes a rotatable reflecting prism, and the shooting method includes: Receiving a first operation of a user; In response to the first operation, starting a camera application and presenting a first interface, where the first interface is a camera preview interface or a camera shooting interface, and a first object with a first size is displayed on the first interface; Receiving a second operation of the user on the first object in the first interface; In response to the second operation, determining a target position of the reflecting prism; Driving the reflecting prism to rotate to the target position; After the reflecting prism rotates to the target position, controlling the camera module to capture an image or video including the first object; where the first object in the image captured by the camera module is a second size, and the second size is larger than the first size.

2. The photographing method according to claim 1, wherein After the reflecting prism rotates to the target position and the camera module is controlled to capture a video including the first object, it further includes: Based on the video captured by the camera module including the first object, displaying a tracking screen window of the first object on the first interface.

3. The shooting method according to claim 1 or 2, characterized in that The camera module further includes a braking component, Before driving the reflecting prism to rotate to the target position, it further includes: driving the braking component to unlock; After driving the reflecting prism to rotate to the target position, it further includes: driving the braking component to lock.

4. The photographing method according to any one of claims 1 to 3, characterized in that, The camera module further includes a focusing component, and the method further includes: During the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing on the first object.

5. The photographing method according to claim 4, wherein Before driving the focusing component to perform focusing, it further includes: Based on the target position of the reflecting prism, compensating the focusing parameters; During the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing includes: During the process of the reflecting prism rotating to the target position, driving the focusing component to perform focusing on the first object with the compensated focusing parameters.

6. The photographing method according to any one of claims 1 to 5, characterized in that, The camera module further includes an anti-shake component, where: during the process of the reflecting prism rotating to the target position, the anti-shake component does not operate, and after the reflecting prism rotates to the target position, the anti-shake component operates to perform optical anti-shake.

7. The photographing method according to claim 6, characterized in that, The component driving the reflecting prism to rotate and the anti-shake component are different components.

8. The photographing method according to any one of claims 1 to 7, characterized in that, The camera module includes a rotatable reflecting prism, the reflecting prism can rotate in a first plane and can rotate in a second plane, and the first plane and the second plane intersect.

9. The shooting method according to any one of claims 1 to 8, characterized in that Responding to the second operation and determining the target position of the reflecting prism includes: Responding to the second operation, based on the position information of the second operation and the position information of the reflecting prism, determining the target position of the reflecting prism.

10. The photographing method according to any one of claims 1 to 8, characterized in that, Responding to the second operation and determining the target position of the reflecting prism includes: In response to the second operation, based on the position information of the second operation, the position information of the reflecting prism, and the image data collected by the camera module at the first moment, determine the target position of the reflecting prism at the second moment, where the second moment is the next moment after the first moment.

11. The photographing method according to any one of claims 1 to 10, characterized in that, The hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for rotating the reflecting prism, and the kernel layer of the operating system includes a driver for rotating the reflecting prism; The determining the target position of the reflecting prism in response to the second operation includes: in response to the second operation, the algorithm module determines the target position of the reflecting prism; The driving the reflecting prism to rotate to the target position includes: the control module for rotating the reflecting prism controls the operation of the driver for rotating the reflecting prism to drive the reflecting prism to rotate to the target position.

12. The photographing method according to claim 11, wherein, The hardware abstraction layer of the operating system of the electronic device includes: a control module for anti-shake, and the kernel layer of the operating system includes an anti-shake driver; The algorithm module is further configured to generate anti-shake parameters, and the electronic device controls an anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver.

13. An electronic device, characterized in that, Including: One or more processors, a memory, a camera module, and a display screen; the camera module includes a rotatable reflecting prism; The memory, the camera module, and the display screen are coupled to the one or more processors, and the memory is configured to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the shooting method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, For storing a computer program, when the computer program is executed, it is specifically configured to implement the shooting method according to any one of claims 1 to 12.

Images