Zoom method, electronic device and readable medium

Abstract

The present application provides a zoom method, an electronic device and a readable storage medium. The zoom method is applied to an electronic device. The zoom method includes: displaying a camera preview interface, the camera preview interface displays a first object, and the first object is a first size; receiving a first operation of a user on the camera preview interface for the first object; in response to the first operation, displaying a picture in which the first object changes from the first size to a second size on the camera preview interface, the second size is larger than the first size, and during the process of the first object changing from the first size to the second size, the first object is located in the central area of ​​the picture, so that the user specifies a close-up object, and the electronic device automatically tracks the user-specified object and shoots an image centered on the close-up object.

Description

Technical Field

[0001] The present application relates to the field of photographing technology, and in particular to a zoom method, an electronic device, a computer program product, and a computer-readable storage medium. Background Art

[0002] In the shooting scene, the user can zoom in the image of the object (including the human body) according to the needs. For example, the user shoots a medium-to-long distance object, especially a moving object, and needs to take a close-up image of it.

[0003] Users can use zooming to shoot close-up images. Electronic devices include multi-camera modules that support wide-angle + telephoto. The electronic device first uses the wide-angle camera module to frame the image, and then switches the wide-angle camera module to the telephoto camera module to shoot close-up images to achieve zooming. However, this method relies on the subject being in the center of the picture before and after the camera module is switched. If it is not in the center of the picture, the user needs to manually move to recompose the picture, which is inconvenient for the user to operate. Summary of the invention

[0004] The present application provides a zoom method, an electronic device, a computer program product, and a computer-readable storage medium, which can enable a user to specify a close-up object, and the electronic device to automatically track the user-specified object and capture an image centered on the close-up object.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a zoom method, which is applied to an electronic device, and the zoom method includes: displaying a camera preview interface, the camera preview interface displays a first object, and the first object is a first size; receiving a first operation of a user on the camera preview interface for the first object; in response to the first operation, displaying a picture in which the first object changes from a first size to a second size on the camera preview interface, or displaying the first object of the second size and located in a center area; the second size is larger than the first size, and during the process of the first object changing from the first size to the second size, the first object is located in the center area of ​​the picture.

[0007] It can be seen from the above content that: when the user inputs the first operation on the first object on the camera preview interface, the camera preview interface displays the first object in an enlarged size, and the first object is located in the central area of ​​the screen, so that the user specifies a close-up object, and the electronic device automatically tracks the user-specified object and captures an image centered on the close-up object.

[0008] In one possible implementation, the electronic device includes a first camera module, which includes a rotatable reflective prism; before the camera preview interface displays a picture of a first object changing from a first size to a second size, it also includes: determining a target position of the reflective prism; and driving the reflective prism to rotate to the target position.

[0009] In the above possible implementation manner, the reflective prism is rotated to the target position, which can ensure that the first object is located in the center area of ​​the picture during the zooming process of the electronic device.

[0010] In a possible implementation, determining the target position of the reflective prism includes: determining the target position of the reflective prism according to position information of the first operation and position information of the reflective prism.

[0011] In a possible implementation, driving the reflective prism to rotate to the target position includes: driving the reflective prism to rotate to the target position at the output moment of the image bound to the target position, and the image bound to the target position is captured and output by the first camera module.

[0012] In the above possible implementation manner, the reflective prism is driven to rotate to the target position at the output moment of the image bound to the target position, so that the reflective prism can be driven to rotate when a specific image is output.

[0013] In a possible implementation, the first camera module further includes a brake component, wherein: before driving the reflective prism to rotate to the target position, it also includes: driving the brake component to unlock; after driving the reflective prism to rotate to the target position, it also includes: driving the brake component to lock.

[0014] In this possible implementation, the camera module includes a brake component. Before the reflective prism rotates, the brake component can be unlocked. After the reflective prism rotates, the brake component is locked, which can facilitate the reflective prism to quickly stabilize after reaching the position.

[0015] In a possible implementation, the first camera module further includes a focusing component, and the method further includes: driving the focusing component to focus on the first object during the process of the reflective prism rotating to the target position.

[0016] In this possible implementation manner, the camera module includes a focusing component. When the reflective prism rotates to the target position, the focusing component is driven to focus on the first object, thereby further ensuring the clarity of the first object in the image captured by the camera module.

[0017] In a possible implementation, before driving the focusing component to perform focusing, it also includes: using the target position of the reflection prism to compensate for the focusing parameters; during the process of the reflection prism rotating to the target position, driving the focusing component to perform focusing, including: during the process of the reflection prism rotating to the target position, driving the focusing component to focus on the first object with the compensated focusing parameters.

[0018] In a possible implementation, the first camera module further includes an anti-shake component, wherein: during the process of the reflective prism rotating to the target position, the anti-shake component does not operate, and after the reflective prism rotates to the target position, the anti-shake component operates to perform optical image stabilization.

[0019] In this possible implementation, the anti-shake component does not operate during the process of the reflective prism rotating to the target position. After the reflective prism rotates to the target position, the anti-shake component operates to perform optical image stabilization. This can avoid the anti-shake component performing ineffective anti-shake during the process of the reflective prism rotating to the target position, thereby increasing power consumption.

[0020] In a possible implementation, the method further includes: periodically acquiring and storing position information of the reflective prism.

[0021] In a possible implementation manner, the anti-shake component updates the anti-shake parameters using the position information of the reflective prism, and operates with the updated anti-shake parameters.

[0022] In one possible embodiment, the electronic device includes a second camera module; after displaying a picture of a first object changing from a first size to a second size on a camera preview interface, it also includes: receiving a second operation of a user on the first object on the camera preview interface; displaying a picture of the first object changing from the second size to the first size on the camera preview interface, the first object being an image of the first size, captured by the second camera module; or, displaying a first object of the first size on the camera preview interface, the second camera module capturing an image of the first object of the first size.

[0023] In one possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for rotating the reflective prism, and the kernel layer of the operating system includes a driver for rotating the reflective prism; determining the target position of the reflective prism includes: the algorithm module determines the target position of the reflective prism; driving the reflective prism to rotate to the target position includes: the control module for rotating the reflective prism controls the driving operation of the reflective prism to drive the reflective prism to rotate to the target position. The control module for rotating the reflective prism refers to the Scan control module, and the driver for rotating the reflective prism refers to the Scan driver.

[0024] In one possible implementation, the hardware abstraction layer of the operating system of the electronic device includes: an anti-shake control module, and the kernel layer of the operating system includes an anti-shake driver; the algorithm module is also used to generate anti-shake parameters, and the electronic device controls the anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver. The anti-shake control module refers to the OIS control module, and the anti-shake driver refers to the OIS driver.

[0025] In one possible embodiment, the hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for rotating the reflective prism, and the kernel layer of the operating system includes a driver for rotating the reflective prism; determining the target position of the reflective prism, including: the algorithm module determines the target position of the reflective prism; driving the reflective prism to rotate to the target position, including: the control module for rotating the reflective prism controls the driving operation of the reflective prism rotation to drive the reflective prism to rotate to the target position; driving the brake component to unlock or lock, including: the driver for rotating the reflective prism, driving the brake component to unlock or lock by controlling the controller for rotating the reflective prism.

[0026] In a second aspect, the present application provides an electronic device comprising: one or more processors, a memory, a first camera module and a display screen; the memory, the first camera module and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the zoom method as described in any one of the first aspects.

[0027] In a third aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the zoom method as described in any one of the first aspects.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the zoom method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 An interface diagram for a user to perform a zoom operation in a photo taking mode provided in an embodiment of the present application;

[0030] Figure 2 Another interface diagram for a user to perform a zoom operation in a photo taking mode provided in an embodiment of the present application;

[0031] Figure 3 Another interface diagram for a user to perform a zoom operation in a photo taking mode provided in an embodiment of the present application;

[0032] Figure 4 A hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 6 A diagram showing the running timing of the module executing the zoom process provided in the embodiment of the present application;

[0035] Figure 7 A timing diagram of a zoom process provided in an embodiment of the present application;

[0036] Figure 8 A timing diagram of another zoom process provided in an embodiment of the present application;

[0037] Fig. 9 A flowchart for implementing a focusing operation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0039] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0040] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0041] In the image shooting scene, the user can zoom in the image of the object according to the needs. For example, the user needs to shoot a close-up image of an object at a medium or long distance, especially a moving object. The user can control the electronic device to zoom to achieve the close-up image.

[0042] The electronic device may include a multi-camera module that supports wide angle + telephoto. In the scene of shooting close-up pictures, the electronic device first uses the wide-angle camera module to frame, and then switches the wide-angle camera module to the telephoto camera module in response to user operation to shoot close-up pictures. However, this method relies on the subject being in the center of the picture before and after the camera module is switched. If it is not in the center of the picture, the user needs to move it manually to recompose the picture, which is inconvenient for the user to operate. In addition, the user's manual movement of the composition may cause image blur due to jitter, affecting the image quality.

[0043] Therefore, for shooting close-up images of moving objects at medium and long distances, electronic devices cannot realize convenient user operation and capture high-quality images. Based on this, the embodiment of the present application provides a shooting method, which can realize that the user only needs to specify the close-up object, and the electronic device automatically tracks the user-specified object, and automatically captures a high-quality image centered on the close-up object through actions such as composition, framing, and imaging.

[0044] The shooting method provided in the embodiment of the present application can be applied to the photo mode or video mode of the camera. The user can input a zoom operation on the shooting object to specify a close-up object, and the camera module of the electronic device can automatically magnify the object.

[0045] The following combination Figures 1 to 3 , the application scenarios of the shooting method provided in the embodiments of the present application are introduced.

[0046] The user can perform zoom operation on the photographed object, and the camera module of the electronic device can automatically magnify the object.

[0047] Figure 1 Taking a mobile phone as an example, the interface diagram of the camera shooting image scene is shown. Figure 1As shown in (a), the mobile phone starts the camera application in the photo mode, and the mobile phone displays the camera preview interface 101. The camera preview interface 101 displays a picture captured by the mobile phone camera module, and the person in the picture is relatively far away from the mobile phone. If the user wants to zoom in on the person 102, he can click (for example, single-click or double-click) the person 102 to input a zoom operation. In response to the zoom operation input by the user, the mobile phone tracks the person 102 by rotating the reflective prism of the camera module, and obtains an enlarged image of the person 102, as shown in FIG. Figure 1 As shown in (b), in the camera preview interface 103 displayed by the mobile phone, the person 102 is magnified and displayed, and the other two persons are not displayed.

[0048] Figure 1 In the zoom scene shown, the phone zooms in on the object or area specified by the user, such as Figure 1 The zoom operation is performed with the person 102 in (a) as the center. Figure 1 (a) Transition to Figure 1 In the process of (b), the character 102 is at the center of the picture.

[0049] In some embodiments, the camera may be configured with a switch for the object tracking function. When the switch is turned on, the user Figure 1 As shown in (a), the person 102 is clicked to input the tracking object and zoom operation. The mobile phone responds to the user input operation by rotating the reflective prism of the camera module to track the person 102 and obtain an enlarged image of the person 102. The switch is in the off state, and the user Figure 1 As shown in (a), the person 102 is clicked to input the tracking object and zoom operation, but the mobile phone does not respond to the operation.

[0050] The user can also perform another zoom operation on the photographed object, and the camera module of the electronic device can automatically reduce the object. The zoom operation can be understood as an operation of exiting the magnified object image mode.

[0051] like Figure 2 As shown in (a), in the camera preview interface 103 displayed by the mobile phone, the person 104 is magnified. If the user wants to reduce the display of the person 104, he can click (for example, single-click or double-click) the person 104 to input another zoom operation. In response to the operation input by the user, the mobile phone generates a camera preview interface using an image captured by another camera module of the mobile phone, and the person 104 is reduced, as shown in FIG. Figure 2 As shown in (b), in the camera preview interface 101 displayed by the mobile phone, the person 104 is reduced and two other persons are displayed.

[0052] The zoom operation input by the user is not limited to clicking on the photographed object, but can also be other operation methods, such as moving the thumb and index finger towards or away from each other. Moving the thumb and index finger towards each other can indicate zooming out of the object, and moving the thumb and index finger away from each other can indicate zooming in on the object.

[0053] like Figure 3 As shown in (a), in the camera preview interface 103 displayed by the mobile phone, the person 104 is magnified. The user inputs another zoom operation by moving the thumb and index finger toward each other in the display area of ​​the person 104, and the mobile phone responds to the other zoom operation input by the user and displays a reduced image of the person 104, as shown in FIG. Figure 3 As shown in (b), in the camera preview interface 101 displayed by the mobile phone, the person 104 is reduced and two other persons are displayed.

[0054] The zoom method provided in the embodiment of the present application can be applied to touchable electronic devices such as mobile phones, tablet computers, personal digital assistants (PDAs), desktops, laptops, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices.

[0055] Take mobile phones as an example. Figure 4 This is an example of the composition of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display screen 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, and a camera module 190, etc.

[0056] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0057] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0058] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0059] The internal memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0060] In some embodiments, the internal memory 120 stores instructions for executing the zoom process in the shooting method. The processor 110 can execute the instructions stored in the internal memory 120 to realize that in the camera's photo mode or video mode, in response to the user's input zoom operation on the object being photographed, the camera module 190 is automatically controlled to zoom in or out the object with the object as the center.

[0061] The electronic device can implement some shooting functions through the ISP, the camera 130, the video codec, the GPU, the display screen 140 and the application processor, etc. The shooting function can be understood as a conventional shooting function.

[0062] The ISP is used to process the data fed back by the camera 130. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.

[0063] The camera 130 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 130, where N is a positive integer greater than 1.

[0064] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy.

[0065] Video codecs are used to compress or decompress digital videos. Electronic devices can support one or more video codecs. In this way, electronic devices can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0066] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0067] The electronic device implements the display function through a GPU, a display screen 140, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 140 and the application processor. The GPU is used for image rendering by performing mathematical and geometric calculations. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0068] The display screen 140 is used to display images, video interfaces, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oled, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 140, where N is a positive integer greater than 1.

[0069] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0070] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0071] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 110 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0072] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0073] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D and the application processor.

[0074] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0075] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device can listen to music or listen to a hands-free call through the speaker 170A.

[0076] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0077] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and directional recording function, etc.

[0078] The earphone interface 170D is used to connect a wired earphone and can be a USB interface, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0079] In the sensor module 180, the pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 140, the electronic device detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0080] The touch sensor 180B is also called a "touch control device". The touch sensor 180B can be arranged on the display screen 140, and the touch sensor 180B and the display screen 140 form a touch screen, also called a "touch control screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 140. In other embodiments, the touch sensor 180B can also be arranged on the surface of the electronic device, which is different from the position of the display screen 140.

[0081] The acceleration sensor 180C can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity, and can also be used to identify the posture of the electronic device.

[0082] The gyro sensor 180D may be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180D.

[0083] In some embodiments, the electronic device implements other shooting functions through the ISP, the camera module 190, the video codec, the GPU, the display screen 140, and the application processor. Other shooting functions may refer to: in the camera's photo mode or video mode, in response to the user's input zoom operation on the object being photographed, the object is automatically zoomed in or out.

[0084] In some embodiments, the camera module 190 includes: a lens photosensitive element Sensor, an OIS IC and a Scan IC. The camera module 190 may also include an AF IC, which is not shown in the figure.

[0085] In some embodiments, the camera module 190 is a telephoto module.

[0086] In the processor 110, the ISP is also used to process the data fed back by the camera module 190. For example, when taking a photo, the shutter is opened, and the light is transmitted to the sensor through the lens. The light signal is converted into an electrical signal, and the sensor transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.

[0087] The lens of the camera module 190 includes a reflective prism, which is located in the incident light path of the sensor and is used to reflect light to the sensor. In some embodiments, the reflective prism supports rotational movement in the xz plane. Exemplarily, the range of rotational movement is +-24 degrees. It also supports rotational movement in the yz plane. Exemplarily, the range of rotational movement is +-7, and the xz plane and the yz plane are two intersecting planes. The rotation of the reflective prism can also be understood as the rotation of the camera module lens. The rotation of the camera module lens can achieve that the subject is in the center of the picture during the zooming in or out process.

[0088] Scan IC can be understood as a controller that controls the rotation of the reflective prism. For example, Scan IC can be a chip with logic control capabilities. The Scan motor receives the control instructions of Scan IC and can drive the reflective prism to rotate. In some embodiments, the Scan motor includes two, one drives the reflective prism to rotate in the xz plane, and the other drives the reflective prism to rotate in the yz plane.

[0089] In some embodiments, the brake component can be used to lock and unlock the lens position. Before the reflective prism rotates, the brake component can be unlocked, and after the reflective prism rotates, the brake component is locked, which can facilitate the lens to quickly stabilize after reaching the position. In some embodiments, the brake component includes a brake IC and an actuator. The brake IC can receive control instructions from the Scan IC to control the actuator to lock or unlock. The brake IC can be understood as a controller, such as a chip with logic control capabilities.

[0090] In some embodiments, an OIS IC (Optical Image Stabilizer IC) is used for anti-shake. The OIS IC can control the OIS motor for anti-shake. In some embodiments, the OIS motor does not perform anti-shake when the Scan motor drives the reflection prism to rotate. Based on this, the Scan IC can control the OIS motor not to run during the operation of the Scan motor, and to run after the Scan motor finishes running. The OIS IC and the OIS motor can be collectively referred to as an anti-shake component.

[0091] In some embodiments, the AF motor is used for focusing, and the OIS IC can control the AF motor to focus after performing algorithm calculation compensation. The AF motor can also be called a focusing component.

[0092] Electronic equipment in Figure 4 The hardware components shown also have operating systems running on them. operating system, operating system, Operating system, etc. Applications such as camera applications can be installed and run on the operating system.

[0093] Figure 5 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0094] The layered architecture divides the operating system of the electronic device into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers, from top to bottom, namely the application (APP) layer, the application framework layer (abbreviated as FWK), the system library, the hardware abstraction layer (HAL) and the kernel layer.

[0095] The application layer can include a series of application packages. Figure 5 As shown, the application package may include applications such as camera and call.

[0096] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0097] like Figure 5 As shown, the application framework layer may include window management services, content providers, phone managers, view systems, resource managers, and camera services, etc.

[0098] The window management service is used to manage window programs. The window management service can implement window addition, deletion, display and hiding control, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data can include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0099] In some embodiments, the camera service may also be referred to as a camera framework, which is used to receive recording requests, image capture requests, and other requests from camera applications, while maintaining the business logic of the internal circulation of recording requests, image capture requests, and other requests, and sending the final result of the request to the camera application.

[0100] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. In some embodiments of the present application, the application cold start will run in the Android runtime, and the Android runtime will obtain the optimized file status parameters of the application. Then, the Android runtime can determine whether the optimized file is outdated due to system upgrades based on the optimized file status parameters, and return the judgment result to the application management module.

[0101] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0102] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0103] The system library may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0104] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0105] The hardware abstraction layer is a software located between the operating system kernel and the hardware circuit. It is usually used to abstract the hardware to realize the interaction between the operating system and the hardware circuit at the logical layer.

[0106] In some embodiments, the hardware abstraction layer includes a camera Hal, which includes an image link Pipeline and CamxHal3. The camera Hal is mainly responsible for the construction, linking, and interactive control of all modules in the image link. CamxHal3 is used to interact with the upper-level module to open or close the camera Hal, and to transmit data to the camera Hal.

[0107] In some embodiments, the image link Pipeline includes: a module control module, an ISP control module and an algorithm module; the module module is responsible for processing the camera module management, and the module module includes a sensor control module, a scan control module, an OIS control module, and a motor actuator control module.

[0108] In some embodiments, the sensor control module is used to connect to the image sensor, and the sensor control module can control the operation of the sensor through the sensor driver and sensor firmware.

[0109] In some embodiments, the Scan control module is used to connect to the Scan motor to realize the rotation of the reflective prism. The Scan control module can control the operation of the Scan motor through the Scan driver and Scan Firmware.

[0110] In some embodiments, the OIS control module is used to implement anti-shake processing during the rotation of the camera module, and the OIS control module can control the operation of the OIS motor through the OIS driver and OIS Firmware. In other embodiments, the Scan control module can control the operation of the OIS motor through the Scan driver, Scan Firmware and control OIS Firmware.

[0111] In some embodiments, the Actuator control module is used for AF focusing during the rotation of the camera module. The Actuator control module can control the operation of the AF motor through AF drive, OIS Firmware and AF Firmware.

[0112] The ISP control module is used to receive the image data sent by the sensor, process the image data and send it to the algorithm module. In some embodiments, the sensor refers to the sensor of the camera module 190.

[0113] The algorithm module is used to receive the image data sent by the ISP control module, analyze the image data, obtain the operation strategy of the lower module, and feed it back to the module in the module control module. In some embodiments, the algorithm module obtains the rotation strategy of the reflective prism and feeds it back to the Scan control module.

[0114] The kernel layer is the layer between hardware and software. The kernel layer receives commands from the control layer and supports the conversion of control commands into actual device operation instructions. Operation instructions include power-on and power-off control, device mode register instruction generation and issuance. The register control commands of the driver layer are passed to the firmware layer through the device bus. The kernel layer includes at least Sensor driver, Scan driver, OIS driver, and AF driver.

[0115] The firmware layer refers to the software system integrated into each component of the camera module, including Sensor Firmware, Scan Firmware, OIS Firmware, AF Firmware and Brake Firmware. In some embodiments, the firmware layer may not include Brake Firmware.

[0116] In some embodiments, Sensor Firmware is a software system running inside the Sensor, responsible for sensor exposure and image data output. Scan Firmware is a software system running inside the Scan IC, used to control the rotation of the camera module lens, and is responsible for converting the code sent down from the upper layer into a control signal, so as to drive the camera module lens to rotate and achieve the focus tracking and framing goals. Brake Firmware is a software system running inside the brake IC, used to lock the position of the camera module, so that the camera module can quickly stabilize after reaching the target position. OIS Firmware is a software system running inside the OIS IC, used for anti-shake. AF Firmware is a software system running inside the AF IC, used for focus tracking, responsible for converting the target position sent down from the upper layer into a motor current, and driving the AF motor to focus.

[0117] Under the above five-layer architecture, the electronic device is also provided with a hardware layer, which may include the above-mentioned electronic device hardware components. Figure 5 The display includes Sensor, Scan IC, OIS IC, AF motor and brake components.

[0118] It should be noted that although the embodiments of the present application are based on The system is used as an example, but the basic principles are also applicable to Electronic devices with other operating systems.

[0119] The following combination Figure 5 The modules in the software structure of the electronic device are displayed to introduce the running sequence of the zoom process in the shooting method provided in the embodiment of the present application.

[0120] The camera application starts to display the camera preview interface. The user enters the zoom operation in the camera preview interface. The camera application sends a frame request to the image link Pipeline through the camera framework to trigger the zoom process. The request carries the coordinates of the zoom operation position.

[0121] Figure 6 The following figure shows the running sequence of the zoom process. Figure 6As shown, the image link Pipeline receives the request and executes 1. Calling the algorithm module to process the request. In some embodiments, the algorithm module parses the coordinates of the zoom operation position and the position information of the reflective prism to obtain the target position of the reflective prism. Since the Scan motor drives the reflective prism to operate, the target position of the reflective prism obtained by the algorithm module is usually the operating parameter of the Scan motor, which is expressed in the form of code. In some embodiments, the algorithm module can also obtain the operating parameters of the OIS motor based on the coordinates of the zoom operation position OIS and the position information of the motor, which can also be expressed in the form of code. In some embodiments, the algorithm module can also calculate the focus parameters of the AF motor, which can also be expressed in the form of code.

[0122] The algorithm module calculates the code of the Scan motor and executes 2. to write the code to the Metadata Pool.

[0123] The Scan control module executes 3-1. Monitoring the data written to the Metadata Pool, the OIS control module executes 3-2. Monitoring the data written to the Metadata Pool, and the Actuator control module executes 3-3. Monitoring the data written to the Metadata Pool.

[0124] When the Scan control module detects that the Metadata Pool is written into the Scan motor code, it executes 4-1. The Scan control module verifies the code and sends it to the Scan driver via the CSL bus and the V4L2 bus.

[0125] The Scan driver converts the code into register instructions, which are passed to the Scan Firmware via buses such as I2C. The Scan Firmware is controlled to implement 5-1. brake unlocking, 5-2. reflective prism rotation, and 5-3. brake locking.

[0126] When the OIS control module detects that the Metadata Pool is written with the code of the OIS motor, it executes 4-2. It sends an OIS command to the OIS driver via the CSL bus and the V4L2 bus. The command can carry a code to drive the OIS driver to perform anti-shake.

[0127] The OIS driver can convert the code into an OIS command, which is usually a register instruction, and execute 5-4. The command is sent to the OIS Firmware through a bus such as I2C to drive the OIS Firmware to perform anti-shake processing.

[0128] Scan Firmware controls the brake firmware to execute 6-1. Before the reflective prism rotates, the brake is unlocked. After the reflective prism rotates, the brake is locked. Scan Firmware also executes 6-2. Controls the rotation of the reflective prism.

[0129] During the process of controlling the rotation of the reflective prism by the Scan Firmware, 7-1 can be executed. TMR data can be updated to the OIS Firmware in real time; and 7-2 can be executed. When controlling the rotation of the reflective prism, the OIS Firmware can be controlled to enter HOLDON; after the rotation is completed, the OIS Firmware can be controlled to enter OISON. In some embodiments, the TMR data indicates the position of the reflective prism.

[0130] In response to the control of Scan Firmware, OIS Firmware executes 8-1. During the rotation, it is in HOLDON, and after the rotation is completed, it switches to OISON; 8-2. Receives and updates the anti-shake parameters according to TMR data; 8-3. Synchronously receives the information driven by OIS. In response to the control of Scan Firmware, OIS Firmware enters HOLDON, and does not execute the command driven by OIS for anti-shake.

[0131] Scan Firmware can also periodically obtain TMP data and execute 9-1. to report TMP data to Scan driver. In addition, OIS Firmware can also periodically obtain HALL data and execute 9-2. to report HALL data, which is used to indicate the position of OIS motor.

[0132] The Scan driver executes 10-1. Reports TMR data to the Scan control module via the V4L2 bus and the CSL bus. The OIS driver executes 10-2. Reports HALL data to the OIS control module via the V4L2 bus and the CSL bus.

[0133] The Scan control module executes 11-1. Reports TMR data to the NCS. The OIS control module executes 11-2. Reports HALL data to the NCS.

[0134] The NCS executes 12. to summarize the data reported by the Scan control module and the data reported by the OIS control module, and reports the summarized data to the algorithm module. The algorithm module stores the summarized data.

[0135] The Actuator control module can also monitor the AF motor code written into the Metadata Pool and send it to the AF motor; the AF motor executes 13. The AF motor code is passed to the OIS Firmware of the OISIC, and the OIS Firmware performs position compensation to obtain a new code. The new code is sent to the AF Firmware of the AFIC to execute focusing to ensure clear imaging.

[0136] When an electronic device performs a zoom process, the Scan driver has two execution modes:

[0137] Method 1: The motor code sent by the Scan control module takes effect directly.

[0138] Method 2: Scan the motor code sent by the control module and bind it to the request id, and make it effective by matching the request id.

[0139] The following combination Figure 7 and Figure 8 , respectively introduce the above two execution methods to achieve the zoom process. Method 1 corresponds to Figure 7 The content of the embodiment shown, mode 2 corresponds to Figure 8 The content of the embodiment shown.

[0140] like Figure 7 As shown, the zoom process of the electronic device provided in the embodiment of the present application includes:

[0141] S701: The user inputs a zoom operation in the camera preview interface.

[0142] In some embodiments, the user starts the camera application, the camera 130 captures images to obtain a camera preview interface, and the camera application displays the camera preview interface on the display screen. The user can select the camera's operating mode, input zoom operations, etc. on the camera preview interface. Figures 1 to 3 Shows the user entering a zoom operation in the camera preview interface.

[0143] In some embodiments, the camera application receives an input operation from a user and may also obtain an operation position of the user.

[0144] like Figure 1 Middle (a), Figure 2 (a) or Figure 3 As shown in (a), the user inputs a zoom operation by clicking an object in the camera preview interface, and the camera application receives the user's zoom operation and obtains the user's operation position.

[0145] In some embodiments, the user's operation position can be understood as the coordinates of the user's operation position when the image input is displayed on the camera preview interface.

[0146] S702: The camera application sends a request to the camera service, where the request carries the coordinates of the zoom operation position.

[0147] After receiving the zoom operation input by the user and obtaining the user's operation position, the camera application can send a request to the camera service. The request carries the user's operation position, that is, carries the coordinates of the user's operation position. The request sent by the camera application to the camera service is used to request the execution of the zoom process.

[0148] S703: The camera service sends a request to the algorithm module, where the request carries the coordinates of the zoom operation position.

[0149] In some embodiments, the camera service sends a request to the algorithm module in the following manner:

[0150] The camera service sends a request to CamxHal3, which carries the coordinates of the zoom operation position. After receiving the request from the camera application, the camera service sends the request to CamxHal3. After receiving the request from the camera service, CamxHal3 sends the request to Pipeline. After receiving the request, Pipeline sends the request to the algorithm module to call the algorithm module to process the request.

[0151] In some embodiments, the user inputs a tracking object and a zoom operation, and after CamxHal3 receives the request sent by the camera service, it sends the request to the Pipeline corresponding to the camera module 190 .

[0152] It should be noted that when the user turns on the camera, the camera 130 of the electronic device (which can be understood as the main camera) runs to collect images, and the functional modules of the Pipeline corresponding to the camera 130 run to display the camera preview interface on the display screen, which displays the image taken by the camera 130.

[0153] The user inputs a zoom operation in the camera preview interface, and the function module of the Pipeline corresponding to the camera module 190 is driven to run, and the image captured by the camera module 190 can be displayed on the display screen to maintain the camera preview interface. The camera module 190 is a telephoto module, and the subject in the image captured by the camera module 190 is large, which can achieve the enlargement of the image of the photographed object. In some embodiments, the Pipeline corresponding to the camera 130 can run or stop running, without limitation.

[0154] S704: The algorithm module analyzes the coordinates and the position information of the reflecting prism to obtain the target position of the reflecting prism.

[0155] In some embodiments, the algorithm module stores data such as the position information of the reflective prism and the state of the brake component. In some embodiments, the position information of the reflective prism can be the detection value of the TMR sensor, referred to as TMR data.

[0156] After receiving the request, the algorithm module obtains the coordinates and the position information of the reflective prism carried in the request, and uses the coordinates and the position information of the reflective prism carried in the request to obtain the position reached by the reflective prism after rotation, that is, the target position. In some embodiments, the target position can be a code.

[0157] In some embodiments, the algorithm module may store calibration data, which includes the rotation angle of the reflective prism corresponding to the coordinates operated by the user. The algorithm module combines the position information and rotation angle of the reflective prism to obtain the target position of the reflective prism. The target position can be understood as the position to be reached by the reflective prism after rotation.

[0158] S705: The algorithm module writes code into the Metadata Pool.

[0159] In some embodiments, the target position of the reflecting prism obtained by the algorithm module, i.e., the code, can be written into the data pool MetadataPool so that other modules can monitor the changes of data in the data pool. In some embodiments, the Metadata Pool includes multiple metadata, each metadata is used to store the value of a different function. The algorithm module writes the code into the metadata corresponding to the zoom function.

[0160] In some embodiments, the algorithm module also stores the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake parameters of the OIS motor are the detection values ​​of the Hall sensor, referred to as HALL data. After receiving the request, the algorithm module can also obtain the indication value of the anti-shake strategy of the OIS motor according to the coordinates carried in the request and the anti-shake parameters of the OIS motor. In some embodiments, the anti-shake strategy includes the operating mode and operating parameters of the OIS.

[0161] In some embodiments, the indicator value of the anti-shake strategy of the OIS motor obtained by the algorithm module can also be written into the metadata pool so that other modules can monitor the changes of data in the data pool. In some embodiments, the indicator value of the anti-shake strategy of the OIS motor obtained by the algorithm module is written into the metadata corresponding to the anti-shake function.

[0162] S706. The Scan control module monitors the data of the Metadata Pool.

[0163] S707, the Scan control module monitors that the Metadata Pool writes the code, verifies the code, and then sends a control command to the Scan driver, where the control command carries the code.

[0164] In some embodiments, before the Scan control module sends the code to the Scan driver, it verifies the code, which can be understood as verifying whether the target position of the reflective prism indicated by the code is a legal position. If the verification passes, the code is sent to the Scan driver.

[0165] In some embodiments, after the algorithm module obtains the target position of the reflective prism, it may also send it directly to the Scan control module.

[0166] S708 , the Scan driver controls the Scan Firmware to execute brake unlocking, reflection prism rotation, and brake locking.

[0167] After the Scan driver receives the target position of the reflective prism, it can control the Scan Firmware to execute brake unlocking, reflective prism rotation and brake locking.

[0168] In some embodiments, the Scan driver converts the target position of the reflective prism into a register instruction, and the register instruction is transmitted to the Scan Firmware via a bus such as I2C. The Scan Firmware executes the following steps S709, S710, S712, and S713 according to the register instruction.

[0169] In an embodiment of the present application, the Scan driver receives a control command carrying a code, directly responds to the control command, and controls the Scan Firmware to execute brake unlocking, reflection prism rotation, and brake locking, so as to realize the rapid rotation of the reflection prism to the target position, thereby realizing the user's zoom operation object to quickly move the center of the screen and enlarge it.

[0170] The execution order of step S709 and step S710 is not limited to Figure 7 As shown, in some embodiments, ScanFirmware may execute step S709 and step S710 in parallel.

[0171] During the rotation of the reflective prism, the anti-shake performed by the electronic device is invalid. Therefore, the electronic device does not need to perform anti-shake. When the rotation of the reflective prism ends, the electronic device performs anti-shake. Based on this, Scan Firmware controls OIS Firmware to enter HOLDON at the beginning of the rotation of the reflective prism, and controls OIS Firmware to enter OISON at the end of the rotation of the reflective prism. Among them, HOLD ON can be understood as the anti-shake pause state, and OIS ON can be understood as the anti-shake on state.

[0172] S709. Scan Firmware sends a HOLD ON command to OIS Firmware.

[0173] The HOLD ON instruction is used to control the OIS Firmware to suspend the anti-shake process and enter the HOLD ON state. When the OIS Firmware enters the HOLD ON state, the control instructions sent by the OIS driver are only cached and not directly executed.

[0174] S710, Scan Firmware controls the brake firmware to execute brake unlocking.

[0175] S711, the brake firmware executes brake unlocking.

[0176] The brake firmware controls the brake components to perform brake unlocking.

[0177] S712, Scan Firmware controls the reflective prism to rotate to the position indicated by the code.

[0178] In some embodiments, the Scan Firmware controls the Scan motor to rotate and drive the reflective prism to rotate until the reflective prism rotates to the target position, that is, the position indicated by the code.

[0179] against Figures 1 to 3 The zoom operation is shown. After the reflecting prism is rotated to the target position, the display shows Figure 1 Middle (b), Figure 2 (b) or Figure 3 In the interface shown in (b), the user can view the zoomed image through the display screen.

[0180] In the embodiment of the present application, the algorithm module analyzes the coordinates of the zoom operation and the position information of the reflective prism through step S704 to obtain the target position of the reflective prism. The target position can be understood as a position to ensure that the zoom operation object is located at the center of the screen after the zoom operation. Therefore, the Scan Firmware controls the Scan motor to rotate and drive the reflective prism to rotate until the reflective prism rotates to the target position. After the user performs a zoom operation on an object, the object is automatically located at the center of the screen in the screen displayed on the display screen without user operation. In addition, the user does not need to operate, and the problem of blurred images captured due to tracking of the user's mobile device is avoided.

[0181] S713, Scan Firmware controls the brake firmware to execute brake locking.

[0182] S714, the brake firmware executes brake locking.

[0183] Brake Firmware controls the brake components to perform brake locking.

[0184] S715. The Scan control module sends a query command to the Scan driver to query TMR data.

[0185] The Scan control module has a timer that regularly reads TMR data and brake status to pass them to NCS, which then passes them to the algorithm module for Scan's next operating mode and code decision, forming a Scan closed-loop control.

[0186] In some embodiments, a timer may also exist in the Scan driver. The Scan driver periodically reads the TMR data and the brake status and reports them to the algorithm module through the NCS.

[0187] In some embodiments, the OIS control module may also have a timer to periodically read HALL data from the OIS driver, and the HALL data is passed to the NCS, which then passes the data to the algorithm module for anti-shake.

[0188] S716. The Scan driver sends a query command to the Scan Firmware to query TMR data.

[0189] The Scan driver receives the query command from the Scan control module, or is triggered by its own timer to send a query command to the ScanFirmware to query the TMR data.

[0190] S717, Scan Firmware reads TMR data.

[0191] The Scan Firmware receives the query command and reads the TMR data from the TMR sensor.

[0192] S718. Scan Firmware sends TMR data and OIS ON command to OIS Firmware.

[0193] After the reflective prism rotates to the target position, the Scan Firmware passes the TMR data and the OIS ON command to the OIS Firmware. The OIS Firmware enters the OIS ON state and updates the anti-shake parameters according to the TMR data, controlling the OIS motor to restore the state before the HOLD ON state or to the cached state.

[0194] In some embodiments, after the reflective prism rotates to the target position, Scan Firmware transmits the OIS ON command to OIS Firmware. The sending of TMR data by Scan Firmware to OIS Firmware is not limited to the condition that the reflective prism rotates to the target position, and can be sent to OIS Firmware after reading TMR data.

[0195] S719, OIS Firmware updates anti-shake parameters according to TMR data.

[0196] In some embodiments, before anti-shake is turned on, the OIS Firmware may update anti-shake parameters according to TMR data to ensure the anti-shake effect when the Scan motor is in different positions.

[0197] S720. Scan Firmware reads the brake status value from the brake firmware.

[0198] The brake status value indicates the operating status of the brake components.

[0199] S721, Scan Firmware reports TMR data and brake status value to Scan driver.

[0200] After the Scan Firmware reads the TMR data and brake status value, it can report the TMR data and brake status value to the Scan driver.

[0201] S722. The Scan driver reports TMR data and brake status value to the Scan control module.

[0202] After receiving the TMR data and the brake status value, the Scan driver reports the TMR data and the brake status value to the Scan control module.

[0203] S723, the Scan control module reports the TMR data and brake status value to the NCS.

[0204] After receiving the TMR data and the brake status value, the Scan control module reports the TMR data and the brake status value to the NCS.

[0205] NCS (Non Camera Sensor Service) Figure 6 As shown in FIG. 1 , it is located in the hardware abstraction layer and is used to manage the data of other auxiliary devices except the sensor.

[0206] The NCS may pass the TMR data to the algorithm module. In some embodiments, the algorithm module registers a TMR data subscription, after which the TMR data may be passed from the Scan control module to the NCS, which then passes the data to the algorithm module.

[0207] S724. NCS sends TMR data and brake status value to the algorithm module.

[0208] The algorithm module receives the TMR data and the brake status value, and can update and store the TMR data and the brake status value.

[0209] In some embodiments, the Scan control module may send TMR data and brake status values ​​directly to the algorithm module, or send TMR data and brake status values ​​to the algorithm module through other modules, and is not limited to the methods of steps S723 and S724 of this embodiment.

[0210] like Figure 8 As shown, the zoom process of the electronic device provided in the embodiment of the present application includes:

[0211] S801: The user inputs a zoom operation in the camera preview interface.

[0212] The specific implementation of step S801 can refer to the content of step S701 in the above embodiment, which will not be repeated here.

[0213] S802: The camera application sends a request to the camera service, where the request carries the coordinates of the zoom operation position.

[0214] The specific implementation method of step S802 can refer to the content of step S702 in the above embodiment, which will not be repeated here.

[0215] S803: The camera service sends a request to the algorithm module, where the request carries the coordinates of the zoom operation position.

[0216] The specific implementation of step S803 can refer to the content of step S703 in the above embodiment, which will not be repeated here.

[0217] S804, the algorithm module parses the coordinates and the position information of the reflecting prism to obtain a code.

[0218] The specific implementation of step S804 can refer to the content of step S704 in the above embodiment, which will not be repeated here.

[0219] In a shooting scene, the user inputs a zoom operation for an object in the camera preview interface, and the electronic device displays a gradually enlarged picture of the object in the camera preview interface. In this shooting scene, the algorithm module parses the coordinates and the position information of the reflective prism to obtain multiple codes, which are used to indicate multiple position points of the moving trajectory of the reflective prism. The moving trajectory of the reflective prism can be understood as the moving trajectory between the current position of the reflective prism and the target position indicated by the zoom operation. The algorithm module can also write multiple codes into the Metadata Pool in sequence.

[0220] It can be understood that each code corresponds to a frame image of the camera module 190 , and multiple codes correspond to multiple consecutive frames of images of the camera module 190 .

[0221] S805. The algorithm module writes code into the Metadata Pool.

[0222] The specific implementation of step S805 can refer to the content of step S705 in the above embodiment, which will not be repeated here.

[0223] S806. The Scan control module monitors the data of the Metadata Pool.

[0224] The specific implementation of step S806 can refer to the content of step S706 in the above embodiment, which will not be repeated here.

[0225] S807, the Scan control module monitors that the Metadata Pool writes the code, verifies the code, and then sends a control command to the Scan driver, where the control command carries the code.

[0226] The specific implementation of step S807 can refer to the content of step S707 in the above embodiment, which will not be repeated here.

[0227] S808. Scan driver writes code in CRM.

[0228] After receiving the control command carrying the code, the Scan driver will not respond to the control command immediately, but will wait for the camera module 190 to output the image before controlling the rotation of the reflective prism to achieve synchronization between the camera module 190 outputting the image and the rotation of the reflective prism.

[0229] In the shooting scenario mentioned above, the Scan driver writes multiple codes into the CRM, each code corresponding to one frame of image.

[0230] CRM (camera request manager) is a module for managing camera application requests. The requirements issued by the hardware abstraction layer module can be first transmitted to CRM, which matches according to the sensor frame timing, and then sends the configuration to the corresponding module in the N+1 or N+2 effective mechanism. The data interruption of the underlying sensor frame can also be reported to the hardware abstraction layer module through CRM.

[0231] S809. IFE receives an interrupt signal corresponding to the image frame.

[0232] IFE (Image front-end engine) can be understood as a hardware processing unit. The image frame output by the sensor of the camera module 190 can be pre-processed by IFE. If IFE receives an interrupt signal corresponding to the image frame, it means that the sensor of the camera module 190 outputs the image frame.

[0233] S810. IFE schedules CRM to send code to Scan driver.

[0234] IFE receives the interrupt signal corresponding to the image frame, which can indicate that the sensor of the camera module 190 outputs the image frame. Therefore, IFE schedules CRM to send code to the Scan driver to control the moment when the sensor of the camera module 190 outputs the image frame. The Scan driver uses the code to drive the rotation of the reflective prism to realize the rotation of the reflective prism following the action of the sensor of the camera module 190 outputting the image frame, and the two are executed synchronously.

[0235] In the shooting scenario proposed in the above content, the sensor of the camera module 190 outputs an image frame, and the IFE schedules the CRM to send the code corresponding to the image frame to the Scan driver. The Scan driver uses a code to control the Scan Firmware to execute brake unlocking, reflective prism rotation and brake locking. For multiple codes, the Scan driver synchronously controls the Scan Firmware to execute brake unlocking, reflective prism rotation and brake locking at the moment when the image frame corresponding to each code is output, so that the reflective prism rotates to follow the sensor image of the camera module 190, that is, the display screen shows the process of the zoom operation object gradually enlarging from the original position to the center of the picture.

[0236] In some embodiments, the IFE may also schedule the CRM to send a code to the Scan driver when the Sensor outputs a specific image frame.

[0237] After the Scan driver receives the code in step S807, the code can be bound to the request id, and the Scan driver can register with the CRM according to the request id. In this way, the CRM can send the code to the Scan driver when the image frame indicated by the request id is output. The CRM can use the number of times the interrupt signal corresponding to the image frame is received to determine the number of frames of the image output by the Sensor.

[0238] It should be noted that, based on the reason for delayed image frame effectiveness, when the Scan driver registers to the CRM according to the request id, the number of frames for which the image frame delay takes effect can be configured. In this way, the CRM can send the code to the CAN driver before the number of frames indicated by the request id for the image frame to take effect.

[0239] For example, the image frame is delayed for 2 frames to take effect. When the Scan driver registers to the CRM according to the request id, the rotation of the reflective prism is registered when the 10th frame image is output. In this way, the CRM receives the interrupt signal corresponding to the image frame for 8 times, that is, the CRM receives the interrupt signal corresponding to the image frame for the 8th time, and sends the code to the Scan driver, thus ensuring that the reflective prism rotates when the 10th frame image is output.

[0240] S811, Scan driver controls Scan Firmware to execute brake unlocking, reflective prism rotation and brake locking.

[0241] The specific implementation method of step S811 can refer to the content of step S708 in the above embodiment, which will not be repeated here.

[0242] S812. Scan Firmware sends a HOLDON command to OIS Firmware.

[0243] The specific implementation method of step S812 can refer to the content of step S709 in the above embodiment, which will not be repeated here.

[0244] S813, Scan Firmware controls the brake firmware to execute brake unlocking.

[0245] The specific implementation method of step S813 can refer to the content of step S710 in the above embodiment, which will not be repeated here.

[0246] S814, the brake firmware executes brake unlocking.

[0247] The specific implementation method of step S814 can refer to the content of step S711 in the above embodiment, which will not be repeated here.

[0248] S815, Scan Firmware controls the reflective prism to rotate to the position indicated by the code.

[0249] The specific implementation method of step S815 can refer to the content of step S712 in the above embodiment, which will not be repeated here.

[0250] In the embodiment of the present application, the algorithm module analyzes the coordinates of the zoom operation and the position information of the reflective prism through step S804 to obtain the target position of the reflective prism. The target position can be understood as a position to ensure that the zoom operation object is located at the center of the screen after the zoom operation. Therefore, the Scan Firmware controls the Scan motor to rotate and drive the reflective prism to rotate until the reflective prism rotates to the target position. After the user performs a zoom operation on an object, the object is automatically located at the center of the screen in the screen displayed on the display screen without user operation. In addition, the user does not need to operate, and the problem of blurred images caused by the user moving the device for tracking by hand is avoided.

[0251] S816, Scan Firmware controls the brake firmware to execute brake locking.

[0252] The specific implementation method of step S816 can refer to the content of step S713 in the above embodiment, which will not be repeated here.

[0253] S817: The brake firmware executes brake locking.

[0254] The specific implementation method of step S817 can refer to the content of step S714 in the above embodiment, which will not be repeated here.

[0255] S818. The Scan control module sends a query command to the Scan driver to query TMR data.

[0256] The specific implementation method of step S818 can refer to the content of step S715 in the above embodiment, which will not be repeated here.

[0257] S819. The Scan driver sends a query command to the Scan Firmware to query TMR data.

[0258] The specific implementation method of step S819 can refer to the content of step S716 in the above embodiment, which will not be repeated here.

[0259] S820, Scan Firmware reads TMR data.

[0260] For the specific implementation of step S820, please refer to the content of step S717 in the above embodiment, which will not be repeated here.

[0261] S821. Scan Firmware sends TMR data and OIS ON command to OIS Firmware.

[0262] The specific implementation method of step S821 can refer to the content of step S718 in the above embodiment, which will not be repeated here.

[0263] S822, OIS Firmware updates anti-shake parameters according to TMR data.

[0264] The specific implementation method of step S822 can refer to the content of step S719 in the above embodiment, which will not be repeated here.

[0265] S823, Scan Firmware to read the brake status value.

[0266] The specific implementation method of step S823 can refer to the content of step S720 in the above embodiment, which will not be repeated here.

[0267] S824, Scan Firmware reports TMR data and brake status value to Scan driver.

[0268] The specific implementation method of step S824 can refer to the content of step S721 in the above embodiment, which will not be repeated here.

[0269] S825. The Scan driver reports the TMR data and brake status value to the Scan control module.

[0270] The specific implementation method of step S825 can refer to the content of step S722 in the above embodiment, which will not be repeated here.

[0271] S826, the Scan control module reports the TMR data and brake status value to the NCS.

[0272] The specific implementation method of step S826 can refer to the content of step S723 in the above embodiment, which will not be repeated here.

[0273] S827. NCS sends TMR data and brake status value to the algorithm module.

[0274] The specific implementation method of step S827 can refer to the content of step S724 in the above embodiment, which will not be repeated here.

[0275] In response to the zoom operation input by the user, the electronic device can also perform a focusing process to ensure the clarity of the captured image. Fig. 9 , the focusing process of the electronic device is introduced.

[0276] It is understood that the electronic device may execute Figure 7 , Figure 8 The zoom process shown, and the focusing process shown in the following embodiments.

[0277] like Fig. 9 As shown, the focusing process of the electronic device provided in the embodiment of the present application includes:

[0278] S901: The user inputs a zoom operation in the camera preview interface.

[0279] The specific implementation of step S901 can refer to the content of step S701 in the above embodiment, which will not be repeated here.

[0280] S902: The camera application sends a request to the camera service, where the request carries the coordinates of the zoom operation position.

[0281] The specific implementation of step S902 can refer to the content of step S702 in the above embodiment, which will not be repeated here.

[0282] S903: The camera service sends a request to the algorithm module, where the request carries the coordinates of the zoom operation position.

[0283] The specific implementation method of step S903 can refer to the content of step S703 in the above embodiment, which will not be repeated here.

[0284] S904: The algorithm module obtains a focus parameter code.

[0285] The focus parameters can be understood as the operating parameters of the AF motor. In some embodiments, the algorithm module can use a contrast focus algorithm or a phase focus algorithm to obtain the focus parameters. In other embodiments, the algorithm module uses a contrast focus algorithm to obtain one focus parameter and uses a phase focus algorithm to obtain another focus parameter. The algorithm module combines the two focus parameters to obtain the focus parameters to be issued.

[0286] S905. The algorithm module writes code into the Metadata Pool.

[0287] The algorithm module obtains the focus parameters and writes them into the Metadata Pool corresponding to the focus function's metadata.

[0288] S906. The Actuator control module monitors the data of the Metadata Pool.

[0289] S907, the Actuator control module monitors that the Metadata Pool writes the code, verifies the code and then sends a control command to the AF driver, the control command carrying the code.

[0290] Before the Actuator control module sends the code to the AF driver, it verifies the code, which can be understood as verifying whether the target position of the AF motor indicated by the code is a legal position. If the verification passes, the code is sent to the AF driver.

[0291] In some embodiments, after obtaining the focus parameters, the algorithm module may directly send the focus parameters to the Actuator control module.

[0292] S909. The AF driver writes code into the CRM.

[0293] After receiving the control command carrying the code, the AF driver will not respond to the control command immediately, but will wait for the moment when the camera module 190 outputs the image before controlling the focus, so as to achieve synchronization between the camera module 190 outputting the image and the operation of the focus motor.

[0294] S909. IFE receives an interrupt signal corresponding to the image frame.

[0295] The specific implementation of step S909 can refer to the content of step S807 in the above embodiment, which will not be repeated here.

[0296] S910. IFE schedules CRM to send code to AF driver.

[0297] The specific implementation method of step S910 can refer to the content of step S808 in the above embodiment, which will not be repeated here.

[0298] S911, the AF driver controls the OIS Firmware to perform focusing according to the code.

[0299] S912, OIS Firmware compensates the focus parameters according to the target position of the reflective prism.

[0300] In some embodiments, the OIS Firmware stores the correspondence between the target positions of multiple reflective prisms and the compensation values ​​of the focus parameters. The OIS Firmware can use the correspondence to determine the compensation value corresponding to the target position of the reflective prism and use the compensation value to compensate the focus parameter.

[0301] In some embodiments, the AF motor includes two, and the OIS Firmware compensates the focus parameters by the compensation values ​​of the two AF motors according to the target position of the reflective prism to obtain the compensated focus parameters of the two AF motors.

[0302] In some embodiments, the Scan Firmware periodically sends the position information of the reflective prism to the OIS Firmware, and the OIS Firmware may use the position information of the reflective prism sent by the Scan Firmware at the latest moment as the target position of the reflective prism.

[0303] S913. The OIS firmware controls the AF firmware to perform focusing according to the compensated focus parameters.

[0304] In some embodiments, when the OIS Firmware receives the focus parameters, step S912 may not be executed, and the focus parameters may be transparently transmitted to the AF Firmware, and the AF Firmware performs focusing according to the focus parameters.

[0305] S914, AF Firmware performs focusing according to the compensated focus parameters.

[0306] The AF Firmware controls the AF motor to operate and achieve focus according to the compensated focus parameters to ensure the clarity of the captured image. In some embodiments, there are two AF motors, and the AF Firmware controls the two AF motors to operate and achieve focus according to the compensated focus parameters.

[0307] In some embodiments, the AF target code value may be passed to the Scan IC, which has TMR data inside and may also support compensation operations.

[0308] In some embodiments, the AF code may also be directly transmitted to the AF motor for execution without compensation based on the TMR data.

[0309] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0310] The computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0311] Another embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

Claims

1. A zoom method, characterized in that: Applied to an electronic device, the electronic device comprises a first camera module, the first camera module comprises a rotatable reflective prism; the zoom method comprises: Displaying a camera preview interface, wherein the camera preview interface displays a first object, and the first object is a first size; receiving a first operation of a user on the camera preview interface with respect to the first object; Determine, by means of an algorithm module, multiple target positions of a moving trajectory of the reflective prism in sequence according to the position information of the first operation and the position information of the reflective prism, and write each of the determined target positions into a data pool in sequence; the multiple target positions correspond one-to-one to the continuous multiple frames of images captured by the first camera module, and the moving trajectory of the reflective prism is: the moving trajectory between the current position of the reflective prism and the target position indicated by the first operation; After the control module for the rotation of the reflective prism monitors that the data pool is written into the target position, a control command carrying the target position is sent to the driver for the rotation of the reflective prism, and the driver for the rotation of the reflective prism writes the target position into the module CRM for managing camera application requests; When the image front-end IFE receives an interrupt signal corresponding to each frame of the continuous multiple frames of images, the CRM is scheduled by the IFE to send the target position corresponding to the image to the driver for rotating the reflective prism, and the reflective prism is driven to rotate to the target position by the driver for rotating the reflective prism until the reflective prism rotates to the last one of the multiple target positions, so as to realize a picture in which the first object is displayed on the camera preview interface from the first size to the second size. During the process in which the first object is changed from the first size to the second size, the first object is located in the central area of ​​the picture, and the second size is larger than the first size.

2. The zoom method according to claim 1, characterized in that: The step of driving the reflective prism to rotate to the target position by driving the reflective prism to rotate includes: The reflective prism is driven to rotate until it is rotated to the target position at the output moment of the image bound to the target position, and the image bound to the target position is collected and output by the first camera module.

3. The zoom method according to claim 1 or 2, characterized in that: The first camera module also includes a brake component, wherein: Before driving the reflective prism to rotate to the target position, the method further includes: driving the brake component to unlock; After driving the reflective prism to rotate to the target position, the method further includes: driving the brake component to lock.

4. The zoom method according to any one of claims 1 to 3, characterized in that: The first camera module further includes a focusing component, and the method further includes: When the reflection prism rotates to the target position, the focusing component is driven to focus on the first object.

5. The zoom method according to claim 4, characterized in that: Before driving the focusing component to focus on the first object, the method further includes: Compensating focus parameters using the target position of the reflecting prism; In the process of rotating the reflective prism to the target position, driving the focusing component to perform focusing includes: When the reflective prism rotates to the target position, the focusing component is driven to focus on the first object with the compensated focusing parameters.

6. The zoom method according to any one of claims 1 to 5, characterized in that: The first camera module also includes an anti-shake component, wherein: during the process of the reflective prism rotating to the target position, the anti-shake component does not operate, and after the reflective prism rotates to the target position, the anti-shake component operates to perform optical image stabilization.

7. The zoom method according to claim 6, characterized in that: Also includes: The position information of the reflecting prism is periodically acquired and stored.

8. The zoom method according to claim 7, characterized in that: The anti-shake component updates anti-shake parameters using the position information of the reflective prism and operates with the updated anti-shake parameters.

9. The zoom method according to any one of claims 1 to 8, characterized in that: The electronic device includes a second camera module; after the camera preview interface displays a picture showing that the first object changes from the first size to the second size, the electronic device further includes: receiving a second operation of the user on the camera preview interface with respect to the first object; The camera preview interface displays a picture in which the first object changes from the second size to the first size, and the first object is an image of the first size, which is captured by the second camera module; or, the camera preview interface displays a first object of the first size, and the second camera module captures the image of the first object of the first size.

10. The zoom method according to claim 1, characterized in that: The hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for the rotation of the reflective prism, and the kernel layer of the operating system includes a driver for the rotation of the reflective prism; The driving of the reflective prism to rotate to the target position by driving the reflective prism to rotate comprises: the control module of the reflective prism rotation controls the driving operation of the reflective prism rotation to drive the reflective prism to rotate to the target position.

11. The zoom method according to claim 10, characterized in that: The hardware abstraction layer of the operating system of the electronic device includes: an anti-shake control module, and the kernel layer of the operating system includes an anti-shake driver; The algorithm module is also used to generate anti-shake parameters, and the electronic device controls the anti-shake component to perform optical anti-shake through the anti-shake control module and the anti-shake driver.

12. The zoom method according to claim 3, characterized in that: The hardware abstraction layer of the operating system of the electronic device includes: a control module and an algorithm module for the rotation of the reflective prism, and the kernel layer of the operating system includes a driver for the rotation of the reflective prism; The driving of the brake component to unlock or lock includes: driving the reflective prism to rotate, and driving the brake component to unlock or lock by a controller that controls the rotation of the reflective prism.

13. An electronic device, characterized in that: include: One or more processors, a memory, a first camera module, and a display screen; The memory, the first camera module and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the zoom method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the zoom method according to any one of claims 1 to 12.

Images