Camera anti-shake method, electronic device and computer readable storage medium

Abstract

The application relates to the technical field of terminals, and provides a camera anti-shake method, an electronic device and a computer readable storage medium. The method is applied to a terminal device, the terminal device comprises a camera and a motor, and the method comprises the following steps: acquiring a current frequency range in which the terminal device is located in a current scene; determining a target anti-shake parameter matched with the current frequency range according to the current frequency range, the target anti-shake parameter being a set of anti-shake parameters in a preset mapping relationship, the preset mapping relationship at least comprising a corresponding relationship between the target anti-shake parameter and a target frequency range, and the current frequency range and the target frequency range being matched; and driving the motor to drive the camera to move according to the target anti-shake parameter. The method can dynamically switch the adaptive anti-shake parameter according to the vibration frequency in different scenes, so that the anti-shake performance of the camera is improved, and the shooting experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, specifically to a camera image stabilization method, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of terminal technology, terminal devices are becoming increasingly feature-rich, and users are experiencing a higher level of user experience.

[0003] A camera can be installed on a terminal device to take photos and record videos. To improve shooting quality, the camera on the terminal device can also have image stabilization. The terminal device can activate image stabilization during shooting to enhance the shooting effect. During image stabilization, the motor drive chip adjusts the output current under the action of PID parameters to drive the motor to run different distances. During the movement of the motor, it can drive the camera to move accordingly, thus maintaining the relative stability of the camera even when the terminal device vibrates, thereby achieving image stabilization.

[0004] However, when users are shooting in various different motion states, the PID parameters cannot meet the needs of all motion scenarios, resulting in poor shooting results in motion scenarios. Summary of the Invention

[0005] This application provides a camera image stabilization method, device, chip, electronic device, computer-readable storage medium, and computer program product that can improve image stabilization performance, thereby enhancing the user's shooting experience.

[0006] In a first aspect, a camera image stabilization method is provided, applied to a terminal device, the terminal device including a camera and a motor. The method includes: obtaining the current frequency range of the terminal device in the current scene; determining target image stabilization parameters that match the current frequency range based on the current frequency range, the target image stabilization parameters being a set of image stabilization parameters in a preset mapping relationship, the preset mapping relationship including at least the correspondence between the target image stabilization parameters and the target frequency range, the current frequency range and the target frequency range matching; and driving the motor to move the camera based on the target image stabilization parameters.

[0007] The motor here is an OIS drive motor, used to move the camera lens for image stabilization.

[0008] The terminal device can have preset mapping relationships in its memory, or it can download preset mapping relationships from the network and store them for later retrieval. The preset mapping relationship includes the correspondence between target stabilization parameters and target frequency ranges. Optionally, it can also include one-to-one correspondences between multiple other frequency ranges and multiple other stabilization parameters. It should be noted that each frequency range corresponds to a specific scenario, and each frequency range represents the range of vibration frequencies of the terminal device within that scenario.

[0009] When a user opens the camera app to take a picture, the camera app displays a preview of the shot on the screen. At this point, the upper-level camera app sends a control request to activate the image stabilization function. Optionally, the camera app can send a control request to the lower-level app to trigger the image stabilization function.

[0010] The sensors on the terminal device can continuously detect the current vibration frequency to determine the current frequency range. When the underlying module receives a control request from the upper layer, the terminal device can query a preset mapping relationship based on the current frequency range, determine the target frequency range that matches the current frequency range, and send the target stabilization parameters corresponding to the target frequency range as the parameters to be used to drive the motor to move accordingly, thereby moving the camera or camera lens accordingly to achieve image stabilization.

[0011] Optionally, determining a target frequency range that matches the current frequency range can be done by selecting a frequency range that covers the current frequency range, or by selecting a frequency range whose overlap with the current frequency range is greater than a certain percentage. For example, if the overlap between a frequency range and the current frequency range is greater than 70%, then that frequency range can be determined to match the current frequency range and be the target frequency range; alternatively, the frequency range with the highest overlap with the current frequency range can be selected as the target frequency range.

[0012] Optionally, when the current frequency range is a frequency range in two or three directions, a target frequency range that matches the current frequency range is determined. This can be achieved by having the frequency range in at least two directions or all directions of the current frequency range match the target frequency range.

[0013] Optionally, the current frequency range is a frequency range in two or three directions, and the target frequency range is also a frequency range in the same direction. To determine the target frequency range that matches the current frequency range, the frequency range in at least two directions or all directions of the current frequency range can match the frequency range in the direction corresponding to the target frequency range.

[0014] This method can dynamically adapt the image stabilization parameters to different scenarios based on the range of vibration frequencies in different scenarios. This allows the camera to use image stabilization parameters that match the scene in different scenarios, achieving dynamic image stabilization in multiple scenarios, improving image stabilization performance in various scenarios, and enhancing the user's shooting experience.

[0015] In some possible implementations, the target stabilization parameter is the target PID parameter.

[0016] PID parameters consist of three aspects: proportional gain (P), integral time (I), and derivative time (D). These PID parameters can be stored in the SOC's memory area. When the OIS driver IC needs to output a large current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor, achieving better anti-shake performance.

[0017] In some possible implementations, the preset mapping relationship includes a one-to-one correspondence between multiple sets of PID parameters and multiple frequency ranges. The multiple sets of PID parameters include target PID parameters, and the multiple frequency ranges include target frequency ranges. Each frequency range in the multiple frequency ranges corresponds to a different motion scenario.

[0018] The preset mapping relationship includes a one-to-one correspondence between multiple sets of PID parameters and multiple frequency ranges, which can cover a variety of scenarios, realize dynamic switching of PID parameters in more scenarios, and have a wider range of applications.

[0019] Among the possible implementations, the exercise scenarios include: holding with one hand, holding with both hands, walking, jogging, climbing, dancing, jumping, and sitting still.

[0020] The vibration frequency of the terminal device varies depending on the user's activity level. The preset mapping relationship defines multiple frequency ranges corresponding to these different activity scenarios, including single-handed use, two-handed use, walking, jogging, hiking, dancing, jumping, and sitting still. This comprehensive mapping covers most everyday scenarios, enhancing the applicability of the method.

[0021] In some possible implementations, obtaining the current frequency range of the terminal device in the current scenario includes: obtaining the initial angular velocity data collected by the angular momentum sensor on the terminal device in the current scenario; filtering the initial angular velocity data to obtain filtered angular velocity data; converting the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; analyzing the frequency domain data to obtain multiple vibration frequencies and the amplitudes of multiple vibration frequencies; and determining the current frequency range based on the multiple vibration frequencies and the amplitudes of multiple vibration frequencies.

[0022] The angular momentum sensor can be a gyroscope. The gyroscope on the terminal device continuously detects angular velocity. The angular velocity measured by the gyroscope over a period of time can be considered the initial angular velocity data. This initial angular velocity data includes information such as disturbances caused by external interference. Based on this, the terminal device can filter the initial angular velocity data to remove background noise and other interference, obtaining filtered angular velocity data. Both the initial and filtered angular velocity data are time-domain data.

[0023] The terminal device can also convert the filtered angular velocity data from the time domain to the frequency domain, for example, by performing a Fourier transform, to obtain the corresponding frequency domain data. Frequency domain data can intuitively reflect the frequency characteristics of the motion. Next, the terminal device analyzes the converted frequency domain data to obtain multiple vibration frequencies and their amplitudes. Based on these multiple vibration frequencies and their amplitudes, the terminal device can filter out the range of vibration frequencies with high frequency and large amplitude to form the current frequency range.

[0024] Using this method, the terminal device can detect and obtain the target frequency range of the current scene in a timely manner through its own sensors, which facilitates the subsequent accurate configuration of the corresponding image stabilization parameters.

[0025] In some possible implementations, the terminal device also includes: an angular momentum sensor, a system-on-a-chip (SOC), an OIS controller, and an OIS driver chip. The angular momentum sensor acquires initial angular velocity data and sends it to the SOC; the SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target PID parameters matching the current frequency range; the SOC sends the target PID parameters to the OIS controller; the OIS controller sends a control signal to the OIS driver chip based on the target PID parameters, the control signal carrying the target PID parameters; the OIS driver chip responds to the control signal and outputs a drive current based on the target PID parameters; the motor drives the camera to move under the action of the drive current.

[0026] In the OIS architecture, the terminal device has an OIS controller, eliminating the need for a Sensorhub. It can interact with the SOC and obtain the corresponding PID parameters for the current scene, thereby dynamically adapting the PID parameters. This allows the camera to use scene-matched PID parameters in different scenarios, achieving dynamic image stabilization in various situations, improving image stabilization performance and enhancing the user's shooting experience.

[0027] In some possible implementations, the SOC is equipped with an OIS control service, a sensor driver module, an OIS drive control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range. This includes: the OIS control service forwarding received control requests to the OIS drive control module (the control requests are sent by the camera application when it is turned on); the sensor drive module receiving the initial angular velocity data from the angular momentum sensor and sending it to the OIS drive control module; the OIS drive control module responding to the control request sending the initial angular velocity data to the frequency range extraction module; the frequency range extraction module performing time-frequency conversion on the initial angular velocity data to obtain the current frequency range; and the SOC determining the target image stabilization parameters matching the current frequency range based on the current frequency range. This includes: the frequency range extraction module sending the current frequency range to the PID parameter matching module; the PID parameter matching module determining the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship based on the current frequency range; and the PID parameter matching module calling the PID parameter loading service to retrieve the target PID parameter from the PID parameter database based on the parameter identifier of the target PID parameter.

[0028] In some possible implementations, the terminal device also includes: an angular momentum sensor, a SOC, a smart sensor hub, and an OIS driver chip; the angular momentum sensor acquires initial angular velocity data and sends it to the smart sensor hub; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the smart sensor hub determines the target PID parameters that match the current frequency range based on the current frequency range; the smart sensor hub sends the target PID parameters to the OIS driver chip; the OIS driver chip outputs a drive current based on the target PID parameters; and the motor drives the camera to move under the action of the drive current.

[0029] In the S-OIS architecture, instead of an OIS controller, a Sensorhub is used in the terminal device. The Sensorhub can interact with the SOC and obtain control requests for the current scene. In response to the control requests, it extracts the frequency range of the current scene based on the angular momentum data and matches the corresponding PID parameters according to the frequency range. This achieves dynamic adaptation of PID parameters, enabling the camera to use scene-matched PID parameters in different scenarios, thus achieving dynamic image stabilization in various scenarios, improving image stabilization performance in multiple scenarios, and enhancing the user's shooting experience.

[0030] In some possible implementations, the S-OIS control service and QMI driver module are deployed on the SOC, while the S-OIS manager, frequency range extraction module, PID parameter loading service, PID parameter matching module, and PID parameter database are deployed on the intelligent sensor hub. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The method further includes: the S-OIS control service forwarding received control requests to the QMI driver module; the control request being a request issued by the camera application upon startup; the QMI driver module forwarding the control request to the S-OIS manager; and the intelligent sensor hub performing time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the S-OIS manager sending the received control request to the frequency range... The S-OIS manager receives initial angular velocity data from the angular momentum sensor and, in response to a control request, sends the initial angular velocity data to the frequency range extraction module. The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range. The intelligent sensor hub determines the target anti-shake parameters matching the current frequency range, including: the frequency range extraction module sending the current frequency range to the PID parameter matching module; the PID parameter matching module determining the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship; and the PID parameter matching module calling the PID parameter loading service to retrieve the target PID parameter from the PID parameter database based on the parameter identifier of the target PID parameter.

[0031] In some possible implementations, whether in the SOC of the OIS architecture or the Sensorhub of the S-OIS architecture, the frequency range extraction module includes: a data filtering module, a time-domain waveform conversion module, a spectrum analysis module, and a frequency range determination module. The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the data filtering module filters the initial angular velocity data to obtain filtered angular velocity data; the time-domain waveform conversion module converts the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; the spectrum analysis module analyzes the frequency domain data to obtain multiple vibration frequencies and the amplitudes of multiple vibration frequencies; and the frequency range determination module determines the current frequency range based on the multiple vibration frequencies and the amplitudes of multiple vibration frequencies.

[0032] In a second aspect, a camera stabilization device is provided, comprising a unit consisting of software and / or hardware, the unit being used to perform any one of the methods described in the first aspect.

[0033] Thirdly, embodiments of this application provide a chip including a processor; the processor is used to read and execute a computer program stored in a memory to perform any one of the methods described in the first aspect.

[0034] Optionally, the chip further includes a memory, which is connected to the processor via a circuit or wire.

[0035] Optionally, the chip is a System-on-a-Chip (SOC);

[0036] Optionally, the chip is a smart sensor hub;

[0037] Alternatively, the chip may further include a communication interface.

[0038] Fourthly, an electronic device is provided, comprising: a processor, a memory, and an interface; the processor, memory, and interface cooperate with each other to enable the electronic device to perform any one of the methods described in the first aspect.

[0039] Fifthly, an electronic device is provided, which includes any one of the chips described in the third aspect.

[0040] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, the processor performs any one of the methods described in the first aspect.

[0041] In a seventh aspect, a computer program product is provided, the computer program product comprising: computer program code, which, when executed on an electronic device, causes the electronic device to perform any one of the methods described in the first aspect. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application;

[0043] Figure 2 This is a software structure block diagram of the terminal device 100 provided in the embodiments of this application;

[0044] Figure 3 This is a graph showing the angular velocity and frequency distribution along the X-axis in a single-handed grip scenario provided in an embodiment of this application.

[0045] Figure 4 This application provides a graph showing the angular velocity and frequency distribution along the Y-axis in a scenario where the device is held in one hand.

[0046] Figure 5 This application provides a graph showing the angular velocity and frequency distribution along the Z-axis in a scenario where the device is held in one hand.

[0047] Figure 6This is a graph showing the angular velocity and frequency distribution along the X-axis in a two-handed grip scenario provided in the embodiments of this application.

[0048] Figure 7 This is a graph showing the angular velocity and frequency distribution along the Y-axis in a two-handed grip scenario provided in the embodiments of this application.

[0049] Figure 8 This is a graph showing the angular velocity and frequency distribution along the Z-axis in a two-handed grip scenario provided in the embodiments of this application.

[0050] Figure 9 This is a hardware architecture diagram of an example of a camera image stabilization method provided in an embodiment of this application;

[0051] Figure 10 This is a hardware and software architecture diagram under an example of an OIS architecture provided in an embodiment of this application;

[0052] Figure 11 This is an interactive diagram of an example of a camera stabilization method under an OIS architecture provided in an embodiment of this application;

[0053] Figure 12 This is a flowchart illustrating a method for determining a frequency range provided in an embodiment of this application;

[0054] Figure 13 This is a hardware and software architecture diagram of an example of an S-OIS architecture provided in an embodiment of this application;

[0055] Figure 14 This is an interactive diagram of an example of a camera stabilization method under the S-OIS architecture provided in an embodiment of this application;

[0056] Figure 15 This is a flowchart of an example of a camera image stabilization method provided in an embodiment of this application;

[0057] Figure 16 This is a schematic diagram of the structure of a camera image stabilization device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0059] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0060] With the development of terminal technology, terminal devices are becoming increasingly feature-rich, and user experience is improving. For example, cameras can be installed on terminal devices to take photos and record videos. To improve shooting quality, cameras on terminal devices can also have image stabilization functions. Terminal devices can enable image stabilization during shooting to enhance the image quality. During image stabilization, the motor drive chip, under the influence of PID parameters, adjusts the output current to drive the motor to travel different distances. As the motor moves, it can move the camera, thus maintaining relative stability even when the terminal device vibrates, achieving image stabilization. However, when users are shooting in various motion states, the PID parameters cannot meet the needs of all motion scenarios, resulting in poor shooting quality in some situations.

[0061] This application provides a camera image stabilization method that dynamically adapts different PID parameters to different scenarios and uses the PID parameters adapted to the current scenario for image stabilization. This method is applicable to various motion scenarios and improves the success rate of shots in low-light, shaky, and other environments, thereby enhancing the camera's image stabilization performance and improving the user's shooting experience.

[0062] The camera stabilization method provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). These terminal devices are equipped with cameras. The cameras are cameras with optical image stabilizers (OIS). This application does not impose any limitations on the specific type of terminal device.

[0063] For example, Figure 1This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0064] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0065] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0066] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0067] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0068] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0069] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0070] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0071] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to enable the display function of the terminal device 100.

[0072] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0073] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0074] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0075] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0076] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0077] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0078] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0079] The camera 193 can also be equipped with a motor, which can be called an OIS drive motor, to move the camera 193, thereby achieving functions such as focusing and image stabilization.

[0080] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0081] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0082] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0083] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0084] The gyroscope sensor 180B can be used to determine the motion attitude of the terminal device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0085] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0086] The 180E accelerometer can detect the magnitude of acceleration of the terminal device 100 in various directions (typically three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the terminal device, and can be applied to applications such as landscape / portrait switching and pedometers.

[0087] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0088] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of terminal device 100.

[0089] Figure 2 This is a software structure block diagram of the terminal device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0090] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0091] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0092] like Figure 2 As shown, the application framework layer (FWK) can include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0093] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0094] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0095] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0096] The phone manager is used to provide communication functions for terminal device 100. For example, it manages call status (including connection, hang-up, etc.).

[0097] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0098] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0099] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0100] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0101] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0102] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0103] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0104] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0105] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0106] A 2D graphics engine is a graphics engine for 2D drawing.

[0107] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0108] For ease of understanding, the following embodiments of this application will be described using the following methods: Figure 1 and Figure 2 Taking the terminal device with the structure shown as an example, and in conjunction with the accompanying drawings and application scenarios, the camera stabilization method provided in this application embodiment will be specifically described.

[0109] Users often use their devices to take photos in different scenarios, such as when they are in different states of motion, such as walking, jogging, climbing, dancing, jumping, or sitting still.

[0110] It has been discovered that the vibration frequency of a user's handheld device varies depending on the environment in which it is located. Therefore, the environment in which the device is situated can be distinguished by its vibration frequency.

[0111] Optionally, the terminal device can collect vibration frequencies under different scenarios to obtain frequency ranges corresponding to various scenarios. Specifically, the terminal device can detect its angular velocity using its own gyroscope (i.e., the gyroscope sensor mentioned earlier). Optionally, the terminal device can be equipped with three gyroscopes, each used to detect the angular velocity of the terminal device in three directions in three-dimensional space. Taking one gyroscope as an example, the gyroscope can detect the angular velocity in that direction at a certain period and report it to the upper layer. Angular velocity is data describing the vibration state in the time domain. The terminal device can then perform time-frequency conversion on the angular velocity to convert it into frequency domain data, thereby obtaining the vibration frequency in that direction.

[0112] The terminal device statistically analyzes the frequency characteristics of vibration frequencies in different scenarios to obtain the frequency range of vibration frequencies in different scenarios, thus forming a correspondence between frequency ranges and scenarios.

[0113] Figures 3 to 5 The diagram shows the angular velocity and frequency distribution along the X, Y, and Z axes in a scenario where a user holds the terminal device with one hand. The X, Y, and Z axes are mutually perpendicular.

[0114] Figure 3 This is a graph showing the angular velocity and frequency distribution along the X-axis in a scenario where a user holds the terminal device with one hand. Figure 3 In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where a user holds the terminal device with one hand, the angular velocity along the X-axis is mostly concentrated between -0.4 and 0.4. The terminal device will... Figure 3 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 3 The curve shown as b in the figure. In Figure 3 In the diagram, the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude (also known as vibration amplitude). Figure 3 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 4.932Hz, 4.832Hz, and 5.032Hz, mainly around 5Hz.

[0115] It should be noted that, Figures 3 to 8 The amplitude value can also be rounded to the nearest integer as the final value; this is not a limitation here.

[0116] Figure 4 This is a graph showing the angular velocity and frequency distribution along the Y-axis in a scenario where a user holds the terminal device with one hand. Figure 4 In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where a user holds the terminal device with one hand, the angular velocity along the Y-axis is mostly concentrated between -0.5 and 0.5. The terminal device will... Figure 4 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 4 The curve shown as b in the figure. In Figure 4 In the diagram (b), the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude, where amplitude indicates the number of times that frequency occurs. Figure 3 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 3.766Hz, 3.499Hz, and 3.199Hz, mainly around 3.5Hz.

[0117] Figure 5 This is a graph showing the angular velocity and frequency distribution along the Z-axis in a scenario where a user holds the terminal device with one hand. Figure 5 In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where a user holds the terminal device with one hand, the angular velocity along the Z-axis is mostly concentrated between -0.5 and 0.5. The terminal device will... Figure 5 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 5 The curve shown as b in the figure. In Figure 5 In the diagram (b), the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude, where amplitude indicates the number of times that frequency occurs. Figure 5 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 4.932Hz, 4.832Hz, and 5.032Hz, mainly around 4Hz.

[0118] Figures 6 to 8 The figure shows the curves of angular velocity and frequency distribution in the X, Y, and Z directions in a scenario where the user holds the terminal device with both hands.

[0119] Figure 6 This is a graph showing the angular velocity and frequency distribution along the X-axis in a scenario where a user holds the terminal device with one hand. Figure 6 In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where the user holds the terminal device with both hands, the angular velocity along the X-axis is mostly concentrated between -0.4 and 0.4. The terminal device will... Figure 6 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 6 The curve shown as b in the figure. In Figure 6 In the diagram (b), the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude, where amplitude indicates the number of times that frequency occurs. Figure 3 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 0.500Hz, 0.607Hz, and 0.536Hz, mainly around 0.55Hz.

[0120] Figure 7 This is a graph showing the angular velocity and frequency distribution along the Y-axis in a scenario where a user holds the terminal device with one hand. Figure 7In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where a user holds the terminal device with one hand, the angular velocity along the Y-axis is mostly concentrated between -0.3 and 0.3. The terminal device will... Figure 7 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 7 The curve shown as b in the figure. In Figure 4 In the diagram (b), the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude, where amplitude indicates the number of times that frequency occurs. Figure 7 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 0.607Hz, 0.750Hz, and 0.785Hz, mainly around 0.7Hz.

[0121] Figure 8 This is a graph showing the angular velocity and frequency distribution along the Z-axis in a scenario where a user holds the terminal device with one hand. Figure 8 In graph 'a', the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in scenarios where a user holds the terminal device with one hand, the angular velocity along the Z-axis is mostly concentrated between -0.4 and 0.4. The terminal device will... Figure 8 The data in 'a' undergoes time-frequency conversion, such as a Fourier transform, to obtain the corresponding frequencies. The terminal device then statistically analyzes the frequencies at different times to obtain, for example... Figure 8 The curve shown as b in the figure. In Figure 8 In the diagram (b), the horizontal axis represents frequency in Hertz (Hz), and the vertical axis represents amplitude, where amplitude indicates the number of times that frequency occurs. Figure 8 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are: 0.393Hz, 0.714Hz, and 0.428Hz.

[0122] Optionally, the terminal device can also collect angular velocities in various directions under other scenarios, thereby obtaining vibration frequencies or frequency ranges in different directions corresponding to different scenarios.

[0123] Optionally, the terminal device can also combine the vibration frequencies or frequency ranges of vibration frequencies in different directions under various scenarios into a single frequency range, which is used as the vibration frequency or frequency range corresponding to that scenario. Table 1 shows the frequency ranges of different directions corresponding to different scenarios.

[0124] As shown in Table 1, the terminal device can record the frequency range corresponding to each direction in different scenarios, or it can record the combined range of frequency ranges in each direction in different scenarios. This application embodiment does not limit this.

[0125] Optionally, the terminal device may also select only one or two frequency ranges in one direction as the frequency range of the corresponding scene, which will not be elaborated here.

[0126] After determining the correspondence between different scenarios and frequency ranges, the terminal device can debug a set of adapted PID parameters for each scenario.

[0127] Table 1

[0128]

[0129] To facilitate understanding of the technical solution of this application, the working principle of PID parameters is explained here. PID parameters include three aspects: proportional gain (P), integral time (I), and derivative time (D).

[0130] The proportional gain P reflects the current error value and is proportional to the error signal, used to rapidly reduce the error. Increasing the proportional gain improves the system's response speed, but may also lead to system instability or oscillations.

[0131] The integral time I reflects the accumulation of errors over a period of time and is used to eliminate static errors. Adjusting the integral time can smooth the system response, but it may also slow down the system response or cause integral saturation.

[0132] The derivative time D reflects the rate of change of the error signal and is used to predict future error trends. Adjusting the derivative time can improve system stability and response speed, but it may also make the system more sensitive to noise.

[0133] When the current output by the motor driver chip does not oscillate, increasing the proportional gain can speed up the system's response. However, an excessively large Kp can lead to system instability.

[0134] When the current output by the motor driver chip does not oscillate, reducing the integration time helps eliminate static errors, but too small a time may cause the system response to slow down or the integration to saturate.

[0135] Increasing the derivative time can improve the system's stability and response speed when the current output by the motor driver chip does not oscillate; however, excessively large derivative times can make the system sensitive to noise.

[0136] PID parameters can be stored in the memory area of ​​the SOC or other processor. When the OIS driver IC needs to output a large current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor.

[0137] For each different scenario, a set of PID parameters can be fine-tuned to ensure effective image stabilization. Specifically, the terminal device can be clamped and vibrated at the corresponding vibration frequency for that scenario. Images are then captured under vibration, and the results are compared to determine if they meet the requirements. If they do, the current PID parameters are sufficient for image stabilization at the corresponding vibration frequency. If not, the PID parameters are adjusted until the captured images at the appropriate vibration frequency meet the requirements.

[0138] Optionally, during the PID parameter tuning process, the proportional gain, integral time, and derivative time can be tuned separately one by one, and then a comprehensive tuning can be performed until the effect of the captured image meets the requirements.

[0139] Optionally, determining whether the captured image meets the requirements can be done by visually inspecting the image to assess its sharpness. If it is sufficiently sharp, the image meets the requirements; if there are blurry areas such as ghosting, the image does not meet the requirements. Alternatively, it can be determined by checking if the image compression ratio meets the requirements. If the compression ratio is greater than or equal to a preset threshold, the image meets the requirements, indicating good shooting results and good image stabilization; if the compression ratio is less than the preset threshold, the image does not meet the requirements, indicating poor shooting results and poor image stabilization.

[0140] Based on this, we can obtain the correspondence between different frequency ranges and different PID parameters (or PID parameter identifiers), as shown in Table 2.

[0141] Table 2

[0142]

[0143] Table 2 above illustrates the correspondence between PID parameter identifiers and frequency ranges. The frequency ranges in Table 2 include different frequency ranges in three directions. Alternatively, the frequency range in Table 2 can be selected from one of the directions, or the union of the selected frequency ranges in the three directions can be used as a comprehensive frequency range to correspond to different PID parameters.

[0144] Optionally, Tables 1 and 2 above can be merged to obtain the correspondence shown in Table 3.

[0145] Optionally, in the above correspondence between frequency ranges and PID parameters, each set of different frequency ranges may partially overlap. For example: 1.2-3.5Hz corresponds to PID1, 2-4Hz corresponds to PID2, 5.5-7.8Hz corresponds to PID3, 5.8-14Hz corresponds to PID4, and 11-20Hz corresponds to PID5. Optionally, the correspondence between frequency ranges and PID parameters may also include correspondences between other frequency ranges and other PID parameters.

[0146] After obtaining the correspondence between the different frequency ranges and the different PID parameters (PID parameter identifiers), they can be preset in the database of the terminal device for subsequent queries.

[0147] Table 3

[0148]

[0149] Optionally, in the above correspondence, different scenarios can also refer to the shooting modes of the camera application on the terminal device. For example, when the shooting mode of the camera application is a video recording mode, the frequency range corresponding to the video recording mode can be considered to be 6Hz-10Hz; when the shooting mode of the camera application is a camera motion mode, the frequency range corresponding to the camera motion mode can be considered to be 8Hz-15Hz. Optionally, the camera motion mode is a short exposure time shooting mode.

[0150] Figure 9 A hardware framework diagram illustrating an example of the camera image stabilization method described in this application is shown. Figure 9 As shown, the gyroscope continuously detects the angular velocity of the terminal device and transmits this data to the OIS controller. The OIS controller converts the angular velocity data into a vibration frequency and, through the vibration frequency detection service on the AP side, determines the PID parameter corresponding to the vibration frequency range from multiple sets of PID parameters. This determined PID parameter is then sent to the OIS driver chip. The OIS driver chip outputs control current to the OIS lens drive motor based on the PID parameter corresponding to the vibration frequency, thereby moving the camera module lens. Simultaneously, the camera module lens is light-sensitive. Under the control of the CMOS controller (AP CMOS controller) on the AP side, the CMOS driver converts the light signal sensed by the camera module lens into an electrical signal. This processed electrical signal is then sent to the display screen for preview via the camera drive service on the AP side, enabling image capture.

[0151] Optionally, the HALL magnetometer (i.e., Hall sensor) can detect changes in the position and attitude of the terminal device and send the detected data to the OIS driver chip, enabling adjustment of the aperture size or triggering of the image stabilization mechanism. When the vibration frequency changes, the terminal device can dynamically switch different PID parameters to adapt to different scenarios through the vibration frequency detection service on the AP side.

[0152] The camera image stabilization method described in this application embodiment can be applied to OIS architecture (an architecture where the camera module has its own OIS controller) and also to S-OIS architecture (an architecture where the platform processor controls the OIS). The following will describe the application in conjunction with... Figures 10 to 14 The architecture diagram and interaction diagram shown provide a detailed description of the camera stabilization methods under these two architectures.

[0153] Figure 10 The diagram illustrates the hardware and software architecture of the camera stabilization method described in this application embodiment applied to the OIS architecture.

[0154] like Figure 10 As shown, Figure 10 The hardware modules include: a System-on-a-Chip (SOC), a gyroscope, an OIS controller, an OIS driver IC, and an OIS drive motor. An operating system is deployed on the SOC. The software architecture of this operating system can be found in [reference needed]. Figure 2 The relevant description. In Figure 2 Based on this, the HAL layer has an OIS control service, and the Kernel layer also includes a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database.

[0155] Figure 11 Is Figure 10 The diagram illustrates the interaction of a camera stabilization method implemented based on the shown architecture. This is combined with... Figure 10 and Figure 11 The method of camera image stabilization is explained in detail. For example... Figure 11 As shown, the method includes:

[0156] S1101, The camera application sends a control request to the camera service.

[0157] When a user opens the camera app to take a picture, the camera app displays a preview of the shot on the screen. At this point, the camera app can send a control request to the camera service located at the FWK layer. This control request is used to trigger the image stabilization function.

[0158] S1102, The camera service sends the control request to the OIS control service.

[0159] S1103, the OIS control service sends the control request to the OIS driver control module.

[0160] After receiving a control request from the upper layer, the camera service forwards the request to the OIS control service in the HAL layer, which then forwards it to the Kernel layer. Optionally, the OIS control service forwards the control request to the OIS driver control module in the Kernel layer.

[0161] Among them, the OIS control service is a service of CSL (Common Services Layer), which can indicate the corresponding resources through the CSL service handle and realize communication with the Kernel layer.

[0162] S1104, the gyroscope reports angular momentum data to the sensor driver module of the SOC.

[0163] A gyroscope (Gyr or G for short) can continuously detect angular momentum data, i.e. angular velocity, and continuously report the angular momentum data to the corresponding driver module in the kernel layer (here, the sensor driver module).

[0164] Alternatively, the gyroscope can be integrated with an acceleration sensor (Acceleration Sensor, or A for short), referred to as an Acceleration / Gyr sensor, or an A+G sensor. When using an A+G sensor, the sensor driver module in the SOC can be an acceleration / gyroscope driver module used to read angular momentum data.

[0165] Accelerometers can also detect shaking of the terminal device and transmit the detected signal to the corresponding processor. The processor immediately calculates the amount of displacement that needs to be compensated, and then performs reverse compensation by moving one or more sets of lenses in the camera lens (i.e., compensation lens group) according to the direction of lens shake and the amount of displacement, thereby achieving image stabilization.

[0166] In this application embodiment, a gyroscope is used as an example sensor, but other sensors may also be used, and this application embodiment does not limit the use of such sensors.

[0167] Optionally, the gyroscope can transmit angular momentum data to the SOC via the I2C bus.

[0168] It should be noted that the order of steps S1104 and S1101 to S1103 is not limited.

[0169] S1105, the sensor driver module sends the angular momentum data to the OIS drive control module.

[0170] The sensor driver module can send the received angular momentum data to the OIS driver control module. The OIS driver control module then distributes the angular momentum data to the corresponding processing modules for processing.

[0171] The sensor driver module is a software module deployed on the SOC. The gyroscope can transmit angular momentum data to the SOC via the I2C bus. In fact, the gyroscope sends angular momentum data to the sensor driver module deployed on the SOC via the I2C bus.

[0172] The S1106 OIS drive control module sends the angular momentum data to the frequency range extraction module.

[0173] When the OIS drive control module receives a control request from the upper layer, it responds by sending the angular momentum data transmitted from the lower layer to the frequency range extraction module. The frequency range extraction module determines the vibration frequency or frequency range of the current scene based on the angular momentum data.

[0174] S1107, The frequency range extraction module calculates the corresponding frequency range based on the angular momentum data.

[0175] Optionally, the frequency range extraction module includes: a data filtering module, a time-domain waveform conversion module, a spectrum analysis module, and a frequency range determination module. Specifically, the OIS drive control module can send the angular momentum data to the data filtering module in the frequency range extraction module.

[0176] For details on how the frequency range extraction module determines the corresponding frequency range based on the angular momentum data, please refer to [link / reference]. Figure 12 As shown, it includes:

[0177] S1201, the data filtering module filters the angular momentum data.

[0178] The angular momentum data here is the angular velocity collected over a period of time, which is time-domain data.

[0179] The data filtering module performs filtering and noise reduction operations on the angular momentum data, removing background noise and other interference. The filtered angular momentum data remains in the time domain. The data filtering module then sends the filtered angular momentum data to the time-domain waveform conversion module.

[0180] S1202, the time-domain waveform conversion module performs time-frequency conversion on the filtered angular momentum data to obtain the corresponding frequency domain data.

[0181] The time-domain waveform conversion module can perform time-frequency conversion on the received filtered angular momentum data, such as performing a Fourier transform, to obtain the corresponding frequency domain data. This frequency domain data can then characterize the vibration frequency in the current scene. The time-domain waveform conversion module then sends the converted frequency domain data to the spectrum analysis module.

[0182] It should be noted that the frequency domain data here represents the vibration frequency and amplitude over a period of time.

[0183] S1203, the spectrum analysis module analyzes the frequency domain data to obtain the vibration frequency and amplitude.

[0184] The spectrum analysis module employs a spectrum analysis algorithm to analyze the frequency domain data and obtain the corresponding vibration frequency and amplitude. Optionally, the vibration frequency obtained here is the vibration frequency over a certain period of time within which the angular momentum data is distributed; the vibration frequency may fluctuate at different moments during this period.

[0185] The spectrum analysis module sends the obtained vibration frequency and amplitude to the frequency range determination module.

[0186] S1204. The frequency range determination module determines the frequency range based on the received vibration frequency and amplitude.

[0187] After receiving the vibration frequencies over a period of time, the frequency range determination module analyzes the vibration frequencies at multiple moments within that period to obtain the frequency ranges corresponding to these vibration frequencies.

[0188] For example, the frequency range determination module can remove vibration frequencies with too low amplitude from the vibration frequencies at multiple moments within this period, filter vibration frequencies with amplitude that meet certain requirements, and remove the highest and lowest vibration frequencies. It can also select the vibration frequencies that appear most frequently and use the range of the selected vibration frequencies as the final frequency range.

[0189] Optionally, Figure 12 In this embodiment, the above processing can be performed on the angular momentum data in one direction to obtain the frequency range in that direction; alternatively, the above processing can be performed on the angular momentum data in three directions separately to obtain the frequency ranges in each of the three directions. Optionally, the frequency range determination module can also merge the frequency ranges in the three directions into a single frequency range.

[0190] S1108, the frequency range extraction module sends the frequency range to the PID parameter matching module.

[0191] After extracting the frequency range corresponding to the current scene, the frequency range extraction module can send the frequency range to the PID parameter matching module.

[0192] S1109. The PID parameter matching module calls the PID parameter loading service to retrieve PID parameters that match the frequency range from the PID parameter database based on the received frequency range.

[0193] The PID parameter database stores multiple sets of PID parameters. These multiple sets of PID parameters correspond to different PID parameter identifiers, such as PID1, PID2, and PID3, etc. The PID parameter identifier here can be the name, sequence number, or other identifier of the PID parameter. This application embodiment does not limit the specific form of the PID parameter identifier, as long as it can distinguish different PID parameters.

[0194] The PID parameter matching module can query the correspondence between multiple preset PID parameter identifiers and multiple frequency ranges, determine a frequency range that matches the received frequency range as the target frequency range, and then use the PID parameter identifier corresponding to the target frequency range as the query result.

[0195] Optionally, the PID parameter matching module determines the target frequency range by selecting one from multiple frequency ranges that includes the received frequency range, or by selecting one from multiple frequency ranges that has the highest overlap with the received frequency range. This application embodiment does not limit this approach.

[0196] The PID parameter matching module calls the PID parameter loading service, which reads a set of PID parameters corresponding to the PID parameter identifier from the query results in the PID database.

[0197] For example, after receiving a frequency range, the PID parameter matching module queries the correspondence between multiple PID parameter identifiers and multiple frequency ranges to obtain the target frequency range that matches the received frequency range, and retrieves the PID parameter identifier corresponding to the target frequency range. Then, it sends a read command to the PID parameter loading service, carrying the PID parameter identifier obtained by the PID parameter matching module. In response to the read command, the PID parameter loading service reads the PID parameter stored in the PID database that corresponds to the received PID parameter identifier, and sends the read PID parameter to the PID parameter matching module.

[0198] S1110, the PID parameter matching module sends the read PID parameters to the OIS controller.

[0199] The PID parameter matching module is a software module deployed on the SOC. This module reads the PID parameters and sends them to the OIS controller. In practice, the SOC sends the read PID parameters to the OIS controller via the PID parameter matching module. Optionally, the SOC can send the PID parameters to the OIS controller via the I2C bus.

[0200] It should be noted that the aforementioned camera application, camera service, OIS control service, sensor driver module, frequency range acquisition module, PID parameter matching module, PID parameter loading service, and PID parameter database are all deployed on the SOC. The steps performed by these modules and services can all be considered as steps performed by the SOC.

[0201] S1111: The OIS controller sends control signals to the OIS driver IC, and the control signals carry PID parameters.

[0202] The OIS controller sends control signals to the OIS driver IC based on the received PID parameters, thereby sending the PID parameters to the OIS driver IC.

[0203] Optionally, the OIS controller can also acquire data from the gyroscope and / or accelerometer via the SPI bus for other control purposes, which will not be elaborated here.

[0204] S1112, the OIS driver IC responds to the control signal and outputs control current to the OIS drive motor.

[0205] The OIS driver IC outputs control current to the OIS drive motor based on the PID parameters carried in the control signal. Under the control of the control current, the OIS drive motor moves accordingly, thereby moving the camera and stabilizing it, thus achieving image stabilization.

[0206] The camera is also equipped with a flash and has the necessary hardware and software architecture for autofocus (AF).

[0207] Depend on Figure 10 and Figure 11 As can be seen from the embodiments, in the OIS architecture, the terminal device is equipped with an OIS controller, which can interact with the SOC and obtain the corresponding PID parameters in the current scene, thereby realizing dynamic adaptation of PID parameters. This allows the camera to use PID parameters that match the scene in different scenarios, achieving dynamic image stabilization in multiple scenarios, improving image stabilization performance in multiple scenarios, and enhancing the user's shooting experience.

[0208] Figure 13 The diagram illustrates the hardware and software architecture of the camera stabilization method described in this application embodiment applied to the S-OIS architecture.

[0209] like Figure 13 As shown, Figure 13 The hardware modules include: a System-on-a-Chip (SOC), a gyroscope, a smart sensor hub (Sensorhub), an OIS driver IC, and an OIS driver motor. The gyroscope, OIS driver IC, and OIS driver motor can be considered part of the hardware layer. Compared to the OIS architecture, the S-OIS architecture does not include an OIS controller; instead, it uses a Sensorhub to implement the relevant algorithms. Specifically, an operating system is deployed on the SOC. The software architecture of this operating system can be found in [link to relevant documentation]. Figure 2 The relevant description. In Figure 2 Building upon this foundation, the HAL layer includes an S-OIS control service, and the Kernel layer further includes a QMI driver module for information exchange between the SOC and Sensorhub. Sensorhub deploys an S-OIS manager, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database.

[0210] Figure 14 Is Figure 13 The diagram illustrates the interaction of a camera stabilization method implemented based on the shown architecture. This is combined with... Figure 13 and Figure 14 The method of camera image stabilization is explained in detail. For example... Figure 14 As shown, the method includes:

[0211] S1401, The camera application sends a control request to the camera service.

[0212] When a user opens the camera app to take a picture, the camera app displays a preview of the shot on the screen. At this point, the camera app can send a control request to the camera service located at the FWK layer. This control request is used to trigger the image stabilization function.

[0213] S1402, The camera service sends the control request to the S-OIS control service.

[0214] S1403 and S-OIS control services send control requests to the QMI driver module.

[0215] After receiving a control request from the upper layer, the camera service forwards the control request to the S-OIS control service in the HAL layer, and then forwards the control request to the Kernel layer through the S-OIS control service. Optionally, the S-OIS control service forwards the control request to the OMI driver module in the Kernel layer.

[0216] Among them, the S-OIS control service is a service of NCS (Network Control System), which can indicate the corresponding resources through the NCS service handle and realize communication with the Kernel layer.

[0217] The S1404 and QMI driver modules send control requests to the S-OIS manager.

[0218] The QMI driver module is deployed on the SOC, and the S-OIS manager is deployed on Sensorhub. The QMI driver module sends control requests to the S-OIS manager; in fact, the SOC sends control requests to the S-OIS manager on Sensorhub through the QMI driver module.

[0219] S1405, angular momentum data reported by the gyroscope to the S-OIS manager.

[0220] The gyroscope can continuously detect angular momentum data, i.e. angular velocity, and continuously report the angular momentum data to the S-OIS manager deployed in Sensorhub.

[0221] Optionally, the gyroscope can transmit angular momentum data to Sensorhub via the SPI bus.

[0222] Optionally, it should be noted that the order of steps S1405 and S1401 to S1404 is not limited.

[0223] S1406, the S-OIS manager sends the angular momentum data to the frequency range extraction module.

[0224] The S-OIS manager can serve as an interface for other hardware, such as SOCs, gyroscopes, and other sensors, to interact with Sensorhub.

[0225] When the S-OIS manager receives a control request from the upper layer, it responds by sending the angular momentum data transmitted from the lower layer to the frequency range extraction module. The frequency range extraction module determines the vibration frequency or frequency range of the current scene based on the angular momentum data.

[0226] S1407 The frequency range extraction module calculates the corresponding frequency range based on the angular momentum data.

[0227] Optionally, the frequency range extraction module includes: a data filtering module, a time-domain waveform conversion module, a spectrum analysis module, and a frequency range determination module. Specifically, the S-OIS manager can send angular momentum data to the data filtering module within the frequency range extraction module.

[0228] The software architecture of the frequency range extraction module, and the specific process of determining the corresponding frequency range based on angular momentum data, can be found in [reference needed]. Figure 12 And related descriptions, which will not be repeated here.

[0229] S1408, the frequency range extraction module sends the frequency range to the PID parameter matching module.

[0230] After extracting the frequency range corresponding to the current scene, the frequency range extraction module can send the frequency range to the PID parameter matching module.

[0231] S1409. The PID parameter matching module calls the PID parameter loading service to retrieve PID parameters that match the frequency range from the PID parameter database based on the received frequency range.

[0232] The PID parameter database stores multiple sets of PID parameters. These multiple sets of PID parameters correspond to different PID parameter identifiers, such as PID1, PID2, and PID3, etc. The PID parameter identifier here can be the name, sequence number, or other identifier of the PID parameter. This application embodiment does not limit the specific form of the PID parameter identifier, as long as it can distinguish different PID parameters.

[0233] The PID parameter matching module can query the correspondence between multiple preset PID parameter identifiers and multiple frequency ranges, determine a frequency range that matches the received frequency range as the target frequency range, and then use the PID parameter identifier corresponding to the target frequency range as the query result.

[0234] Optionally, the PID parameter matching module determines the target frequency range by selecting one from multiple frequency ranges that includes the received frequency range, or by selecting one from multiple frequency ranges that has the highest overlap with the received frequency range. This application embodiment does not limit this approach.

[0235] The PID parameter matching module calls the PID parameter loading service, which reads a set of PID parameters corresponding to the PID parameter identifier from the query results in the PID database.

[0236] For example, after receiving a frequency range, the PID parameter matching module queries the correspondence between multiple PID parameter identifiers and multiple frequency ranges to obtain the target frequency range that matches the received frequency range, and retrieves the PID parameter identifier corresponding to the target frequency range. Then, it sends a read command to the PID parameter loading service, carrying the PID parameter identifier obtained by the PID parameter matching module. In response to the read command, the PID parameter loading service reads the PID parameter stored in the PID database that corresponds to the received PID parameter identifier, and sends the read PID parameter to the PID parameter matching module.

[0237] S1410, the PID parameter matching module sends the read PID parameters to the OIS driver IC.

[0238] The PID parameter matching module is a software module deployed on the SOC. This module reads the PID parameters and sends them to the OIS driver IC. In practice, the SOC sends the read PID parameters to the OIS driver IC through the PID parameter matching module. Optionally, the SOC can send the PID parameters to the OIS driver IC via the I2C bus.

[0239] It should be noted that the aforementioned camera application, camera service, OIS control service, S-OIS driver module, and QMI driver module are all deployed on the SOC. The steps performed by these modules and services can all be considered as steps performed by the SOC.

[0240] The frequency range acquisition module, PID parameter matching module, PID parameter loading service, and PID parameter database are all deployed on Sensorhub. The steps performed by these modules and services can all be regarded as the steps performed by Sensorhub.

[0241] The S1411 OIS driver IC outputs control current to the OIS drive motor according to the PID parameters.

[0242] The OIS driver IC outputs control current to the OIS drive motor based on the received PID parameters. Under the control of the control current, the OIS drive motor moves accordingly, thereby moving the camera and stabilizing it, thus achieving image stabilization.

[0243] Depend on Figure 13 and Figure 14As can be seen from the embodiments, in the S-OIS architecture, the terminal device does not have an OIS controller, but instead has a Sensorhub. The Sensorhub can interact with the SOC and obtain control requests for the current scene. In response to the control requests, it extracts the frequency range of the current scene based on the angular momentum data, and matches the corresponding PID parameters according to the frequency range. This achieves dynamic adaptation of PID parameters, enabling the camera to use scene-matched PID parameters in different scenarios, thus achieving dynamic image stabilization in various scenarios, improving image stabilization performance in multiple scenarios, and enhancing the user's shooting experience.

[0244] Figure 15 This is a flowchart illustrating an example of a camera image stabilization method provided in an embodiment of this application. The method is applied to a terminal device, which includes a camera and a motor, such as... Figure 15 As shown, the method includes:

[0245] S1501. Obtain the current frequency range of the terminal device in the current scenario.

[0246] S1502. Based on the current frequency range, determine the target image stabilization parameters that match the current frequency range. The target image stabilization parameters are a set of image stabilization parameters in a preset mapping relationship. The preset mapping relationship includes at least the correspondence between the target image stabilization parameters and the target frequency range. The current frequency range and the target frequency range match.

[0247] S1503: Based on the target image stabilization parameters, drive the motor to move the camera.

[0248] The motor here is an OIS drive motor, used to move the camera lens for image stabilization. The terminal device can have preset mapping relationships in its memory, or it can download and store preset mapping relationships from the network for later retrieval. The preset mapping relationship includes the correspondence between target stabilization parameters and target frequency ranges; optionally, it can also include one-to-one correspondences between multiple other frequency ranges and multiple other stabilization parameters. It should be noted that each frequency range corresponds to a specific scenario, and each frequency range represents the range of vibration frequencies of the terminal device within that scenario.

[0249] When a user opens the camera app to take a picture, the camera app displays a preview of the shot on the screen. At this point, the upper-level camera app sends a control request to activate the image stabilization function. Optionally, the camera app can send a control request to the lower-level app to trigger the image stabilization function.

[0250] The sensors on the terminal device can continuously detect the current vibration frequency to determine the current frequency range. When the underlying module receives a control request from the upper layer, the terminal device can query a preset mapping relationship based on the current frequency range, determine the target frequency range that matches the current frequency range, and send the target stabilization parameters corresponding to the target frequency range as the parameters to be used to drive the motor to move accordingly, thereby moving the camera or camera lens accordingly to achieve image stabilization.

[0251] Optionally, determining a target frequency range that matches the current frequency range can be done by selecting a frequency range that covers the current frequency range, or by selecting a frequency range whose overlap with the current frequency range is greater than a certain percentage. For example, if the overlap between a frequency range and the current frequency range is greater than 70%, then that frequency range can be determined to match the current frequency range and be the target frequency range; alternatively, the frequency range with the highest overlap with the current frequency range can be selected as the target frequency range.

[0252] Optionally, when the current frequency range is a frequency range in two or three directions, a target frequency range that matches the current frequency range is determined. This can be achieved by having the frequency range in at least two directions or all directions of the current frequency range match the target frequency range.

[0253] Optionally, the current frequency range is a frequency range in two or three directions, and the target frequency range is also a frequency range in the same direction. To determine the target frequency range that matches the current frequency range, the frequency range in at least two directions or all directions of the current frequency range can match the frequency range in the direction corresponding to the target frequency range.

[0254] This method can dynamically adapt the image stabilization parameters to different scenarios based on the range of vibration frequencies in different scenarios. This allows the camera to use image stabilization parameters that match the scene in different scenarios, achieving dynamic image stabilization in multiple scenarios, improving image stabilization performance in various scenarios, and enhancing the user's shooting experience.

[0255] In some possible implementations, the target stabilization parameter is the target PID parameter.

[0256] PID parameters consist of three aspects: proportional gain (P), integral time (I), and derivative time (D). These PID parameters can be stored in the SOC's memory area. When the OIS driver IC needs to output a large current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor, achieving better anti-shake performance.

[0257] In some possible implementations, the preset mapping relationship includes a one-to-one correspondence between multiple sets of PID parameters and multiple frequency ranges. The multiple sets of PID parameters include target PID parameters, and the multiple frequency ranges include target frequency ranges. Each frequency range in the multiple frequency ranges corresponds to a different motion scenario.

[0258] The preset mapping relationship includes a one-to-one correspondence between multiple sets of PID parameters and multiple frequency ranges, which can cover a variety of scenarios, realize dynamic switching of PID parameters in more scenarios, and have a wider range of applications.

[0259] Among the possible implementations, the exercise scenarios include: holding with one hand, holding with both hands, walking, jogging, climbing, dancing, jumping, and sitting still.

[0260] The vibration frequency of the terminal device varies depending on the user's activity level. The preset mapping relationship defines multiple frequency ranges corresponding to these different activity scenarios, including single-handed use, two-handed use, walking, jogging, hiking, dancing, jumping, and sitting still. This comprehensive mapping covers most everyday scenarios, enhancing the applicability of the method.

[0261] In some possible implementations, obtaining the current frequency range of the terminal device in the current scenario includes: obtaining the initial angular velocity data collected by the angular momentum sensor on the terminal device in the current scenario; filtering the initial angular velocity data to obtain filtered angular velocity data; converting the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; analyzing the frequency domain data to obtain multiple vibration frequencies and the amplitudes of multiple vibration frequencies; and determining the current frequency range based on the multiple vibration frequencies and the amplitudes of multiple vibration frequencies.

[0262] The angular momentum sensor can be a gyroscope. The gyroscope on the terminal device continuously detects angular velocity. The angular velocity measured by the gyroscope over a period of time can be considered the initial angular velocity data. This initial angular velocity data includes information such as disturbances caused by external interference. Based on this, the terminal device can filter the initial angular velocity data to remove background noise and other interference, obtaining filtered angular velocity data. Both the initial and filtered angular velocity data are time-domain data.

[0263] The terminal device can also convert the filtered angular velocity data from the time domain to the frequency domain, for example, by performing a Fourier transform, to obtain the corresponding frequency domain data. Frequency domain data can intuitively reflect the frequency characteristics of the motion. Next, the terminal device analyzes the converted frequency domain data to obtain multiple vibration frequencies and their amplitudes. Based on these multiple vibration frequencies and their amplitudes, the terminal device can filter out the range of vibration frequencies with high frequency and large amplitude to form the current frequency range.

[0264] Using this method, the terminal device can detect and obtain the target frequency range of the current scene in a timely manner through its own sensors, which facilitates the subsequent accurate configuration of the corresponding image stabilization parameters.

[0265] In some possible implementations, the terminal device also includes: an angular momentum sensor, a system-on-a-chip (SOC), an OIS controller, and an OIS driver chip. The angular momentum sensor acquires initial angular velocity data and sends it to the SOC; the SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target PID parameters matching the current frequency range; the SOC sends the target PID parameters to the OIS controller; the OIS controller sends a control signal to the OIS driver chip based on the target PID parameters, the control signal carrying the target PID parameters; the OIS driver chip responds to the control signal and outputs a drive current based on the target PID parameters; the motor drives the camera to move under the action of the drive current.

[0266] In the OIS architecture, the terminal device has an OIS controller, eliminating the need for a Sensorhub. It can interact with the SOC and obtain the corresponding PID parameters for the current scene, thereby dynamically adapting the PID parameters. This allows the camera to use scene-matched PID parameters in different scenarios, achieving dynamic image stabilization in various situations, improving image stabilization performance and enhancing the user's shooting experience.

[0267] In some possible implementations, the SOC is equipped with an OIS control service, a sensor driver module, an OIS drive control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range. This includes: the OIS control service forwarding received control requests to the OIS drive control module (the control requests are sent by the camera application when it is turned on); the sensor drive module receiving the initial angular velocity data from the angular momentum sensor and sending it to the OIS drive control module; the OIS drive control module responding to the control request sending the initial angular velocity data to the frequency range extraction module; the frequency range extraction module performing time-frequency conversion on the initial angular velocity data to obtain the current frequency range; and the SOC determining the target PID parameter matching the current frequency range based on the current frequency range. This includes: the frequency range extraction module sending the current frequency range to the PID parameter matching module; the PID parameter matching module determining the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship based on the current frequency range; and the PID parameter matching module calling the PID parameter loading service to retrieve the target PID parameter from the PID parameter database based on the parameter identifier of the target PID parameter.

[0268] In some possible implementations, the terminal device also includes: an angular momentum sensor, a SOC, a smart sensor hub, and an OIS driver chip; the angular momentum sensor acquires initial angular velocity data and sends it to the smart sensor hub; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the smart sensor hub determines the target PID parameters that match the current frequency range based on the current frequency range; the smart sensor hub sends the target PID parameters to the OIS driver chip; the OIS driver chip outputs a drive current based on the target PID parameters; and the motor drives the camera to move under the action of the drive current.

[0269] In the S-OIS architecture, instead of an OIS controller, a Sensorhub is used in the terminal device. The Sensorhub can interact with the SOC and obtain control requests for the current scene. In response to the control requests, it extracts the frequency range of the current scene based on the angular momentum data and matches the corresponding PID parameters according to the frequency range. This achieves dynamic adaptation of PID parameters, enabling the camera to use scene-matched PID parameters in different scenarios, thus achieving dynamic image stabilization in various scenarios, improving image stabilization performance in multiple scenarios, and enhancing the user's shooting experience.

[0270] In some possible implementations, the S-OIS control service and QMI driver module are deployed on the SOC, while the S-OIS manager, frequency range extraction module, PID parameter loading service, PID parameter matching module, and PID parameter database are deployed on the intelligent sensor hub. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The method further includes: the S-OIS control service forwarding received control requests to the QMI driver module; the control requests being requests issued by the camera application upon startup; the QMI driver module forwarding the control requests to the S-OIS manager; and the intelligent sensor hub performing time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the S-OIS manager sending the received control requests to the frequency range... The extraction module receives initial angular velocity data from the angular momentum sensor and, in response to a control request, sends the initial angular velocity data to the frequency range extraction module. The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range. The intelligent sensor hub determines the target PID parameter matching the current frequency range based on this range. This includes: the frequency range extraction module sending the current frequency range to the PID parameter matching module; the PID parameter matching module determining the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship based on the current frequency range; and the PID parameter matching module calling the PID parameter loading service to retrieve the target PID parameter from the PID parameter database based on the parameter identifier of the target PID parameter.

[0271] In some possible implementations, whether in the SOC of the OIS architecture or the Sensorhub of the S-OIS architecture, the frequency range extraction module includes: a data filtering module, a time-domain waveform conversion module, a spectrum analysis module, and a frequency range determination module. The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the data filtering module filters the initial angular velocity data to obtain filtered angular velocity data; the time-domain waveform conversion module converts the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; the spectrum analysis module analyzes the frequency domain data to obtain multiple vibration frequencies and the amplitudes of multiple vibration frequencies; and the frequency range determination module determines the current frequency range based on the multiple vibration frequencies and the amplitudes of multiple vibration frequencies.

[0272] The foregoing has detailed examples of the methods provided in this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0273] This application can divide the camera image stabilization device into functional modules based on the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0274] Figure 16 A schematic diagram of a camera image stabilization device provided in this application is shown. The device 1600 includes:

[0275] The acquisition module 1601 is used to acquire the current frequency range of the terminal device in the current scenario.

[0276] The matching module 1602 is used to determine the target image stabilization parameters that match the current frequency range based on the current frequency range. The target image stabilization parameters are a set of image stabilization parameters in a preset mapping relationship. The preset mapping relationship includes at least the correspondence between the target image stabilization parameters and the target frequency range, and the current frequency range matches the target frequency range.

[0277] The processing module 1603 is used to drive the motor to move the camera according to the target stabilization parameters.

[0278] The specific manner in which the device 1600 performs the camera stabilization method and the beneficial effects thereof can be found in the relevant descriptions in the method embodiments, and will not be repeated here.

[0279] This application also provides an electronic device, including the processor described above. The electronic device provided in this embodiment may be... Figure 1The terminal device 100 shown is used to execute the aforementioned camera image stabilization method. When using integrated units, the terminal device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the terminal device; for example, it can support the terminal device in executing the steps performed by the display unit, detection unit, and processing unit. The storage module can be used to support the terminal device in executing stored program code and data. The communication module can be used to support communication between the terminal device and other devices.

[0280] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a digital signal processor (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other terminal devices.

[0281] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device involved in this embodiment can be a device having... Figure 1 The device with the structure shown.

[0282] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the camera stabilization method described in any of the above embodiments.

[0283] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the camera stabilization method described in the above embodiments.

[0284] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0285] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units. The replaced units may or may not be physically separate. The component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0286] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0287] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0288] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A camera image stabilization method, characterized in that, Applied to a terminal device, the terminal device including a camera, a motor, and an angular momentum sensor, the method includes: Displays a preview of the camera application; The initial X-axis angular velocity data, initial Y-axis angular velocity data, and initial Z-axis angular velocity data are acquired through the angular momentum sensor. The initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data are filtered to obtain filtered X-axis angular velocity data, filtered Y-axis angular velocity data, and filtered Z-axis angular velocity data. The filtered X-axis angular velocity data, the filtered Y-axis angular velocity data, and the filtered Z-axis angular velocity data are converted from the time domain to the frequency domain to obtain X-axis frequency domain data, Y-axis frequency domain data, and Z-axis frequency domain data, respectively. The X-axis frequency domain data, the Y-axis frequency domain data, and the Z-axis frequency domain data are analyzed respectively to obtain multiple X-axis vibration frequencies and their amplitudes, multiple Y-axis vibration frequencies and their amplitudes, and multiple Z-axis vibration frequencies and their amplitudes. Remove vibration frequencies with excessively low amplitudes from the plurality of X-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements; remove the highest and lowest vibration frequencies; select the plurality of X-axis vibration frequencies that appear most frequently; and determine the range of the distribution of the selected plurality of X-axis vibration frequencies as the current X-axis frequency range. Remove vibration frequencies with excessively low amplitudes from the plurality of Y-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements; remove the highest and lowest vibration frequencies; select the plurality of Y-axis vibration frequencies that appear most frequently; and determine the range of the distribution of the selected plurality of Y-axis vibration frequencies as the current Y-axis frequency range. Remove vibration frequencies with excessively low amplitudes from the plurality of Z-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements; remove the highest and lowest vibration frequencies; select the plurality of Z-axis vibration frequencies that appear most frequently; and determine the range of the distribution of the selected plurality of Z-axis vibration frequencies as the current Z-axis frequency range. Based on the current X-axis frequency range, the current Y-axis frequency range, and the current Z-axis frequency range, a target frequency range that matches all three frequencies is found from a preset mapping relationship. The target frequency range includes a target X-direction frequency range, a target Y-direction frequency range, and a target Z-direction frequency range. The current X-axis frequency range is within the target X-direction frequency range, the current Y-axis frequency range is within the target Y-direction frequency range, and the current Z-axis frequency range is within the target Z-direction frequency range. The preset mapping relationship includes a one-to-one correspondence between multiple frequency ranges, multiple motion scenarios, and multiple sets of PID parameters. Any one of the multiple frequency ranges represents the X-direction frequency range, Y-direction frequency range, and Z-direction frequency range of the vibration frequency of the terminal device in the corresponding motion scenario. Based on the target frequency range, a target PID parameter corresponding to the target frequency range is determined from the preset mapping relationship. The target PID parameter corresponds to the target motion scene in the preset mapping relationship. The target frequency range is the frequency range in which the terminal device is in the target motion scene. Based on the target PID parameters, the drive motor moves the camera and captures a second image.

2. The method according to claim 1, characterized in that, The various exercise scenarios include: holding with one hand, holding with both hands, walking, jogging, climbing mountains, dancing, jumping, and sitting still, some or all of which are included.

3. The method according to claim 1, characterized in that, The terminal device also includes: a system-on-a-chip (SOC), an OIS controller, and an OIS driver chip; The acquisition of initial X-axis angular velocity data, initial Y-axis angular velocity data, and initial Z-axis angular velocity data through the angular momentum sensor includes: The angular momentum sensor acquires the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data, and sends them to the SOC; The step of driving the motor to move the camera according to the target PID parameters includes: The SOC sends the target PID parameters to the OIS controller; The OIS controller sends a control signal to the OIS driver chip based on the target PID parameters, and the control signal carries the target PID parameters. The OIS driver chip responds to the control signal and outputs a drive current according to the target PID parameters; The motor drives the camera to move under the action of the driving current.

4. The method according to claim 3, characterized in that, The SOC is equipped with an OIS control service, a sensor driver module, an OIS drive control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The method involves filtering the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data to obtain filtered X-axis angular velocity data. Before obtaining filtered Y-axis angular velocity data and filtered Z-axis angular velocity data, the method includes: The SOC forwards the received control request to the OIS driver control module through the OIS control service. The control request is a request sent by the camera application when it is turned on. The SOC receives the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data sent by the angular momentum sensor through the sensor driving module, and sends them to the OIS drive control module. The SOC responds to the control request through the OIS drive control module by sending the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data received by the sensor drive module to the frequency range extraction module; The step of determining the target PID parameters corresponding to the target frequency range from the preset mapping relationship based on the target frequency range includes: The SOC sends the target frequency range to the PID parameter matching module through the frequency range extraction module; The SOC, through the PID parameter matching module, determines the parameter identifier of the target PID parameter corresponding to the target frequency range from the preset mapping relationship based on the target frequency range; The SOC, through the PID parameter matching module, retrieves the target PID parameter from the PID parameter database by calling the PID parameter loading service based on the parameter identifier of the target PID parameter.

5. The method according to claim 1, characterized in that, The terminal device also includes: a system-on-a-chip (SOC), a smart sensor hub, and an OIS driver chip; The acquisition of initial X-axis angular velocity data, initial Y-axis angular velocity data, and initial Z-axis angular velocity data through the angular momentum sensor includes: The angular momentum sensor acquires the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data, and sends them to the intelligent sensor hub; The step of driving the motor to move the camera according to the target PID parameters includes: The intelligent sensor hub sends the target PID parameters to the OIS driver chip; The OIS driver chip outputs a drive current according to the target PID parameters; The motor drives the camera to move under the action of the driving current.

6. The method according to claim 5, characterized in that, The SOC is equipped with an S-OIS control service and a QMI drive module. The intelligent sensor hub is equipped with an S-OIS manager, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module, and a PID parameter database. The PID parameter database stores multiple sets of PID parameters, including the target PID parameter. The method involves filtering the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data to obtain filtered X-axis angular velocity data. Before obtaining filtered Y-axis angular velocity data and filtered Z-axis angular velocity data, the method includes: The SOC forwards the received control request to the QMI driver module through the S-OIS control service. The control request is a request issued by the camera application when it is turned on. The SOC forwards the control request to the S-OIS manager of the smart sensor hub through the QMI driver module; The intelligent sensor hub sends the received control request to the frequency range extraction module through the S-OIS manager; The intelligent sensor hub receives the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data sent by the angular momentum sensor through the S-OIS manager, and in response to the control request, sends the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data to the frequency range extraction module. The step of determining the target PID parameters corresponding to the target frequency range from the preset mapping relationship based on the target frequency range includes: The intelligent sensor hub sends the target frequency range to the PID parameter matching module through the frequency range extraction module; The intelligent sensor hub, through the PID parameter matching module, determines the parameter identifier of the target PID parameter corresponding to the target frequency range from the preset mapping relationship according to the target frequency range; The intelligent sensor hub, through the PID parameter matching module, retrieves the target PID parameter from the PID parameter database by calling the PID parameter loading service based on the parameter identifier of the target PID parameter.

7. The method according to claim 4 or 6, characterized in that, The frequency range extraction module includes: a data filtering module, a time-domain waveform conversion module, a spectrum analysis module, and a frequency range determination module; The initial X-axis angular velocity data, initial Y-axis angular velocity data, and initial Z-axis angular velocity data are filtered to obtain filtered X-axis angular velocity data, filtered Y-axis angular velocity data, and filtered Z-axis angular velocity data, respectively. These filtered X-axis, Y-axis, and Z-axis angular velocity data are then converted from the time domain to the frequency domain to obtain X-axis frequency domain data, Y-axis frequency domain data, and Z-axis frequency domain data, respectively. The X-axis, Y-axis, and Z-axis frequency domain data are analyzed to obtain multiple X-axis vibration frequencies and their amplitudes, multiple Y-axis vibration frequencies and their amplitudes, and multiple Z-axis vibration frequencies and their amplitudes. Vibration frequencies with excessively low amplitudes are removed from the multiple X-axis vibration frequencies to filter those with amplitudes meeting certain requirements. The process involves: determining the vibration frequencies, removing the highest and lowest frequencies, selecting the most frequent X-axis vibration frequencies, and defining the range of these selected X-axis vibration frequencies as the current X-axis frequency range; removing vibration frequencies with excessively low amplitudes from the multiple Y-axis vibration frequencies to filter for frequencies with a certain amplitude requirement, removing the highest and lowest frequencies, selecting the most frequent Y-axis vibration frequencies, and defining the range of these selected Y-axis vibration frequencies as the current Y-axis frequency range; removing vibration frequencies with excessively low amplitudes from the multiple Z-axis vibration frequencies to filter for frequencies with a certain amplitude requirement, removing the highest and lowest frequencies, selecting the most frequent Z-axis vibration frequencies, and defining the range of these selected Z-axis vibration frequencies as the current Z-axis frequency range; including: The data filtering module filters the initial X-axis angular velocity data, the initial Y-axis angular velocity data, and the initial Z-axis angular velocity data respectively to obtain filtered X-axis angular velocity data, filtered Y-axis angular velocity data, and filtered Z-axis angular velocity data, and sends them to the time-domain waveform conversion module. The time-domain waveform conversion module converts the filtered X-axis angular velocity data, the filtered Y-axis angular velocity data, and the filtered Z-axis angular velocity data from the time domain to the frequency domain, respectively, to obtain X-axis frequency domain data, Y-axis frequency domain data, and Z-axis frequency domain data, and sends them to the spectrum analysis module. The spectrum analysis module analyzes the X-axis frequency domain data, the Y-axis frequency domain data, and the Z-axis frequency domain data respectively to obtain multiple X-axis vibration frequencies and their amplitudes, multiple Y-axis vibration frequencies and their amplitudes, and multiple Z-axis vibration frequencies and their amplitudes, and sends them to the frequency range determination module. The frequency range determination module removes vibration frequencies with excessively low amplitudes from the plurality of X-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements, removes the highest and lowest vibration frequencies, selects the plurality of X-axis vibration frequencies that appear most frequently, and determines the range of the selected plurality of X-axis vibration frequencies as the current X-axis frequency range. The frequency range determination module removes vibration frequencies with excessively low amplitudes from the plurality of Y-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements, removes the highest and lowest vibration frequencies, selects the plurality of Y-axis vibration frequencies that appear most frequently, and determines the range of the selected plurality of Y-axis vibration frequencies as the current Y-axis frequency range. The frequency range determination module removes vibration frequencies with excessively low amplitudes from the plurality of Z-axis vibration frequencies to filter vibration frequencies with amplitudes that meet certain requirements, removes the highest and lowest vibration frequencies, selects the plurality of Z-axis vibration frequencies that appear most frequently, and determines the range of the selected plurality of Z-axis vibration frequencies as the current Z-axis frequency range.

8. An electronic device, characterized in that, include: Processor, memory, and interface; The processor, the memory, and the interface cooperate with each other to enable the electronic device to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 7.

Images