Camera anti-shake method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of terminals, and provides a camera anti-shake method, electronic equipment and a computer readable storage medium. The method is applied to the terminal equipment, the terminal equipment comprises a camera and a motor, and the method comprises the following steps: acquiring a current frequency range of the terminal equipment in a current scene; according to the current frequency range, target anti-shake parameters matched with the current frequency range are determined, the target anti-shake parameters are one set of anti-shake parameters in a preset mapping relation, the preset mapping relation at least comprises the corresponding relation between the target anti-shake parameters and the target frequency range, and the current frequency range is matched with the target frequency range; and driving the motor to drive the camera to move according to the target anti-shake parameter. According to the method, the adaptive anti-shake parameters can be dynamically switched according to the vibration frequencies in different scenes, so that the anti-shake performance of the camera is improved, and the shooting experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a camera anti-shake method, an electronic device and a computer-readable storage medium. Background Art

[0002] With the development of terminal technology, the functions of terminal devices are becoming more and more abundant, and the user experience is also getting better and better.

[0003] Cameras can be installed on terminal devices to take photos and record videos. In order to improve the shooting effect, the camera on the terminal device can also have an anti-shake function. The terminal device can enable the anti-shake function during the shooting process to improve the shooting effect. During the anti-shake process, the motor driver chip adjusts the output current size under the action of PID parameters to drive the motor to run different distances. During the movement, the motor can drive the camera to move accordingly, so that the camera can remain relatively stable when the terminal device vibrates, thereby achieving anti-shake.

[0004] However, when the user takes photos in various motion states, the PID parameters cannot meet all motion scenes, resulting in poor shooting effects in motion scenes. Summary of the invention

[0005] The present application provides a camera anti-shake method, device, chip, electronic device, computer-readable storage medium and computer program product, which can improve the anti-shake performance and thus enhance the user's shooting experience.

[0006] In a first aspect, a camera stabilization method is provided, which is applied to a terminal device, wherein the terminal device includes a camera and a motor, and the method includes: obtaining a current frequency range of the terminal device in a current scene; determining, based on the current frequency range, a target stabilization parameter that matches the current frequency range, wherein the target stabilization parameter is a set of stabilization parameters in a preset mapping relationship, wherein the preset mapping relationship includes at least a correspondence between the target stabilization parameter and the target frequency range, and the current frequency range matches the target frequency range; and according to the target stabilization parameter, driving the motor to drive the camera to move.

[0007] The motor here is the OIS drive motor, which is used to drive the camera lens to move for anti-shake.

[0008] The terminal device can preset a preset mapping relationship in the memory, or download the preset mapping relationship from the network and store it for query. The correspondence between the target anti-shake parameter and the target frequency range in the preset mapping relationship can optionally include a one-to-one correspondence between other multiple frequency ranges and other multiple anti-shake parameters. It should be noted that each frequency range corresponds to a scenario, and each frequency range is the range of the vibration frequency of the terminal device in the corresponding scenario.

[0009] When the user opens the camera application to shoot, the camera application will display a preview of the shot on the display. At this time, the upper-layer camera application will send a control request to turn on the anti-shake function. Optionally, the camera application can send a control request to the lower layer to trigger the anti-shake function.

[0010] The sensor on the terminal device can also continuously detect the current vibration frequency to determine the current frequency range. When the bottom layer module receives the control request sent by the upper layer, the terminal device can query the preset mapping relationship according to the current frequency range, determine the target frequency range that matches the current frequency range, and use the target anti-shake parameters corresponding to the target frequency range as the parameters to be used, and send them to the bottom layer to drive the motor to move accordingly, thereby driving the camera or the camera lens to move accordingly to achieve anti-shake.

[0011] Optionally, determining the target frequency range that matches the current frequency range may be to select a frequency range that covers the current frequency range as the target frequency range, or to select a frequency range that overlaps the current frequency range by a certain percentage as the target frequency range. For example, if the ratio of a frequency range that overlaps the current frequency range to the current frequency range is greater than 70%, it may be determined that the frequency range matches the current frequency range and is the target frequency range; or the frequency range that has the highest overlap with the current frequency range may be selected as the target frequency range.

[0012] Optionally, when the current frequency range is a frequency range in two or three directions, determining the target frequency range matching the current frequency range may be that frequency ranges in at least two directions or all directions in the current frequency range match the target frequency range.

[0013] Optionally, the current frequency range is a frequency range in two or three directions, and the target frequency range is also a frequency range in the same direction. The target frequency range that matches the current frequency range is determined, and the frequency ranges in at least two directions or all directions in the current frequency range match the frequency ranges in the direction corresponding to the target frequency range.

[0014] This method can dynamically adapt the anti-shake parameters of different scenes according to the range of vibration frequencies in different scenes, so that the camera can use anti-shake parameters that match the scenes in different scenes, realize dynamic anti-shake in multiple scenes, improve the anti-shake performance in multiple scenes, and enhance the user's shooting experience.

[0015] In some possible implementations, the target anti-shake parameter is a target PID parameter.

[0016] PID parameters include proportional gain (P), integral time (I) and derivative time (D). PID parameters can be stored in the storage area of ​​​​the SOC. When the OIS driver IC needs to output a larger current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor to operate, achieving a better anti-shake effect.

[0017] In some possible implementations, the preset mapping relationship includes a one-to-one correspondence between multiple groups of PID parameters and multiple frequency ranges, the multiple groups of PID parameters include target PID parameters, the multiple frequency ranges include a target frequency range, and each frequency range in the multiple frequency ranges corresponds to a different motion scene.

[0018] The preset mapping relationship includes a one-to-one correspondence between multiple groups of PID parameters and multiple frequency ranges, which can cover a variety of scenarios, realize dynamic switching of PID parameters in more scenarios, and has a wider range of uses.

[0019] In some possible implementations, the sports scene includes: one of holding with one hand, holding with two hands, walking, jogging, climbing, dancing, jumping, and sitting still.

[0020] When the user is in different sports scenes, the frequency range of the vibration frequency of the terminal device is different. The multiple frequency ranges in the preset mapping relationship are the frequency ranges of the vibration frequency of the terminal device in different sports scenes. The multiple frequency ranges in the preset mapping relationship correspond to sports scenes such as one-handed holding, two-handed holding, walking, jogging, climbing, dancing, jumping, and sitting, which can cover most of the daily use scenes, achieve comprehensive coverage of daily life, and enhance the wide range of applicable scenarios of this method.

[0021] In some possible implementations, obtaining a current frequency range of a terminal device in a current scenario includes: obtaining initial angular velocity data collected by an angular momentum sensor on the terminal device in the current scenario; filtering the initial angular velocity data to obtain filtered angular velocity data; converting the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; analyzing the frequency domain data to obtain multiple vibration frequencies and amplitudes of the multiple vibration frequencies; and determining the current frequency range based on the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

[0022] The angular momentum sensor may be a gyroscope. The gyroscope provided on the terminal device can continuously detect the angular velocity. The angular velocity over a period of time detected by the gyroscope may be the initial angular velocity data. The initial angular velocity data includes information such as disturbances caused by external interference. Based on this, the terminal device may filter the initial angular velocity data, filter out interference information such as background noise, and obtain filtered angular velocity data. Both the initial angular velocity data and the filtered angular velocity data are time domain data.

[0023] The terminal device can also convert the filtered angular velocity data from the time domain to the frequency domain, such as performing a Fourier transform, to obtain the corresponding frequency domain data. The frequency domain data can intuitively reflect the frequency characteristics of the motion. Then, the terminal device analyzes the converted frequency domain data to obtain multiple vibration frequencies and the amplitudes of the multiple vibration frequencies. The terminal device can filter out the range of vibration frequencies with high frequency and large amplitude according to the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies to form the current frequency range.

[0024] In this way, the terminal device can detect and promptly obtain the target frequency range in the current scenario through its own sensors, which facilitates the subsequent accurate configuration of the corresponding anti-shake parameters.

[0025] In some possible implementations, the terminal device further includes: an angular momentum sensor, a system chip SOC, an OIS controller, and an OIS driver chip. The angular momentum sensor collects initial angular velocity data and sends it to the SOC; the SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target PID parameters that match the current frequency range based on the current frequency range; the SOC sends the target PID parameters to the OIS controller; the OIS controller sends a control signal to the OIS driver chip based on the target PID parameters, and the control signal carries the target PID parameters; the OIS driver chip responds to the control signal and outputs a driving current based on the target PID parameters; the motor drives the camera to move under the action of the driving current.

[0026] In the OIS architecture, an OIS controller is set up in the terminal device without the need to include a Sensorhub. It can interact with the SOC and obtain the corresponding PID parameters in the current scene, thereby dynamically adapting the PID parameters, allowing the camera to use the PID parameters that match the scene in different scenes, achieving dynamic anti-shake in various scenes, improving the anti-shake performance in various scenes, and improving the user's shooting experience.

[0027] In some possible implementations, an OIS control service, a sensor drive module, an OIS drive control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database are arranged on the SOC. The PID parameter database stores multiple groups of PID parameters, and the multiple groups of PID parameters include target PID parameters. The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the OIS control service forwards the received control request to the OIS drive control module, where the control request is a request issued by the camera application when it is turned on; the sensor drive module receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the OIS drive control module; the OIS drive control module responds to the control request and sends the initial angular velocity data to the frequency range extraction module; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target anti-shake parameter matching the current frequency range according to the current frequency range, including: the frequency range extraction module sends the current frequency range to the PID parameter matching module; the PID parameter matching module determines the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship according to the current frequency range; the PID parameter matching module calls the PID parameter loading service to obtain the target PID parameter from the PID parameter database according to the parameter identifier of the target PID parameter.

[0028] In some possible implementations, the terminal device also includes: an angular momentum sensor, a SOC, a smart sensor hub and an OIS driver chip; the angular momentum sensor collects initial angular velocity data and sends it to the smart sensor hub; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain a current frequency range; the smart sensor hub determines a target PID parameter that matches the current frequency range based on the current frequency range; the smart sensor hub sends the target PID parameter to the OIS driver chip; the OIS driver chip outputs a driving current based on the target PID parameter; and the motor drives the camera to move under the action of the driving current.

[0029] In the S-OIS architecture, the terminal device does not have an OIS controller, but a Sensorhub. The Sensorhub can interact with the SOC and obtain the control request in the current scene. In response to the control request, it extracts the frequency range of the current scene based on the angular momentum data, and matches the corresponding PID parameters based on the frequency range to achieve dynamic adaptation of the PID parameters, so that the camera can use the PID parameters that match the scene in different scenes, achieve dynamic anti-shake in multiple scenes, improve the anti-shake performance in multiple scenes, and enhance the user's shooting experience.

[0030] In some possible implementations, an S-OIS control service and a QMI driver module are arranged on the SOC, and an S-OIS manager, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database are arranged on the smart sensor hub, wherein a plurality of groups of PID parameters are stored in the PID parameter database, and the plurality of groups of PID parameters include target PID parameters; the method further comprises: the S-OIS control service forwards the received control request to the QMI driver module, wherein the control request is a request issued by the camera application when it is turned on; the QMI driver module forwards the control request to the S-OIS manager; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, comprising: the S-OIS manager sends the received control request to the frequency range The S-OIS manager receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the frequency range extraction module in response to the control request; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the intelligent sensor hub determines the target anti-shake parameters matching the current frequency range according to the current frequency range, including: the frequency range extraction module sends the current frequency range to the PID parameter matching module; the PID parameter matching module determines the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship according to the current frequency range; the PID parameter matching module calls the PID parameter loading service to obtain the target PID parameter from the PID parameter database according to the parameter identifier of the target PID parameter.

[0031] In some possible implementations, whether in the SOC in the OIS architecture or in the Sensorhub in the S-OIS architecture, the frequency range extraction module includes: a data filtering module, a time domain waveform conversion module, a spectrum analysis module, and a frequency range determination module; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the data filtering module filters the initial angular velocity data to obtain filtered angular velocity data; the time domain waveform conversion module converts the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; the spectrum analysis module analyzes the frequency domain data to obtain multiple vibration frequencies and amplitudes of multiple vibration frequencies; the frequency range determination module determines the current frequency range based on the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

[0032] In a second aspect, a camera anti-shake device is provided, comprising a unit composed of software and / or hardware, wherein the unit is used to execute any one of the methods in the technical solution described in the first aspect.

[0033] In a third aspect, an embodiment of the present application provides a chip, comprising a processor; the processor is used to read and execute a computer program stored in a memory to execute any one of the methods in the technical solution described in the first aspect.

[0034] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0035] Optionally, the chip is a SOC; Optionally, the chip is a smart sensor hub; Further optionally, the chip also includes a communication interface.

[0036] In a fourth aspect, an electronic device is provided, comprising: a processor, a memory and an interface; the processor, the memory and the interface cooperate with each other so that the electronic device executes any one of the methods in the technical solution described in the first aspect.

[0037] In a fifth aspect, an electronic device is provided, the electronic device comprising any chip in the technical solution as described in the third aspect.

[0038] In a sixth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the processor executes any one of the methods in the technical solution described in the first aspect.

[0039] In a seventh aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is run on an electronic device, the electronic device executes any one of the methods in the technical solution described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of the present application; Figure 2 is a software structure block diagram of the terminal device 100 provided in an embodiment of the present application; Figure 3 is a curve diagram of the angular velocity and frequency distribution in the X-axis direction in a one-handed holding scenario provided in an embodiment of the present application; Figure 4 The embodiment of the present application provides a curve diagram of the angular velocity and frequency distribution in the Y-axis direction in a one-handed holding scenario; Figure 5 The embodiment of the present application provides a curve diagram of angular velocity and frequency distribution in the Z-axis direction in a one-handed holding scenario; Figure 6 is a curve diagram of the angular velocity and frequency distribution in the X-axis direction in the scenario of holding with both hands provided in an embodiment of the present application; Figure 7 is a curve diagram of the angular velocity and frequency distribution in the Y-axis direction in the scenario of holding with both hands provided in an embodiment of the present application; Figure 8 is a curve diagram of the angular velocity and frequency distribution in the Z-axis direction in the scenario of holding with both hands provided in an embodiment of the present application; Fig. 9 is a hardware architecture diagram of a camera anti-shake method provided in an embodiment of the present application; Fig.10 This is a diagram of the software and hardware architecture of an OIS architecture provided in an embodiment of the present application; Fig.11 is an interactive diagram of a camera anti-shake method under an OIS architecture provided in an embodiment of the present application; Fig.12 is a flow chart of a method for determining a frequency range provided in an embodiment of the present application; Fig.13 This is a diagram of the software and hardware architecture of an S-OIS architecture provided in an embodiment of the present application; Fig.14 is an interactive diagram of a camera anti-shake method under an S-OIS architecture provided in an embodiment of the present application; Fig.15 is a flow chart of a camera anti-shake method provided in an embodiment of the present application; Fig.16 It is a structural schematic diagram of a camera anti-shake device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0042] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0043] With the development of terminal technology, the functions of terminal devices are becoming more and more abundant, and the user experience is also getting better and better. For example, a camera can be installed on the terminal device to take pictures and record videos. In order to improve the shooting effect, the camera on the terminal device can also have an anti-shake function. The terminal device can enable the anti-shake function during shooting to improve the shooting effect. During the anti-shake process, the motor driver chip adjusts the output current under the action of the PID parameters to drive the motor to run different distances. During the movement, the motor can drive the camera to move, so that the camera can remain relatively stable when the terminal device vibrates, achieving anti-shake. However, when the user is shooting in various motion states, the PID parameters cannot meet all motion scenes, resulting in poor shooting effects in some scenes.

[0044] This application provides a camera anti-shake method that can dynamically adapt different PID parameters in different scenes, and use the PID parameters adapted to the current scene for anti-shake. This method can be applied to scenes with a variety of different motion states, and improve the filming rate in low light, shaking and other environments, improve the anti-shake performance of the camera, and improve the user's shooting experience.

[0045] The camera anti-shake method provided in the embodiment of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. These terminal devices are provided with cameras. The camera is a camera with an optical image stabilizer (OIS). The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0046] For example, Figure 11 is a schematic diagram of the structure of an example of a terminal device 100 provided in an embodiment of the present application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0047] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0048] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0049] The controller may be the nerve center and command center of the terminal device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0050] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0051] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0052] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0053] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0054] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0055] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0056] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0057] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0058] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0059] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0060] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0061] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0062] The camera 193 may also be provided with a motor, which may be called an OIS drive motor, for driving the camera 193 to move, thereby achieving functions such as focusing and anti-shake.

[0063] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0064] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0065] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0066] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0067] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0068] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0069] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0070] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0071] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0072] Figure 2 1 is a software structure diagram of the terminal device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

[0073] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0074] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0075] like Figure 2 As shown, the application framework layer (framework, FWK) can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0076] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0077] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0078] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0079] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0080] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0081] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and disappear automatically after a short stay without user interaction.

[0082] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0083] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0084] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0085] The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0086] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0087] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0088] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0089] A 2D graphics engine is a drawing engine for 2D drawings.

[0090] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0091] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 and Figure 2 Taking the terminal device with the shown structure as an example, the camera anti-shake method provided in the embodiment of the present application is specifically explained in combination with the accompanying drawings and application scenarios.

[0092] Users often use terminal devices to take photos in different scenarios, such as in different motion states, such as walking, jogging, climbing, dancing, jumping, sitting still, etc.

[0093] It is found that when a user holds a terminal device in different scenarios, the vibration frequency of the terminal device is different. Based on this, the scenario in which the terminal device is located can be distinguished by the vibration frequency.

[0094] Optionally, the terminal device can collect vibration frequencies in different scenarios to obtain frequency ranges corresponding to a variety of different scenarios. Specifically, the terminal device can detect the angular velocity of the terminal device through its own gyroscope (i.e., the gyroscope sensor mentioned above). Optionally, three gyroscopes can be installed on the terminal device, which are used to detect the angular velocity of the terminal device in three directions in three-dimensional space. Taking one of the gyroscopes as an example, the gyroscope can detect the angular velocity in this direction according to a certain period and report it to the upper layer. Angular velocity is data that describes the vibration state in the time domain. The terminal device can then perform time-frequency conversion on the angular velocity and convert it into data in the frequency domain to obtain the vibration frequency in this direction.

[0095] The terminal device collects statistics on the frequency characteristics of the vibration frequencies corresponding to different scenarios, thereby obtaining the frequency ranges of the vibration frequencies corresponding to different scenarios, and forming a corresponding relationship between the frequency ranges and the scenarios.

[0096] Figures 3 to 5 The figure shows a curve diagram of angular velocity and frequency distribution in three directions of X-axis, Y-axis and Z-axis when a user holds a terminal device with one hand, wherein the X-axis, Y-axis and Z-axis are perpendicular to each other.

[0097] Figure 3 This is a curve diagram of the angular velocity and frequency distribution in the X-axis direction when the user holds the terminal device with one hand. Figure 3 In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in the scenario where the user holds the terminal device with one hand, the angular velocity in the X-axis direction is concentrated between -0.4 and 0.4 most of the time. The terminal device will Figure 3 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 3 The curve b is shown in Figure 3 In b, the horizontal axis is frequency in Hertz (Hz), and the vertical axis is amplitude (also called amplitude). Figure 3 From b in the figure, we can see that the frequencies with the largest and most concentrated amplitudes are 4.932Hz, 4.832Hz, and 5.032Hz, mainly concentrated around 5Hz.

[0098] It should be noted that Figures 3 to 8 The amplitude value in can also be rounded off to an integer as the final value, which is not limited here.

[0099] Figure 4 This is a curve diagram of the angular velocity and frequency distribution in the Y-axis direction when the user holds the terminal device with one hand. Figure 4 In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in the scenario where the user holds the terminal device with one hand, the angular velocity in the Y-axis direction is concentrated between -0.5 and 0.5 most of the time. The terminal device will Figure 4 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 4 The curve b is shown in Figure 4 In b, the horizontal axis is frequency, in Hertz (Hz), and the vertical axis is amplitude, where the amplitude indicates the number of times the frequency occurs. Figure 3 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are 3.766Hz, 3.499Hz, and 3.199Hz, mainly concentrated around 3.5Hz.

[0100] Figure 5 This is a curve diagram of the angular velocity and frequency distribution in the Z-axis direction when the user holds the terminal device with one hand. Figure 5 In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that when the user holds the terminal device with one hand, the angular velocity in the Z-axis direction is concentrated between -0.5 and 0.5 most of the time. The terminal device will Figure 5 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 5 The curve b is shown in Figure 5 In b, the horizontal axis is frequency, in Hertz (Hz), and the vertical axis is amplitude, where the amplitude indicates the number of times the frequency occurs. Figure 5 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are: 4.932Hz, 4.832Hz, and 5.032Hz, mainly concentrated around 4Hz.

[0101] Figures 6 to 8 The figure shows a curve diagram of the angular velocity and frequency distribution in the X, Y and Z directions when the user holds the terminal device with both hands.

[0102] Figure 6 This is a curve diagram of the angular velocity and frequency distribution in the X-axis direction when the user holds the terminal device with one hand. Figure 6 In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in the scenario where the user holds the terminal device with both hands, the angular velocity in the X-axis direction is concentrated between -0.4 and 0.4 most of the time. The terminal device will Figure 6 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 6 The curve b is shown in Figure 6 In b, the horizontal axis is frequency, in Hertz (Hz), and the vertical axis is amplitude, where the amplitude indicates the number of times the frequency occurs. Figure 3 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are: 0.500Hz, 0.607Hz, and 0.536Hz, mainly concentrated around 0.55Hz.

[0103] Figure 7 This is a curve diagram of the angular velocity and frequency distribution in the Y-axis direction when the user holds the terminal device with one hand. Figure 7In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in the scenario where the user holds the terminal device with one hand, the angular velocity in the Y-axis direction is concentrated between -0.3 and 0.3 most of the time. The terminal device will Figure 7 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 7 The curve b is shown in Figure 4 In b, the horizontal axis is frequency, in Hertz (Hz), and the vertical axis is amplitude, where the amplitude indicates the number of times the frequency occurs. Figure 7 From b in the figure, we can see that the frequencies with the largest and most concentrated amplitudes are: 0.607Hz, 0.750Hz, and 0.785Hz, mainly concentrated around 0.7Hz.

[0104] Figure 8 This is a curve diagram of the angular velocity and frequency distribution in the Z-axis direction when the user holds the terminal device with one hand. Figure 8 In a, the horizontal axis represents time in seconds, and the vertical axis represents angular velocity in degrees per second (dps). It can be seen that in the scenario where the user holds the terminal device with one hand, the angular velocity in the Z-axis direction is concentrated between -0.4 and 0.4 most of the time. The terminal device will Figure 8 The data in a is converted into time-frequency by, for example, Fourier transform, to obtain the corresponding frequency. The terminal device counts the frequencies at different times and can obtain the following: Figure 8 The curve b is shown in Figure 8 In b, the horizontal axis is frequency, in Hertz (Hz), and the vertical axis is amplitude, where the amplitude indicates the number of times the frequency occurs. Figure 8 As can be seen from b, the frequencies with the largest and most concentrated amplitudes are: 0.393Hz, 0.714Hz, and 0.428Hz.

[0105] Optionally, the terminal device may also collect angular velocities in various directions in other scenarios, so as to obtain vibration frequencies or frequency ranges of vibration frequencies in different directions corresponding to different scenarios.

[0106] Optionally, the terminal device may also combine the vibration frequencies or frequency ranges of vibration frequencies in different directions in each scenario into one frequency range as the vibration frequency or frequency range of vibration frequencies corresponding to the scenario. Table 1 shows the frequency ranges of different directions corresponding to different scenarios.

[0107] As shown in Table 1, the terminal device can record the frequency ranges corresponding to each direction in different scenarios respectively, or record the combined range of the frequency ranges in each direction in each scenario. The embodiments of the present application are not limited to this.

[0108] Optionally, the terminal device may also select only the frequency range in one direction or two directions as the frequency range of the corresponding scene, which is not repeated here.

[0109] After determining the correspondence between different scenarios and frequency ranges, the terminal device can debug a set of adaptive PID parameters for each scenario.

[0110] Table 1

[0111] In order to facilitate understanding of the technical solution of the present application, the working principle of PID parameters is explained here. PID parameters include three parameters: proportional gain (Proportional Gain, P), integral time (Integral Time, I) and derivative time (Derivative Time, D).

[0112] Among them, the proportional gain P is used to reflect the current error value, is proportional to the error signal, and is used to quickly reduce the error. Increasing the proportional gain will increase the response speed of the system, but it may also cause system instability or oscillation.

[0113] The integral time I is used to reflect the error accumulation over a period of time and to eliminate static errors. Adjusting the integral time can smooth the system response, but it may also cause the system response to slow down or produce integral saturation.

[0114] The derivative time D is used to reflect the rate of change of the error signal and to predict future error trends. The stability and response speed of the system can be improved by adjusting the derivative time, but it may also be sensitive to noise.

[0115] When the current output by the motor driver chip does not oscillate, the proportional gain increases, which can speed up the system's response. However, if Kp is too large, the system will become unstable.

[0116] When the current output by the motor driver chip does not oscillate, reducing the integration time will help eliminate static errors, but too small a time may cause the system to respond more slowly or the integration to saturate.

[0117] When the current output by the motor driver chip does not oscillate, increasing the differential time can improve the stability and response speed of the system, but too large a differential time will make the system sensitive to noise.

[0118] PID parameters can be stored in the storage area of ​​​​the SOC or other processors. When the OIS driver IC needs to output a larger current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor to operate.

[0119] People can debug a set of PID parameters that can ensure the anti-shake effect for each different scene. Specifically, the terminal device can be clamped with a fixture and vibrated at the vibration frequency corresponding to the scene. Then take an image in the vibrating state and judge whether the taken image meets the requirements. If the requirements are met, it means that the current PID parameters can meet the anti-shake requirements at the corresponding vibration frequency. If the requirements are not met, continue to adjust the PID parameters until the image taken meets the requirements at the corresponding vibration frequency.

[0120] Optionally, in the process of debugging PID parameters, the proportional gain, integral time and differential time may be debugged one by one respectively, and then debugged comprehensively until the effect of the captured image meets the requirements.

[0121] Optionally, judging whether the captured image meets the requirements may be to observe the image with human eyes and judge whether the image clarity is clear enough. If it is clear enough, it means that the image meets the requirements; if there is an unclear situation such as ghosting, it means that the image does not meet the requirements. It can also be determined whether the image meets the requirements by judging whether the compression ratio of the image meets the requirements. If the compression ratio is greater than or equal to a preset threshold, it means that the image meets the requirements and the shooting effect is good, that is, the anti-shake effect is good; if the compression ratio is less than the preset threshold, it means that the image does not meet the requirements and the shooting effect is not good, that is, the anti-shake effect is not good.

[0122] Based on this, the corresponding relationship between different frequency ranges and different PID parameters (or PID parameter identifiers) can be obtained, as shown in Table 2.

[0123] Table 2

[0124] The above Table 2 uses the correspondence between the PID parameter identifier and the frequency range as an example. The frequency range in Table 2 includes different frequency ranges in three directions. The frequency range in Table 2 can also select the frequency range in one direction, or select the union of the frequency ranges in the three directions as a comprehensive frequency range to correspond to different PID parameters.

[0125] Optionally, the above Table 1 and Table 2 may also be combined to obtain the corresponding relationship shown in Table 3.

[0126] Optionally, in the correspondence between the above frequency ranges and PID parameters, each group of different frequency ranges may also partially overlap, for example: 1.2-3.5 Hz corresponds to PID1, 2-4 Hz corresponds to PID2, 5.5-7.8 Hz corresponds to PID3, 5.8-14 Hz corresponds to PID4, and 11-20 Hz corresponds to PID5. Optionally, the correspondence between the frequency ranges and PID parameters may also include other frequency ranges and other correspondences between PID parameters.

[0127] After the corresponding relationships between the above-mentioned different frequency ranges and different PID parameters (PID parameter identifiers) are obtained, they can be preset in the database of the terminal device for subsequent query.

[0128] Table 3

[0129] Optionally, in the above correspondence, different scenes may also be shooting modes of the camera application of the terminal device. For example, when the shooting mode of the camera application is a video recording-related mode, it can be considered that the frequency range corresponding to the video recording-related mode is 6Hz-10Hz, and when the shooting mode of the camera application is a camera motion mode, it can be considered that the frequency range corresponding to the camera motion mode is 8Hz-15Hz. Optionally, the camera motion mode is a photo taking mode with a short exposure time.

[0130] Fig. 9 FIG. 1 shows a hardware framework diagram of an example of the camera anti-shake method used in the embodiment of the present application. Fig. 9 As shown, the gyroscope can continuously detect the angular velocity of the terminal device and transmit the angular velocity data to the OIS controller. The OIS controller can convert the angular velocity data into vibration frequency, and determine the PID parameters corresponding to the vibration frequency range from multiple sets of PID parameters through the vibration frequency detection service on the AP side, and send the determined set of PID parameters to the OIS driver chip. The OIS driver chip can output a control current to the OIS lens drive motor according to the PID parameters corresponding to the vibration frequency, which is used to drive the camera module lens to move. At the same time, the camera module lens can be sensitive to light, and the CMOS driver, under the control of the CMOS controller on the AP side (AP CMOS controller), converts the light signal sensed by the camera module lens into an electrical signal, and sends the processed electrical signal to the display screen for preview through the camera driver service on the AP side to achieve shooting.

[0131] Optionally, the HALL magnet sensor (i.e., Hall sensor) can detect the position and posture changes of the terminal device and send the detected data to the OIS driver chip to adjust the aperture size or trigger the anti-shake mechanism. When the vibration frequency changes, the terminal device can dynamically switch different PID parameters to adapt to different scenarios through the vibration frequency detection service on the AP side.

[0132] The camera anti-shake method described in the embodiment of the present application can be applied to the OIS architecture (architecture in which the camera module has its own OIS controller) and can also be applied to the S-OIS architecture (architecture in which the platform processor controls OIS). Figures 10 to 14 The architecture diagram and interaction diagram shown provide a detailed description of the camera anti-shake methods under these two architectures.

[0133] Fig.10 The software and hardware architecture diagram of the camera anti-shake method described in the embodiment of the present application is applied in the OIS architecture.

[0134] like Fig.10 As shown, Fig.10 The hardware modules in the system include: SOC, gyroscope, OIS controller, OIS driver IC and OIS driver motor. The operating system is deployed on the SOC. The software architecture of the operating system can be found in Figure 2 Related description of . Figure 2 On the basis of , the HAL layer is equipped with OIS control service, and the Kernel layer also includes frequency range extraction module, PID parameter loading service, PID parameter matching module and PID parameter database.

[0135] Fig.11 is Fig.10 An interactive diagram of the camera anti-shake method implemented based on the architecture shown. Fig.10 and Fig.11 The camera anti-shake method is described in detail. Fig.11 As shown, the method includes: S1101. The camera application sends a control request to the camera service.

[0136] When the user opens the camera application to shoot, the camera application will display a preview of the shot on the display. At this time, the camera application can send a control request to the camera service located in the FWK layer. The control request is used to trigger the anti-shake function.

[0137] S1102: The camera service sends a control request to the OIS control service.

[0138] S1103: The OIS control service sends the control request to the OIS drive control module.

[0139] After receiving the control request sent by the upper layer, the camera service sends the control request to the OIS control service of the HAL layer, and forwards the control request to the Kernel layer through the OIS control service. Optionally, the OIS control service forwards the control request to the OIS driver control module of the Kernel layer.

[0140] Among them, the OIS control service is a service of the CSL (Common Services Layer), which can indicate the corresponding resources through the handle of the CSL service to achieve communication with the Kernel layer.

[0141] S1104. The gyroscope reports angular momentum data to the sensor drive module of the SOC.

[0142] The gyroscope (Gyr or G for short) can continuously detect angular momentum data, that is, angular velocity, and continuously report the angular momentum data to the corresponding driver module in the Kernel layer (here is the sensor driver module).

[0143] Optionally, the gyroscope can also be integrated with an acceleration sensor (Acc or A for short), called an Acc / Gyr sensor, or an A+G sensor. When the A+G sensor is used, the sensor driver module in the SOC can be an acceleration / gyroscope driver module for reading angular momentum data.

[0144] The accelerometer can also detect the shaking of the terminal device and transmit the detected signal to the corresponding processor. The processor will immediately calculate the displacement that needs to be compensated, and then perform reverse compensation according to the shaking direction and displacement of the lens by moving one or more lenses in the camera lens (i.e. compensation lens group), thereby achieving anti-shake.

[0145] In the embodiment of the present application, the sensor is described as a gyroscope for exemplary purposes, but it may also be other sensors, which is not limited in the embodiment of the present application.

[0146] Optionally, the gyroscope can transmit angular momentum data to the SOC via the I2C bus.

[0147] It should be noted that the order of the above steps S1104 and S1101 to S1103 is not limited.

[0148] S1105 . The sensor driving module sends the angular momentum data to the OIS driving control module.

[0149] The sensor driving module can send the received angular momentum data to the OIS driving control module, and the OIS driving control module distributes the angular momentum data to the corresponding processing module for corresponding processing.

[0150] The sensor driver module is a software module deployed on the SOC. The gyroscope can transmit angular momentum data to the SOC through the I2C bus. In fact, the gyroscope sends angular momentum data to the sensor driver module deployed on the SOC through the I2C bus.

[0151] S1106. The OIS driving control module sends the angular momentum data to the frequency range extraction module.

[0152] When the OIS drive control module receives a control request from the upper layer, it responds to the control request and sends the angular momentum data transmitted from the lower layer to the frequency range extraction module, which is used to determine the vibration frequency or frequency range of the current scene based on the angular momentum data.

[0153] S1107. The frequency range extraction module calculates the corresponding frequency range according to the angular momentum data.

[0154] Optionally, the frequency range extraction module includes: a data filtering module, a time domain waveform conversion module, a spectrum analysis module and a frequency range determination module. Specifically, the OIS drive control module can send the angular momentum data to the data filtering module in the frequency range extraction module.

[0155] The specific process of the frequency range extraction module determining the corresponding frequency range based on the angular momentum data can be found in Fig.12 As shown, including: S1201. A data filtering module filters the angular momentum data.

[0156] The angular momentum data here refers to the angular velocity collected over a period of time, which is time domain data.

[0157] The data filtering module performs filtering, denoising and other processing operations on the angular momentum data, and can filter out interference information such as bottom noise. The angular momentum data after filtering is still time domain data. The data filtering module sends the filtered angular momentum data to the time domain waveform conversion module.

[0158] S1202. The time domain waveform conversion module performs time-frequency conversion on the filtered angular momentum data to obtain corresponding frequency domain data.

[0159] The time domain waveform conversion module can perform time-frequency conversion on the received filtered angular momentum data, such as Fourier transform, to obtain corresponding frequency domain data. The frequency domain data at this time can characterize the vibration frequency in the current scenario. The time domain waveform conversion module sends the converted frequency domain data to the spectrum analysis module.

[0160] It should be noted that the frequency domain data here represent the vibration frequency and amplitude over a period of time.

[0161] S1203, the spectrum analysis module analyzes the frequency domain data to obtain the vibration frequency and amplitude.

[0162] The spectrum analysis module uses a spectrum analysis algorithm to analyze the frequency domain data to obtain the corresponding vibration frequency and amplitude. Optionally, the vibration frequency obtained here is the vibration frequency of the angular momentum data distributed over a period of time, and the vibration frequency may fluctuate at different times during this period of time.

[0163] The spectrum analysis module sends the vibration frequency and amplitude obtained by analysis to the frequency range determination module.

[0164] S1204: The frequency range determination module determines the frequency range according to the received vibration frequency and amplitude.

[0165] After receiving the vibration frequencies within a period of time, the frequency range determination module analyzes the vibration frequencies at multiple moments within the period of time to obtain the frequency ranges corresponding to the vibration frequencies.

[0166] For example, the frequency range determination module can delete the vibration frequencies with too low amplitudes from the vibration frequencies at multiple moments during this period, screen the vibration frequencies with amplitudes that meet certain requirements, and delete the highest and lowest vibration frequencies. It can also select multiple vibration frequencies with the highest frequency of occurrence, and use the distribution range of the selected multiple vibration frequencies as the final frequency range.

[0167] Optionally, Fig.12 In the embodiment, the above processing may be performed on the data of angular momentum in one direction to obtain the frequency range in the direction; or the above processing may be performed on the data of angular momentum in three directions respectively to obtain the frequency ranges of the three directions respectively. Optionally, the frequency range determination module may also merge the frequency ranges of the three directions respectively into one frequency range.

[0168] S1108. The frequency range extraction module sends the frequency range to the PID parameter matching module.

[0169] After extracting the frequency range corresponding to the current scene, the frequency range extraction module may send the frequency range to the PID parameter matching module.

[0170] S1109. The PID parameter matching module calls the PID parameter loading service according to the received frequency range to obtain PID parameters matching the frequency range from the PID parameter database.

[0171] The PID parameter database stores multiple groups of PID parameters. These multiple groups of PID parameters correspond to different PID parameter identifiers, such as PID1, PID2, and PID3, etc. The PID parameter identifier here can be the name, serial number, and other identifiers of the PID parameter. The embodiment of the present application does not limit the specific form of the PID parameter identifier, as long as it can distinguish different PID parameters.

[0172] The PID parameter matching module can query the correspondence between multiple preset PID parameter identifiers and multiple frequency ranges, determine a frequency range that matches the received frequency range as the target frequency range, and then use the PID parameter identifier corresponding to the target frequency range as the query result.

[0173] Optionally, the PID parameter matching module determines that a frequency range that matches the received frequency range may be to select one frequency range that includes the received frequency range from multiple frequency ranges, or to select one frequency range that has the highest overlap with the received frequency range from multiple frequency ranges as the target frequency range. This embodiment of the application is not limited to this.

[0174] The PID parameter matching module calls the PID parameter loading service, and reads a group of PID parameters corresponding to the PID parameter identifier in the query result from the PID database through the PID parameter loading service.

[0175] For example, after the PID parameter matching module receives the frequency range, it queries the correspondence between multiple PID parameter identifiers and multiple frequency ranges to obtain a target frequency range that matches the received frequency range, and obtains the PID parameter identifier corresponding to the target frequency range. Then, a read instruction is sent to the PID parameter loading service, and the read instruction carries the PID parameter identifier obtained by the PID parameter matching module. In response to the read instruction, the PID parameter loading service reads the PID parameters stored in the PID database that correspond to the received PID parameter identifier, and sends the read PID parameters to the PID parameter matching module.

[0176] S1110. The PID parameter matching module sends the read PID parameters to the OIS controller.

[0177] The PID parameter matching module is a software module deployed on the SOC. The PID parameter matching module sends the read PID parameters to the OIS controller. In fact, the SOC sends the read PID parameters to the OIS controller through the PID parameter matching module. Optionally, the SOC can send the PID parameters to the OIS controller through the I2C bus.

[0178] It should be noted that the above-mentioned camera application, camera service, OIS control service, sensor driver module, frequency range acquisition module, PID parameter matching module, PID parameter loading service and PID parameter database are all deployed on the SOC. The steps executed by these modules and services can be regarded as the steps executed by the SOC.

[0179] S1111. The OIS controller sends a control signal to the OIS driver IC, where the control signal carries a PID parameter.

[0180] The OIS controller sends a control signal to the OIS driver IC according to the received PID parameters, thereby sending the PID parameters to the OIS driver IC.

[0181] Optionally, the OIS controller can also obtain data collected by the gyroscope and / or acceleration sensor through the SPI bus for performing other controls, which will not be described in detail here.

[0182] S1112 , the OIS driving IC outputs a control current to the OIS driving motor in response to the control signal.

[0183] The OIS driver IC outputs a control current to the OIS driver motor according to the PID parameters carried by the control signal. The OIS driver motor moves accordingly under the control of the control current, thereby driving the camera to move, making the camera relatively stable, thereby achieving anti-shake.

[0184] The camera is also equipped with a flash, and has software and hardware structures related to the auto focus (AF) function.

[0185] Depend on Fig.10 and Fig.11 It can be seen from the implementation examples that in the OIS architecture, an OIS controller is provided in the terminal device, which can interact with the SOC and obtain the corresponding PID parameters in the current scene, thereby realizing dynamic adaptation of the PID parameters, so that the camera can use PID parameters matching the scene in different scenes, realize dynamic anti-shake in various scenes, improve the anti-shake performance in various scenes, and enhance the user's shooting experience.

[0186] Fig.13 The software and hardware architecture diagram of the camera anti-shake method described in the embodiment of the present application is applied in the S-OIS architecture.

[0187] like Fig.13 As shown, Fig.13 The hardware modules in the S-OIS architecture include: SOC, gyroscope, smart sensor hub (Sensorhub), OIS driver IC and OIS driver motor. The gyroscope, OIS driver IC and OIS driver motor can be regarded as belonging to the hardware layer. Compared with the OIS architecture, the S-OIS architecture does not include an OIS controller, and uses the Sensorhub to implement related algorithms. Specifically, an operating system is deployed on the SOC. The software architecture of the operating system can be seen in Figure 2 Related description of . Figure 2 On the basis of the HAL layer, the S-OIS control service is set up, and the Kernel layer also includes the QMI driver module to realize the information interaction between the SOC and the Sensorhub. The S-OIS manager, frequency range extraction module, PID parameter loading service, PID parameter matching module and PID parameter database are deployed in the Sensorhub.

[0188] Fig.14 is Fig.13 An interactive diagram of the camera anti-shake method implemented based on the architecture shown. Fig.13 and Fig.14 The camera anti-shake method is described in detail. Fig.14 As shown, the method includes: S1401. The camera application sends a control request to the camera service.

[0189] When the user opens the camera application to shoot, the camera application will display a preview of the shot on the display. At this time, the camera application can send a control request to the camera service located in the FWK layer. The control request is used to trigger the anti-shake function.

[0190] S1402: The camera service sends a control request to the S-OIS control service.

[0191] S1403. The S-OIS control service sends the control request to the QMI driver module.

[0192] After receiving the control request sent by the upper layer, the camera service sends the control request to the S-OIS control service of the HAL layer, and forwards the control request to the Kernel layer through the SOIS control service. Optionally, the S-OIS control service forwards the control request to the OMI driver module of the Kernel layer.

[0193] Among them, S-OIS control service is a service of NCS (Network Control System), which can indicate the corresponding resources through the handle of NCS service to realize communication with Kernel layer.

[0194] S1404. The QMI driver module sends the control request to the S-OIS manager.

[0195] The QMI driver module is placed on the SOC, and the S-OIS manager is placed on the Sensorhub. The QMI driver module sends the control request to the S-OIS manager, which is actually the SOC sending the control request to the S-OIS manager on the Sensorhub through the QMI driver module.

[0196] S1405. The gyroscope reports angular momentum data to the S-OIS manager.

[0197] The gyroscope can continuously detect angular momentum data, that is, angular velocity, and continuously report the angular momentum data to the S-OIS manager deployed in the Sensorhub.

[0198] Optionally, the gyroscope can transmit angular momentum data to the Sensorhub via the SPI bus.

[0199] Optionally, it should be noted that the order of the above steps S1405 and S1401 to S1404 is not limited.

[0200] S1406. The S-OIS manager sends the angular momentum data to the frequency range extraction module.

[0201] The S-OIS manager can be an interface for other hardware, such as SOC, gyroscope and other sensors to interact with the Sensorhub.

[0202] When the S-OIS manager receives a control request sent by the upper layer, it responds to the control request and sends the angular momentum data transmitted by the lower layer to the frequency range extraction module, wherein the frequency range extraction module is used to determine the vibration frequency or the frequency range of the vibration frequency in the current scene according to the angular momentum data.

[0203] S1407. The frequency range extraction module calculates the corresponding frequency range according to the angular momentum data.

[0204] Optionally, the frequency range extraction module includes: a data filtering module, a time domain waveform conversion module, a spectrum analysis module and a frequency range determination module. Specifically, the S-OIS manager can send the angular momentum data to the data filtering module in the frequency range extraction module.

[0205] The software architecture of the frequency range extraction module and the specific process of determining the corresponding frequency range based on angular momentum data can be found in Fig.12 And related descriptions will not be repeated here.

[0206] S1408. The frequency range extraction module sends the frequency range to the PID parameter matching module.

[0207] After extracting the frequency range corresponding to the current scene, the frequency range extraction module may send the frequency range to the PID parameter matching module.

[0208] S1409. The PID parameter matching module calls the PID parameter loading service according to the received frequency range to obtain PID parameters matching the frequency range from the PID parameter database.

[0209] The PID parameter database stores multiple groups of PID parameters. These multiple groups of PID parameters correspond to different PID parameter identifiers, such as PID1, PID2, and PID3, etc. The PID parameter identifier here can be the name, serial number, and other identifiers of the PID parameter. The embodiment of the present application does not limit the specific form of the PID parameter identifier, as long as it can distinguish different PID parameters.

[0210] The PID parameter matching module can query the correspondence between multiple preset PID parameter identifiers and multiple frequency ranges, determine a frequency range that matches the received frequency range as the target frequency range, and then use the PID parameter identifier corresponding to the target frequency range as the query result.

[0211] Optionally, the PID parameter matching module determines that a frequency range that matches the received frequency range may be to select one frequency range that includes the received frequency range from multiple frequency ranges, or to select one frequency range that has the highest overlap with the received frequency range from multiple frequency ranges as the target frequency range. This embodiment of the application is not limited to this.

[0212] The PID parameter matching module calls the PID parameter loading service, and reads a group of PID parameters corresponding to the PID parameter identifier in the query result from the PID database through the PID parameter loading service.

[0213] For example, after the PID parameter matching module receives the frequency range, it queries the correspondence between multiple PID parameter identifiers and multiple frequency ranges to obtain a target frequency range that matches the received frequency range, and obtains the PID parameter identifier corresponding to the target frequency range. Then, a read instruction is sent to the PID parameter loading service, and the read instruction carries the PID parameter identifier obtained by the PID parameter matching module. In response to the read instruction, the PID parameter loading service reads the PID parameters stored in the PID database that correspond to the received PID parameter identifier, and sends the read PID parameters to the PID parameter matching module.

[0214] S1410 , the PID parameter matching module sends the read PID parameters to the OIS driver IC.

[0215] The PID parameter matching module is a software module deployed on the SOC. The PID parameter matching module sends the read PID parameters to the OIS driver IC. In fact, the SOC sends the read PID parameters to the OIS driver IC through the PID parameter matching module. Optionally, the SOC can send the PID parameters to the OIS driver IC through the I2C bus.

[0216] It should be noted that the above-mentioned camera application, camera service, OIS control service, S-OIS driver module and QMI driver module are all deployed on the SOC. The steps executed by these modules and services can be regarded as the steps executed by the SOC.

[0217] The frequency range acquisition module, PID parameter matching module, PID parameter loading service and PID parameter database are all deployed on Sensorhub. The steps performed by these modules and services can be regarded as the steps performed by Sensorhub.

[0218] S1411 . The OIS driving IC outputs a control current to the OIS driving motor according to the PID parameters.

[0219] The OIS driver IC outputs a control current to the OIS driver motor according to the received PID parameters. The OIS driver motor moves accordingly under the control of the control current, thereby driving the camera to move, making the camera relatively stable, thereby achieving anti-shake.

[0220] Depend on Fig.13 and Fig.14It can be seen from the embodiments that in the S-OIS architecture, the OIS controller is not set in the terminal device, but the Sensorhub is set. The Sensorhub can interact with the SOC and obtain the control request in the current scene. In response to the control request, the frequency range of the current scene is extracted according to the angular momentum data, and the corresponding PID parameters are matched according to the frequency range to achieve dynamic adaptation of the PID parameters, so that the camera can use the PID parameters that match the scene in different scenes, realize dynamic anti-shake in various scenes, improve the anti-shake performance in various scenes, and improve the user's shooting experience.

[0221] Fig.15 is a flow chart of a camera anti-shake method provided in an embodiment of the present application. The method is applied to a terminal device, the terminal device includes a camera and a motor, such as Fig.15 As shown, the method includes: S1501. Obtain the current frequency range of the terminal device in the current scenario.

[0222] S1502: Determine, according to the current frequency range, a target anti-shake parameter that matches the current frequency range, where the target anti-shake parameter is a set of anti-shake parameters in a preset mapping relationship, where the preset mapping relationship at least includes a correspondence between the target anti-shake parameter and the target frequency range, and the current frequency range matches the target frequency range.

[0223] S1503: According to the target anti-shake parameter, the driving motor drives the camera to move.

[0224] The motor here is an OIS drive motor, which is used to drive the camera lens to move for anti-shake. The terminal device can preset a preset mapping relationship in the memory, or download the preset mapping relationship from the network and store it for query. The correspondence between the target anti-shake parameter and the target frequency range in the preset mapping relationship may optionally include a one-to-one correspondence between other multiple frequency ranges and other multiple anti-shake parameters. It should be noted that each frequency range corresponds to a scene, and each frequency range is the range of the vibration frequency of the terminal device in the corresponding scene.

[0225] When the user opens the camera application to shoot, the camera application will display a preview of the shot on the display. At this time, the upper-layer camera application will send a control request to turn on the anti-shake function. Optionally, the camera application can send a control request to the lower layer to trigger the anti-shake function.

[0226] The sensor on the terminal device can also continuously detect the current vibration frequency to determine the current frequency range. When the bottom layer module receives the control request sent by the upper layer, the terminal device can query the preset mapping relationship according to the current frequency range, determine the target frequency range that matches the current frequency range, and use the target anti-shake parameters corresponding to the target frequency range as the parameters to be used, and send them to the bottom layer to drive the motor to move accordingly, thereby driving the camera or the camera lens to move accordingly to achieve anti-shake.

[0227] Optionally, determining the target frequency range that matches the current frequency range may be to select a frequency range that covers the current frequency range as the target frequency range, or to select a frequency range that overlaps the current frequency range by a certain percentage as the target frequency range. For example, if the ratio of a frequency range that overlaps the current frequency range to the current frequency range is greater than 70%, it may be determined that the frequency range matches the current frequency range and is the target frequency range; or the frequency range that has the highest overlap with the current frequency range may be selected as the target frequency range.

[0228] Optionally, when the current frequency range is a frequency range in two or three directions, determining the target frequency range matching the current frequency range may be that frequency ranges in at least two directions or all directions in the current frequency range match the target frequency range.

[0229] Optionally, the current frequency range is a frequency range in two or three directions, and the target frequency range is also a frequency range in the same direction. The target frequency range that matches the current frequency range is determined, and the frequency ranges in at least two directions or all directions in the current frequency range match the frequency ranges in the direction corresponding to the target frequency range.

[0230] This method can dynamically adapt the anti-shake parameters of different scenes according to the range of vibration frequencies in different scenes, so that the camera can use anti-shake parameters that match the scenes in different scenes, realize dynamic anti-shake in multiple scenes, improve the anti-shake performance in multiple scenes, and enhance the user's shooting experience.

[0231] In some possible implementations, the target anti-shake parameter is a target PID parameter.

[0232] PID parameters include proportional gain (P), integral time (I) and derivative time (D). PID parameters can be stored in the storage area of ​​​​the SOC. When the OIS driver IC needs to output a larger current, it will perform linear compensation based on the PID parameters and output a linear current to drive the OIS motor to operate, achieving a better anti-shake effect.

[0233] In some possible implementations, the preset mapping relationship includes a one-to-one correspondence between multiple groups of PID parameters and multiple frequency ranges, the multiple groups of PID parameters include target PID parameters, the multiple frequency ranges include a target frequency range, and each frequency range in the multiple frequency ranges corresponds to a different motion scene.

[0234] The preset mapping relationship includes a one-to-one correspondence between multiple groups of PID parameters and multiple frequency ranges, which can cover a variety of scenarios, realize dynamic switching of PID parameters in more scenarios, and has a wider range of uses.

[0235] In some possible implementations, the sports scene includes: one of holding with one hand, holding with two hands, walking, jogging, climbing, dancing, jumping, and sitting still.

[0236] When the user is in different sports scenes, the frequency range of the vibration frequency of the terminal device is different. The multiple frequency ranges in the preset mapping relationship are the frequency ranges of the vibration frequency of the terminal device in different sports scenes. The multiple frequency ranges in the preset mapping relationship correspond to sports scenes such as one-handed holding, two-handed holding, walking, jogging, climbing, dancing, jumping, and sitting, which can cover most of the daily use scenes, achieve comprehensive coverage of daily life, and enhance the wide range of applicable scenarios of this method.

[0237] In some possible implementations, obtaining a current frequency range of a terminal device in a current scenario includes: obtaining initial angular velocity data collected by an angular momentum sensor on the terminal device in the current scenario; filtering the initial angular velocity data to obtain filtered angular velocity data; converting the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; analyzing the frequency domain data to obtain multiple vibration frequencies and amplitudes of the multiple vibration frequencies; and determining the current frequency range based on the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

[0238] The angular momentum sensor may be a gyroscope. The gyroscope provided on the terminal device can continuously detect the angular velocity. The angular velocity over a period of time detected by the gyroscope may be the initial angular velocity data. The initial angular velocity data includes information such as disturbances caused by external interference. Based on this, the terminal device may filter the initial angular velocity data, filter out interference information such as background noise, and obtain filtered angular velocity data. Both the initial angular velocity data and the filtered angular velocity data are time domain data.

[0239] The terminal device can also convert the filtered angular velocity data from the time domain to the frequency domain, such as performing a Fourier transform, to obtain the corresponding frequency domain data. The frequency domain data can intuitively reflect the frequency characteristics of the motion. Then, the terminal device analyzes the converted frequency domain data to obtain multiple vibration frequencies and the amplitudes of the multiple vibration frequencies. The terminal device can filter out the range of vibration frequencies with high frequency and large amplitude according to the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies to form the current frequency range.

[0240] In this way, the terminal device can detect and promptly obtain the target frequency range in the current scenario through its own sensors, which facilitates the subsequent accurate configuration of the corresponding anti-shake parameters.

[0241] In some possible implementations, the terminal device further includes: an angular momentum sensor, a system chip SOC, an OIS controller, and an OIS driver chip. The angular momentum sensor collects initial angular velocity data and sends it to the SOC; the SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target PID parameters that match the current frequency range based on the current frequency range; the SOC sends the target PID parameters to the OIS controller; the OIS controller sends a control signal to the OIS driver chip based on the target PID parameters, and the control signal carries the target PID parameters; the OIS driver chip responds to the control signal and outputs a driving current based on the target PID parameters; the motor drives the camera to move under the action of the driving current.

[0242] In the OIS architecture, an OIS controller is set up in the terminal device without the need to include a Sensorhub. It can interact with the SOC and obtain the corresponding PID parameters in the current scene, thereby dynamically adapting the PID parameters, allowing the camera to use the PID parameters that match the scene in different scenes, achieving dynamic anti-shake in various scenes, improving the anti-shake performance in various scenes, and improving the user's shooting experience.

[0243] In some possible implementations, an OIS control service, a sensor drive module, an OIS drive control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database are arranged on the SOC. The PID parameter database stores multiple groups of PID parameters, and the multiple groups of PID parameters include target PID parameters. The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the OIS control service forwards the received control request to the OIS drive control module, the control request is a request issued by the camera application when it is turned on; the sensor drive module receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the OIS drive control module; the OIS drive control module responds to the control request and sends the initial angular velocity data to the frequency range extraction module; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the SOC determines the target PID parameter matching the current frequency range according to the current frequency range, including: the frequency range extraction module sends the current frequency range to the PID parameter matching module; the PID parameter matching module determines the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship according to the current frequency range; the PID parameter matching module calls the PID parameter loading service to obtain the target PID parameter from the PID parameter database according to the parameter identifier of the target PID parameter.

[0244] In some possible implementations, the terminal device also includes: an angular momentum sensor, a SOC, a smart sensor hub and an OIS driver chip; the angular momentum sensor collects initial angular velocity data and sends it to the smart sensor hub; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain a current frequency range; the smart sensor hub determines a target PID parameter that matches the current frequency range based on the current frequency range; the smart sensor hub sends the target PID parameter to the OIS driver chip; the OIS driver chip outputs a driving current based on the target PID parameter; and the motor drives the camera to move under the action of the driving current.

[0245] In the S-OIS architecture, the terminal device does not have an OIS controller, but a Sensorhub. The Sensorhub can interact with the SOC and obtain the control request in the current scene. In response to the control request, it extracts the frequency range of the current scene based on the angular momentum data, and matches the corresponding PID parameters based on the frequency range to achieve dynamic adaptation of the PID parameters, so that the camera can use the PID parameters that match the scene in different scenes, achieve dynamic anti-shake in multiple scenes, improve the anti-shake performance in multiple scenes, and enhance the user's shooting experience.

[0246] In some possible implementations, an S-OIS control service and a QMI driver module are arranged on the SOC, and an S-OIS manager, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database are arranged on the smart sensor hub, and multiple groups of PID parameters are stored in the PID parameter database, and the multiple groups of PID parameters include target PID parameters; the method also includes: the S-OIS control service forwards the received control request to the QMI driver module, and the control request is a request issued by the camera application when it is turned on; the QMI driver module forwards the control request to the S-OIS manager; the smart sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the S-OIS manager sends the received control request to the frequency range Extraction module; the S-OIS manager receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the frequency range extraction module in response to the control request; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; the intelligent sensor hub determines the target PID parameters matching the current frequency range according to the current frequency range, including: the frequency range extraction module sends the current frequency range to the PID parameter matching module; the PID parameter matching module determines the parameter identifier of the target PID parameter matching the current frequency range from a preset mapping relationship according to the current frequency range; the PID parameter matching module calls the PID parameter loading service to obtain the target PID parameters from the PID parameter database according to the parameter identifier of the target PID parameter.

[0247] In some possible implementations, whether in the SOC in the OIS architecture or in the Sensorhub in the S-OIS architecture, the frequency range extraction module includes: a data filtering module, a time domain waveform conversion module, a spectrum analysis module, and a frequency range determination module; the frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: the data filtering module filters the initial angular velocity data to obtain filtered angular velocity data; the time domain waveform conversion module converts the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; the spectrum analysis module analyzes the frequency domain data to obtain multiple vibration frequencies and amplitudes of multiple vibration frequencies; the frequency range determination module determines the current frequency range based on the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

[0248] The above describes in detail an example of the method provided by the present application. It is understandable that, in order to implement the above functions, the corresponding device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0249] The present application can divide the functional modules of the camera anti-shake device according to the above method example. For example, each function can be divided into each functional module, or two or more functions can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0250] Fig.16 The schematic diagram of the structure of a camera anti-shake device provided by the present application is shown. The device 1600 includes: The acquisition module 1601 is used to acquire the current frequency range of the terminal device in the current scenario.

[0251] The matching module 1602 is used to determine, according to the current frequency range, a target anti-shake parameter that matches the current frequency range, where the target anti-shake parameter is a set of anti-shake parameters in a preset mapping relationship, where the preset mapping relationship at least includes a correspondence between the target anti-shake parameter and the target frequency range, and the current frequency range matches the target frequency range.

[0252] The processing module 1603 is used to drive the motor to drive the camera to move according to the target anti-shake parameter.

[0253] The specific manner in which the device 1600 executes the camera anti-shake method and the beneficial effects produced can be found in the relevant description in the method embodiment, which will not be repeated here.

[0254] The present application also provides an electronic device, including the above processor. The electronic device provided by this embodiment can be Figure 1The terminal device 100 shown is used to perform the above-mentioned camera anti-shake method. In the case of an integrated unit, the terminal device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the terminal device, for example, it can be used to support the terminal device to execute the steps performed by the display unit, the detection unit and the processing unit. The storage module can be used to support the terminal device to execute stored program codes and data, etc. The communication module can be used to support the communication between the terminal device and other devices.

[0255] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other terminal devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0256] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device involved in this embodiment may be a Figure 1 Device of the structure shown.

[0257] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the camera anti-shake method described in any of the above embodiments.

[0258] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the camera anti-shake method in the above-mentioned embodiment.

[0259] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0260] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, the replaced units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0261] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0262] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0263] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A camera anti-shake method, characterized in that: Applied to a terminal device, the terminal device includes a camera and a motor, including: Obtaining a current frequency range of the terminal device in a current scenario; Determine, according to the current frequency range, a target anti-shake parameter matching the current frequency range, the target anti-shake parameter being a set of anti-shake parameters in a preset mapping relationship, the preset mapping relationship at least including a correspondence between the target anti-shake parameter and a target frequency range, and the current frequency range matches the target frequency range; According to the target anti-shake parameter, a driving motor drives the camera to move.

2. The method according to claim 1, characterized in that The target anti-shake parameter is a target PID parameter.

3. The method according to claim 2, characterized in that The preset mapping relationship includes a one-to-one correspondence between multiple groups of PID parameters and multiple frequency ranges, the multiple groups of PID parameters include the target PID parameters, the multiple frequency ranges include the target frequency range, and each frequency range in the multiple frequency ranges corresponds to a different motion scene.

4. The method according to claim 3, characterized in that The sports scene includes one of: holding with one hand, holding with both hands, walking, jogging, climbing, dancing, jumping, and sitting still.

5. The method according to any one of claims 1 to 4, characterized in that The obtaining of the current frequency range of the terminal device in the current scenario includes: Acquire initial angular velocity data collected by an angular momentum sensor on the terminal device in the current scene; Filtering the initial angular velocity data to obtain filtered angular velocity data; Converting the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; Analyzing the frequency domain data to obtain a plurality of vibration frequencies and amplitudes of the plurality of vibration frequencies; The current frequency range is determined according to the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

6. The method according to claim 2, characterized in that The terminal device also includes: an angular momentum sensor, a system chip SOC, an OIS controller and an OIS driver chip; The obtaining of the current frequency range of the terminal device in the current scenario includes: The angular momentum sensor collects initial angular velocity data and sends it to the SOC; The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; The determining, according to the current frequency range, a target anti-shake parameter matching the current frequency range includes: The SOC determines, according to the current frequency range, the target PID parameter matching the current frequency range; The driving motor drives the camera to move according to the target anti-shake parameter, including: The SOC sends the target PID parameter to the OIS controller; The OIS controller sends a control signal to the OIS driver chip according to the target PID parameter, wherein the control signal carries the target PID parameter; The OIS driving chip outputs a driving current according to the target PID parameter in response to the control signal; The motor drives the camera to move under the action of the driving current.

7. The method according to claim 6, characterized in that The SOC is provided with an OIS control service, a sensor driving module, an OIS driving control module, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database, wherein the PID parameter database stores multiple groups of PID parameters, and the multiple groups of PID parameters include the target PID parameters; The SOC performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: The OIS control service forwards the received control request to the OIS drive control module, wherein the control request is a request issued by the camera application when it is turned on; The sensor driving module receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the OIS driving control module; The OIS driving control module sends the initial angular velocity data to the frequency range extraction module in response to the control request; The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; The SOC determines, according to the current frequency range, the target PID parameter matching the current frequency range, including: The frequency range extraction module sends the current frequency range to the PID parameter matching module; The PID parameter matching module determines, according to the current frequency range, a parameter identifier of the target PID parameter that matches the current frequency range from the preset mapping relationship; The PID parameter matching module calls the PID parameter loading service to obtain the target PID parameter from the PID parameter database according to the parameter identifier of the target PID parameter.

8. The method according to claim 2, characterized in that: The terminal device also includes: an angular momentum sensor, a SOC, an intelligent sensor hub and an OIS driver chip; The obtaining of the current frequency range of the terminal device in the current scenario includes: The angular momentum sensor collects initial angular velocity data and sends it to the intelligent sensor hub; The intelligent sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; The determining, according to the current frequency range, a target anti-shake parameter matching the current frequency range includes: The intelligent sensor hub determines the target PID parameter matching the current frequency range according to the current frequency range; The driving motor drives the camera to move according to the target anti-shake parameter, including: The smart sensor hub sends the target PID parameters to the OIS driver chip; The OIS driving chip outputs a driving current according to the target PID parameter; The motor drives the camera to move under the action of the driving current.

9. The method according to claim 8, characterized in that The SOC is provided with an S-OIS control service and a QMI driving module, and the smart sensor hub is provided with an S-OIS manager, a frequency range extraction module, a PID parameter loading service, a PID parameter matching module and a PID parameter database, wherein the PID parameter database stores multiple groups of PID parameters, and the multiple groups of PID parameters include the target PID parameters; The method further comprises: The S-OIS control service forwards the received control request to the QMI driver module, where the control request is a request issued by the camera application when it is turned on; The QMI driver module forwards the control request to the S-OIS manager; The intelligent sensor hub performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: The S-OIS manager sends the received control request to the frequency range extraction module; The S-OIS manager receives the initial angular velocity data sent by the angular momentum sensor, and sends the initial angular velocity data to the frequency range extraction module in response to the control request; The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range; The intelligent sensor hub determines the target PID parameter matching the current frequency range according to the current frequency range, including: The frequency range extraction module sends the current frequency range to the PID parameter matching module; The PID parameter matching module determines, according to the current frequency range, a parameter identifier of the target PID parameter that matches the current frequency range from the preset mapping relationship; The PID parameter matching module calls the PID parameter loading service to obtain the target PID parameter from the PID parameter database according to the parameter identifier of the target PID parameter.

10. The method according to claim 7 or 9, characterized in that: The frequency range extraction module includes: a data filtering module, a time domain waveform conversion module, a spectrum analysis module, and a frequency range determination module; The frequency range extraction module performs time-frequency conversion on the initial angular velocity data to obtain the current frequency range, including: The data filtering module filters the initial angular velocity data to obtain filtered angular velocity data; The time domain waveform conversion module converts the filtered angular velocity data from the time domain to the frequency domain to obtain frequency domain data; The spectrum analysis module analyzes the frequency domain data to obtain a plurality of vibration frequencies and amplitudes of the plurality of vibration frequencies; The frequency range determination module determines the current frequency range according to the multiple vibration frequencies and the amplitudes of the multiple vibration frequencies.

11. An electronic device, characterized in that: include: Processors, memory, and interfaces; The processor, the memory and the interface cooperate with each other so that the electronic device executes the method as claimed in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 10.

Images