Lens control method, electronic device and storage medium

Abstract

The present application provides a lens control method, an electronic device, and a storage medium. The method includes: turning on the OIS function of the camera module and generating a first indication message; the SensorHub generates a second indication message in response to the first indication message, the second indication message indicating the target position of the lens of the camera module; in response to the second indication message, the lens is moved from the current position to the target position. The lens control method simplifies the composition of the camera module, can effectively save the hardware cost of the camera module, and reduce the occupied space.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a lens control method, an electronic device, and a storage medium. Background Art

[0002] With the widespread use of electronic devices, it is becoming more and more common to use electronic devices for shooting. When people use electronic devices to shoot, the images they take are easily blurred due to hand shaking, device movement, etc. In order to improve the clarity of the image and reduce the impact of motion blur, optical image stabilization (OIS) was introduced. OIS is a hardware-based anti-shake solution. The movable camera module installed in the electronic device can compensate for the displaced optical path, thereby reducing or eliminating the motion blur effect of the image. Summary of the invention

[0003] The embodiments of the present application provide a lens control method, an electronic device, and a storage medium. The lens control method can simplify the structure of a camera module, save the hardware cost of the camera module, and reduce the space occupied by the camera module.

[0004] In a first aspect, an embodiment of the present application provides a lens control method, which is applied to an electronic device, wherein the electronic device includes a smart sensor hub (SensorHub) and a camera module, and the method includes: turning on the OIS function of the camera module and generating first indication information, the SensorHub generates second indication information in response to the first indication information, and the second indication information indicates the target position of the lens of the camera module; in response to the second indication information, the lens is moved from the current position to the target position.

[0005] The lens control method described in the first aspect can use the computing power of the SensorHub side to determine the target position of the lens. The camera module does not need to have an embedded MCU unit, and the camera module has a simpler structure, which can effectively save the hardware cost of the camera module and reduce the space occupied by the camera module. In addition, the lens control method can provide a corresponding control path, so that the lens movement can be controlled based on the control path to achieve optical image stabilization.

[0006] In combination with the first aspect, in an optional implementation, the electronic device further includes an application processor (AP), the SensorHub includes an OIS control algorithm, and the SensorHub responds to the first indication information and generates the second indication information, including: the AP determines whether the camera module supports the SOIS mechanism based on the capability information of the camera module, and if the SOIS mechanism is supported, the SensorHub responds to the first indication information and runs the OIS control algorithm to generate the second indication information. The SOIS mechanism is an OIS mechanism in which the SensorHub runs the OIS control algorithm.

[0007] Optionally, the method further includes: if the DOIS mechanism is supported, the camera module responds to the first indication information and runs the OIS control algorithm to generate the second indication information. The DOIS mechanism is an OIS mechanism in which the camera module runs the OIS control algorithm. The camera module includes the OIS control algorithm.

[0008] In this implementation, when the AP determines that the camera module supports the SOIS mechanism, the AP runs the OIS control algorithm through the SensorHub; when the camera module supports the DOIS mechanism, the AP runs the OIS control algorithm through the camera module, which is compatible with the SOIS mechanism and the DOIS mechanism.

[0009] In combination with the first aspect, in an optional implementation, the method further includes: the SensorHub obtains the position information of the lens; the SensorHub sends the position information of the lens to the AP. The position information of the lens includes the current position of the lens and / or the target position of the lens. In this implementation, the lens control method can provide a corresponding data path so that the position information of the camera module can be uploaded to the AP based on the data path, which is beneficial for the AP to perform electronic image stabilization (EIS) services, automatic focus (AF) services, etc. based on the position information of the lens.

[0010] In combination with the first aspect, in an optional implementation, the SensorHub writes the position information of the lens into the first memory; the AP reads the position information of the lens from the first memory; the first memory is a shared memory of the AP and the SensorHub. In this implementation, the method of communicating between the AP and the SensorHub through the shared memory avoids the problems of increased data transmission delay or data loss caused by the QMI communication mechanism, and can effectively ensure the stability and real-time performance of data transmission.

[0011] In combination with the first aspect, in an optional implementation, the camera module includes N cameras, the AP includes N OIS software instances, the OIS software instances are used to abstractly describe the behavior implementation of the OIS service based on a structured data structure; the SensorHub includes N OIS hardware instances, the OIS hardware instances are used to abstractly describe the characteristics of the camera based on a structured data structure, and N is a positive integer; the N cameras include a first camera, the N OIS software instances include a first OIS software instance, the N OIS hardware instances include a first OIS hardware instance, and the first camera, the first OIS software instance and the first OIS hardware instance are interrelated.

[0012] In one implementation, the first OIS software instance generates first indication information, and the first indication information is used to indicate that the OIS function of the first camera has been turned on. In this implementation, for the first camera, a control path between the AP and the SensorHub can be implemented based on the first OIS software instance.

[0013] In one implementation, the first OIS hardware instance generates second indication information, where the second indication information is used to indicate a target position of a lens of the first camera. In this implementation, for the first camera, a control path between the SensorHub and the camera can be implemented based on the first OIS hardware instance.

[0014] In one implementation, the first OIS hardware instance sends the position information of the lens of the first camera to the SensorHub. In this implementation, for the first camera, a data path between the SensorHub and the camera can be implemented based on the first OIS hardware instance.

[0015] In one implementation, the first OIS software instance sends the location information of the lens of the first camera to the AP. In this implementation, for the first camera, a data path between the AP and the SensorHub can be implemented based on the first OIS software instance.

[0016] In a second aspect, an embodiment of the present application further provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in the first aspect.

[0017] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is run on a computer, it enables the computer to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments of the present application will be described below.

[0020] Figure 1 A schematic diagram showing the principle of optical image stabilization is shown;

[0021] Figure 2 The schematic diagram of the architecture of the OIS system under two mechanisms is shown;

[0022] Figure 3 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of an interactive process of a lens control method provided in an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of a shared memory interaction process provided by an embodiment of the present application;

[0026] Fig. 7A and Figure 7B This is a schematic diagram of a user interface provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] Some terms in this application are explained below to facilitate understanding by those skilled in the art.

[0028] The application (Application, App) involved in the embodiments of the present application, which can also be called an application program, is a software program that can realize one or more specific functions. Usually, multiple applications can be installed in an electronic device, for example, instant messaging applications, audio applications, image capture applications, and the like. Among them, instant messaging applications, for example, may include text messaging applications, WeChat (WeChat), DingTalk, and the like. Audio applications, for example, may include Kugou Music, Xiami Music, and the like. Image capture applications, for example, may include camera applications (factory setting system camera or third-party camera application). The applications mentioned in the following embodiments may be applications that are installed on the electronic device when it leaves the factory, or they may be applications that the user downloads from the Internet or obtains from other electronic devices during the use of the electronic device.

[0029] It should be understood that, in this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this application is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more, and "more than one" means two or more.

[0030] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0031] With the widespread use of electronic devices, it is becoming more and more common to use electronic devices for shooting. In the process of shooting with electronic devices, the generation of each frame of the image relies on the accumulation of photons during the exposure time, and converting them into electrons through photoelectric conversion, and further transforming them into images recognizable to the human eye. During this time, if the electronic device has a large amplitude of movement, the movement information will also be accumulated and recorded, and the generated image will be accompanied by strong motion blur.

[0032] In order to improve the clarity of the image and reduce the impact of motion blur, an OIS system based on OIS technology is introduced into electronic devices. The OIS system can detect the jitter data of the electronic device based on a motion sensor (such as a gyroscope, an accelerometer) during the photo exposure, and calculate the displacement that needs to be compensated based on the detected jitter data, and then move the lens or image sensor in the camera module according to the displacement to offset the displacement generated by the jitter, thereby obtaining a clear exposed image. Figure 1 FIG. 1 is a schematic diagram showing a principle of optical image stabilization. Figure 1 As shown in (a) in the figure, when the electronic device is not shaken, the ambient light reflected by the object can be transmitted to the lens through the optical path. The lens focuses the ambient light so that the light converges and forms an image on the image sensor. However, when the electronic device is shaken, the lens and image sensor will be displaced, and the image on the image sensor will shift during the exposure period, causing the image to become blurred. Figure 1 If the lens is moved downward according to the displacement, when the ambient light reflected by the object is transmitted to the lens, the lens can deflect the light path to the image sensor, so that the position of the image on the image sensor remains unchanged during the exposure period, so that the image taken is a clear image, that is, by compensating for the displacement generated by the jitter, a clear exposed image is obtained, as shown in Figure 1It should be noted that in other implementations, the image sensor may also be moved to compensate for the displacement, which will not be described in detail.

[0033] Exemplarily, the OIS system may include: a DOIS mechanism and a SOIS mechanism, wherein the DOIS mechanism refers to an OIS mechanism in which the camera module runs the OIS control algorithm, and the SOIS mechanism refers to an OIS mechanism in which the smart sensor hub (SensorHub) runs the OIS control algorithm.

[0034] Optionally, the software function modules of the OIS system may include: an OIS service algorithm, an OIS control algorithm and an OIS control driver.

[0035] 1. OIS Service Algorithm

[0036] The OIS service algorithm is used to manage the OIS service functions of the camera module, including but not limited to: turning on or off the OIS function of the camera module, adjusting the OIS mode of the OIS function, etc.

[0037] (1) Enable or disable the camera module OIS function based on the OIS service algorithm.

[0038] Among them, turning on the OIS function of the camera module means using the OIS system to reduce or eliminate the motion blur effect of the image when the camera application uses the camera module to take pictures.

[0039] For example, the shooting scenes of the electronic device include night scene shooting scenes and daytime shooting scenes, and the OIS service algorithm includes: when the electronic device is in a night scene shooting scene, turning on the OIS function of the camera module; when the electronic device is in a daytime shooting scene, turning off the OIS function of the camera module.

[0040] For another example, the shooting scenes of the electronic device include video recording scenes and photo taking scenes, and the OIS service algorithm includes: when the electronic device is in the video recording scene, turning on the OIS function of the camera module; when the electronic device is in the photo taking scene, turning off the OIS function of the camera module.

[0041] Furthermore, the camera module may refer to N cameras, where N is a positive integer, such as the camera module includes at least one of a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera. The OIS function of some or all cameras may be turned on or off based on the OIS service algorithm. For example, the OIS service algorithm may include: when shooting a night scene in an open grassland, turning on the OIS function of the camera module means turning on the OIS function of the wide-angle camera. For another example, when shooting a night scene in a narrow room, turning on the OIS function of the camera module means turning on the OIS function of the wide-angle camera and the ultra-wide-angle camera at the same time. For the sake of ease of description, the following text will take the camera module as an example of a camera.

[0042] (2) An OIS mode that adjusts the OIS function based on an OIS service algorithm.

[0043] Among them, different OIS modes are used to implement OIS functions for different business requirements. For example, the first OIS mode is used to implement an OIS function focusing on fluency, and the second OIS mode is used to implement an OIS function focusing on real-time performance.

[0044] Optionally, the movement of the lens in different OIS modes is different. Taking the first OIS mode and the second OIS mode as an example, the lens in the first OIS mode can move slowly in small steps, which is conducive to improving the smoothness of the OIS function, and the lens in the second OIS mode can move quickly, which is conducive to ensuring the real-time performance of the OIS function.

[0045] For ease of explanation, the following text uses the OIS service algorithm to exemplify how to turn on or off the OIS function of the camera module.

[0046] 2. OIS Control Algorithm

[0047] The OIS control algorithm is used to determine the target position of the lens in the camera module. Specifically, when the jitter data and the current position of the lens are input into the OIS control algorithm, the OIS control algorithm can be run to obtain the target position of the lens.

[0048] 3. OIS control drive

[0049] The OIS control driver is used to output control information, and the control information is used to control the OIS motor to drive the lens to move.

[0050] Optionally, for the DOIS mechanism and the SOIS mechanism, the hardware structure of the OIS system is different, and the deployment method of the software function module is also different. Figure 2 The architectural diagram of the OIS system under the DOIS mechanism and SOIS mechanism is elaborated in detail.

[0051] like Figure 2As shown in (a), the OIS system under the DOIS mechanism may include an application processor (AP) 201 and a camera module 202, and the AP 201 is connected to the camera module 202; and the OIS service algorithm is deployed in the AP 201, and the OIS control algorithm and the OIS control driver are deployed in the camera module 202 (for example, the camera module 202 includes an OIS driver chip 2021, an OIS motor 2022 and a lens 2023, and the OIS control algorithm and the OIS control driver may be deployed in the OIS driver chip 2021). When the OIS function is enabled by the OIS service algorithm in the AP 201, the target position of the lens may be determined by the OIS control algorithm in the OIS driver chip 2021, and control information may be generated by the OIS control driver in the OIS driver chip 2021, and the control information may be used to control the OIS motor 222 to push the lens 223 to move to the target position.

[0052] like Figure 2 As shown in (b), the OIS system under the SOIS mechanism may include AP211, SensorHub212 and camera module 213, AP211 is connected to SensorHub212, and SensorHub212 is connected to camera module 213; and, the OIS service algorithm is deployed in AP211, the OIS control algorithm is deployed in SensorHub212, and the OIS control driver is deployed in camera module 213 (for example, camera module 213 includes OIS driver chip 2131, OIS motor 2132 and lens 2133, and the OIS control driver may be deployed in OIS driver chip 2131). When the OIS function is enabled through the OIS service algorithm in AP211, the target position of the lens may be determined through the OIS control algorithm in SensorHub212, and control information may be generated through the OIS control driver in the OIS driver chip 2131, and the control information may be used to control the OIS motor 2132 to push the lens 2133 to move to the target position.

[0053] It can be seen that by comparing the DOIS mechanism and the SOIS mechanism, the OIS system under the SOIS mechanism can use the SensorHub to run the OIS control algorithm to determine the target position of the lens, and can achieve optical image stabilization when the OIS driver chip of the camera module does not have an embedded microcontroller unit (MCU). The cost of the OIS driver chip without an embedded MCU unit is lower and occupies less space, so the hardware cost of the camera module is also lower and occupies less space. The camera module in the OIS system under the SOIS mechanism is simpler in structure, and the cost of the camera module is lower and occupies less space.

[0054] Exemplarily, the above-mentioned OIS system (including the OIS system under the DOIS mechanism and the OIS system under the SOIS mechanism) can be run in an electronic device. The electronic device provided in the embodiment of the present application can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) device, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0055] like Figure 3 As shown, it is a structural schematic diagram of an electronic device. Among them, the electronic device may include: a processor, an external memory interface, an internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, an antenna 1, an antenna 2, a mobile communication module, a wireless communication module, an audio module, a sensor module, a camera module, a motor, an indicator, a display screen, a button, and a subscriber identification module (SIM) card slot, etc. Among them, the audio module may include a speaker, a receiver, a microphone, an earphone interface, etc., the sensor module may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., and the camera module may include an OIS driver chip, an OIS motor, a lens, etc.

[0056] It is to be understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] The processor may include one or more processing units, for example, the processor may include an AP, a modem processor (also called a baseband processor), a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, a neural network processor (neural-network processing unit, NPU), etc. In some embodiments, the Sensor Hub involved in the present application is deployed in a Modem.

[0058] Among them, different processing units can be independent devices or integrated into one or more processors. The processor is the nerve center and command center of the terminal. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0059] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module, wireless communication module, and Modem, etc. In some embodiments, the antenna 1 of the electronic device is coupled with the mobile communication module, and the antenna 2 is coupled with the wireless communication module, so that the electronic device can communicate with the network side device and other electronic devices through wireless communication technology.

[0060] Among them, the gyroscope sensor can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor. In an embodiment of the present application, the jitter data may include the angular velocity of the three axes collected by the gyroscope sensor.

[0061] Among them, the acceleration sensor can detect the acceleration of the electronic device in various directions (i.e., x, y and z axes). When the electronic device is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device, and is applied to applications such as horizontal and vertical screen switching and pedometers. In an embodiment of the present application, the jitter data may include the acceleration in three directions collected by the acceleration sensor.

[0062] The magnetic sensor includes a Hall sensor. In the embodiment of the present application, the electronic device can use the Hall sensor to detect the position information of the lens. For example, the Hall sensor includes an X-axis Hall sensor and a Y-axis Hall sensor, the X-axis Hall sensor is used to detect the position of the lens in the X-axis direction, and the Y-axis Hall sensor is used to detect the position of the lens in the Y-axis direction.

[0063] Among them, the touch sensor is also called a "touch control device". The touch sensor can be set on the display screen, and the touch sensor and the display screen form a touch screen, also called a "touch control screen". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the AP to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen. In other embodiments, the touch sensor can also be set on the surface of the electronic device, which is different from the position of the display screen. In an embodiment of the present application, the electronic device can detect whether there is a user operation acting on the display screen of the electronic device through the touch sensor. After the touch sensor detects the user operation acting on the camera application, the electronic device can execute the lens control method.

[0064] Optionally, the above-mentioned electronic devices may be equipped with various mobile platforms, including but not limited to Qualcomm Snapdragon mobile platform (Qualcomm), HiSilicon Kirin mobile platform, or MediaTek Dimensity mobile platform.

[0065] The following is a detailed introduction to the interaction process of the lens control method.

[0066] Please refer to Figure 4 , Figure 4 A schematic diagram of an interaction process of a lens control method is exemplarily shown.

[0067] like Figure 4 As shown, the electronic device may include an AP, a SensorHub and a camera module, the AP includes an OIS business algorithm, the SensorHub includes an OIS control algorithm, the camera module includes an OIS control driver, and optionally, the camera module includes an OIS driver chip, an OIS motor and a lens, and the OIS control driver of the camera module may be specifically deployed in the OIS driver chip of the camera module.

[0068] S101. The AP turns on the OIS function of the camera module and generates first indication information, where the first indication information indicates that the OIS function has been turned on.

[0069] In one implementation, the AP turns on the OIS function of the camera module by default when the camera application is turned on, and generates the first indication information. In this case, turning on the OIS function of the camera module while turning on the camera application is simpler and more convenient.

[0070] In another implementation, the AP determines whether the current shooting scene needs to turn on the OIS function of the camera module based on the OIS business algorithm. If the current shooting scene needs to turn on the OIS function of the camera module, the OIS function of the camera module is turned on and a first indication message is generated; if the current shooting scene needs to turn off the OIS function of the camera module, step S101 is not executed. For example, if the current shooting scene is a night scene, it is determined based on the OIS business algorithm that the OIS function of the camera module needs to be turned on. If the current shooting scene is a daytime shooting scene, it is determined based on the OIS business algorithm that the OIS function of the camera module needs to be turned off. In this case, whether the OIS function of the camera module needs to be turned on is determined by the current shooting scene, and the OIS function is more closely matched with the current shooting scene.

[0071] Optionally, the AP can determine the current shooting scene based on the ambient light brightness detected by the ambient light sensor. For example, if the current ambient light brightness is greater than the brightness threshold, the current shooting scene is determined to be a daytime shooting scene; if the current ambient light brightness is less than or equal to the brightness threshold, the current shooting scene is determined to be a night scene shooting scene. It should be noted that the detection of the current shooting scene includes but is not limited to the above methods. In actual use, other methods can be used to obtain the current shooting scene according to actual needs without limitation.

[0072] S102. The AP sends first indication information to the Sensor Hub.

[0073] Correspondingly, the SensorHub receives the first indication information from the AP.

[0074] S103 . The SensorHub generates second indication information in response to the first indication information, where the second indication information indicates a target position of the lens of the camera module.

[0075] In one embodiment, the SensorHub responds to the first indication information and runs the OIS control algorithm to generate the second indication information, and the second indication information indicates the target position of the lens of the camera module. Specifically, the SensorHub responds to the first indication information, inputs the jitter data and the current position of the lens into the OIS control algorithm in the SensorHub, runs the OIS control algorithm to obtain the target position of the lens, and generates the second indication information.

[0076] Optionally, the jitter data may include angular velocities of three axes (ie, x, y and z axes) collected in real time by a gyroscope sensor, and / or accelerations of three axes (ie, x, y and z axes) collected in real time by an acceleration sensor.

[0077] In an optional embodiment, a sensor (such as a gyroscope sensor, an acceleration sensor, a Hall sensor, etc.) can collect data at intervals of a preset duration (such as 1 millisecond, 2 milliseconds, etc.). It can be seen that in this embodiment, the interval at which the sensor collects data is a fixed value, which is simpler and more convenient. In another optional embodiment, the time interval for the sensor to collect data can be adjusted in real time according to the sports scene in which the user is located. For example, if the current sports scene is a running scene, the frequency of the jitter is relatively high, and the time interval for the sensor to collect data can be adjusted to be small, such as 1 millisecond, 2 milliseconds, etc.; if the current sports scene is a walking scene, the frequency of the jitter is relatively low, and the time interval for the sensor to collect data can be adjusted to be large, such as 5 milliseconds, 10 milliseconds, etc. It can be seen that in this embodiment, the interval at which the sensor collects data can change with the change of the sports scene. In scenes with low jitter frequencies, there is no need to frequently collect jitter data, which can effectively save power.

[0078] Optionally, the gyroscope sensor is connected to the SensorHub via a serial peripheral interface (SPI), and the angular velocity of the three axes collected by the gyroscope sensor can be transmitted to the SensorHub via the SPI interface. Similarly, the acceleration sensor is also connected to the SensorHub via an SPI interface, and the acceleration of the three axes collected by the acceleration sensor can also be transmitted to the SensorHub via the corresponding SPI interface.

[0079] Optionally, the SensorHub may include a driver module and an OIS control algorithm module. The driver module is used to realize data communication between the SensorHub and other modules, such as data communication between the SensorHub and sensors (such as gyroscope sensors, acceleration sensors, Hall sensors, etc.), or data communication between the SensorHub and APs and camera modules; the OIS control algorithm module is used to run the OIS control algorithm.

[0080] S104. SensorHub sends second instruction information to the camera module.

[0081] Correspondingly, the camera module receives the second indication information from the SensorHub.

[0082] In an optional implementation, the Sensor Hub and the camera module may interact via an Inter-Integrated Circuit (I2C) interface. The Sensor Hub may send the second indication information to the camera module via the I2C interface.

[0083] S105: The camera module responds to the second instruction information and moves the lens from the current position to the target position.

[0084] Optionally, the camera module receives the second indication information from the SensorHub, including: the OIS driver chip of the camera module receives the second indication information from the SensorHub. The OIS driver chip can process the second indication information based on the OIS control drive, generate control information, and control the OIS motor to push the lens to the target position through the control signal. For example, the control information controls the X-axis OIS motor to push the lens in the x-axis direction, and controls the Y-axis OIS motor to push the lens in the y-axis direction, so that the lens is finally moved to the target position, thereby compensating for the displacement generated by the jitter.

[0085] In some optional embodiments, the camera module can also send the real-time position information of the lens (also referred to as the "current position") to the SensorHub. The SensorHub can determine the real-time second indication information based on the real-time position information and the real-time jitter data, so as to achieve real-time jitter compensation, so that the optical path can be kept stable during the entire shooting period.

[0086] It can be seen that in the OIS system, the OIS business algorithm can be deployed on the AP side, the OIS control algorithm can be deployed on the SensorHub side, and the OIS control driver can be deployed on the camera module side. The OIS control algorithm can be moved to the SensorHub side, and the computing power of the SensorHub side can be used to run the OIS control algorithm. Optical image stabilization can be achieved even when the MCU unit is removed, which can effectively save the cost of the camera module and reduce the space occupied by the camera module.

[0087] In addition, the lens control solution can provide a corresponding control path, so that the AP can control the lens movement of the camera module based on the control path, that is, when the AP turns on the OIS function of the camera module and generates the first indication information, the SensorHub can receive the first indication information from the AP, and run the OIS control algorithm in response to the first indication information to generate the second indication information, and the second indication information indicates the target position of the lens. When the camera module receives the second indication information from the SensorHub, it can respond to the second indication information and move the lens to achieve optical image stabilization.

[0088] In some embodiments, the camera module can also transmit data information to the upper layer through the data path. Taking the position information of the lens as an example, the camera module can send the position information of the lens to the Sensor Hub. After the Sensor Hub receives the position information of the lens, the Sensor Hub can upload the position information of the lens to the AP side, so that the AP side can perform related operations based on the position information of the lens, for example, perform electronic image stabilization (EIS) services based on the position information of the lens. Optionally, the position information of the lens may include the current position of the lens and / or the target position of the lens. It should be noted that during the movement, the current position of the lens refers to the position of the lens after it moves.

[0089] The following is a detailed description of steps S106 to S108.

[0090] S106. The camera module sends the position information of the lens to the Sensor Hub.

[0091] Correspondingly, the SensorHub obtains the position information of the lens, that is, the SensorHub receives the position information of the lens from the camera module.

[0092] In an optional implementation, similar to step S104, the camera module may send the lens position information to the Sensor Hub via the I2C interface.

[0093] S107. SensorHub sends the location information of the lens to the AP.

[0094] Correspondingly, the AP receives the location information of the lens from the SensorHub.

[0095] Exemplarily, if the AP also performs EIS service based on the position information of the lens, the method may further include step S108: the AP performs EIS service based on the EIS control algorithm. Specifically, the AP may use the position information of the lens and the jitter data through the EIS control algorithm to perform jitter compensation on the image sequence captured by the image sensor, thereby improving the stability of the image.

[0096] It can be seen that in this implementation, the camera module can report the location information of the lens to the AP via the data path via the SensorHub, which is beneficial for the AP to perform EIS services, AF services, etc. based on the location information of the lens.

[0097] The communication between AP and SensorHub is between multiple processors. The communication between AP and SensorHub is based on the CS architecture, that is, AP and SensorHub use the CS architecture communication method for communication. The CS architecture communication method refers to a communication method in which one party is the client and the other party is the server.

[0098] For example, taking the example of AP sending the first indication information to SensorHub, AP can be used as client and SensorHub as server. AP can first send a request message to SensorHub, and the request message is used to request to establish a communication connection. When SensorHub receives the request message, it can return a response message corresponding to the request message to AP, and the response message indicates the result of establishing the communication connection. When AP receives the response message indicating that the communication connection is successfully established, AP can send the first indication message to SensorHub.

[0099] In one embodiment, since the AP and the Sensor Hub interact with each other in a CS architecture communication mode, the information transmitted between the AP and the Sensor Hub (such as the aforementioned request information, response information, and first indication information) can be encapsulated as a Qualcomm Messaging Interface (QMI) message. Based on this, the QMI message can be encapsulated through the QMI interface.

[0100] For example, Figure 4 In the step S102, when the AP sends the first indication information to the Sensor Hub, the AP can encapsulate the first indication information into a QMI message using the QMI interface, and send the QMI message to the Sensor Hub. The Sensor Hub can decapsulate the QMI message to obtain the first indication information.

[0101] For example, Figure 4 In the step S107, when the Sensor Hub sends the lens location information to the AP, the Sensor Hub can use the QMI interface to encapsulate the lens location information into a QMI message, and send the QMI message to the AP, and the AP can decapsulate the QMI message to obtain the lens location information.

[0102] It can be seen that in this implementation, communication interaction between the AP and the SensorHub can be achieved based on QMI messages.

[0103] In another embodiment, considering that the essence of QMI messages is inter-core communication, they are greatly affected by inter-core scheduling, which may cause problems such as increased data transmission delay or data loss. For example, for Hall data such as position information, using the QMI communication mechanism to transmit the data is prone to increase data transmission delay or data loss.

[0104] To address this issue, the AP and SensorHub in this embodiment can exchange data in a shared memory manner. For example, the shared memory between the AP and the SensorHub is the first memory. The first memory refers to the memory that both the AP and the SensorHub can access. The first memory can be used as the memory of the AP to at least store the first indication information generated by the AP, and the first memory can also be used as the memory of the SensorHub to at least store the location information of the lens received by the SensorHub. Then, Figure 4 The step S102 may include: the AP writes the first indication information into the first memory, and the SensorHub reads the first indication information from the first memory. Figure 4 The step S107 may include: the SensorHub writes the position information of the lens into the first memory; and the AP reads the position information of the lens from the first memory.

[0105] In one implementation, shared memory can be implemented based on FastRPC. FastRPC is a remote procedure call (RPC) protocol based on XML-RPC. Like XML-RPC, it uses Hypertext Transfer Protocol (HTTP) as a transmission protocol. The FastRPC framework allows users to transparently make remote calls between APs and SensorHubs.

[0106] The following example uses the AP as the client and the SensorHub as the server, where the AP reads the lens location information stored in the SensorHub from the shared memory, to illustrate the method of implementing shared memory based on FastRPC. Figure 5 As shown, taking the AP and SensorHub of Qualcomm Snapdragon mobile platform as an example, AP may include a non-camera sensor (None Camera Sensor, NCS) module, a FastRPC Stub module, a FastRPC User module and a FastRPC Kernel driver module, and SensorHub may include a sensor instance (Sensor Implementation), a FastRPC Stel module, a FastRPC User module and a FastRPC Kernel driver module.

[0107] S201. The NCS module of the AP calls the service in a local calling mode, and passes the method, parameter type, and parameters to be called to the FastRPC Stub module of the AP.

[0108] S202, after receiving the call service, the FastRPC Stub module is responsible for assembling the method, parameter type, parameter and other data into a message body that can be transmitted over the network, serializing the message body object into binary data, and transmitting the message body to the FastRPC User module of the AP.

[0109] S203. The FastRPC User module of the AP sends the message body to the SensorHub through the FastRPC Kernel driver module, and the FastRPC User module of the SensorHub receives it.

[0110] S204. The FastRPC User module of SensorHub passes the received message body to the FastRPC Stel module.

[0111] S205. The FastRPC Stel module of SensorHub deserializes the received message body.

[0112] S206. The FastRPC Stel module of SensorHub parses the service call method, parameter type, parameter and other information according to the deserialization result, and calls the sensor instance of SensorHub.

[0113] S207, after the SensorHub executes the corresponding method, it returns the execution result data to the FastRPC Stel module of the SensorHub.

[0114] S208. The FastRPC Stel module of SensorHub serializes the result data and packages it into a transmittable message body.

[0115] S209, the FastRPC Stel module of SensorHub passes the message body to the FastRPCUser module of SensorHub.

[0116] S210 , the FastRPC User module of the SensorHub sends the message body to the AP through the FastRPC Kernel driver module, and the FastRPC User module of the AP receives it.

[0117] S211. The FastRPC User module of the AP sends the message body to the FastRPC Stub module of the AP.

[0118] S212. The FastRPC Stub module of the AP deserializes the message body.

[0119] The deserialized data is the result data, that is, the position information of the lens.

[0120] The above is an example of AP as the client and SensorHub as the server. For the relevant implementation examples with SensorHub as the client and AP as the server, please refer to Figure 5 The relevant steps will not be repeated here.

[0121] Need to explain, Figure 5 The embodiment described is described by taking the implementation of shared memory in Qualcomm Snapdragon mobile platform as an example. For other mobile platforms, please refer to Figure 5 The embodiments described above will not be described in detail.

[0122] Need to explain, Figure 5 The described embodiments are merely exemplary. In other ways, shared memory may also be implemented in other ways, which will not be elaborated herein.

[0123] It can be seen that in this implementation, the method of communicating between the AP and the SensorHub through shared memory avoids problems such as increased data transmission delay or data loss caused by the QMI communication mechanism, and can effectively ensure the stability and real-time performance of data transmission.

[0124] In addition, operating systems are running on the above components, such as the iOS operating system developed by Apple, the Android open source operating system developed by Google, and the Windows operating system developed by Microsoft.

[0125] The operating system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device. It should be noted that although the embodiment of the present application is illustrated by taking the Android system as an example, its basic principle is also applicable to electronic devices based on operating systems such as iOS or Windows.

[0126] Figure 6It is a schematic diagram of the software structure of an electronic device. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. Taking the Android system as an example, in some embodiments, the Android system is divided into six layers, from top to bottom, namely, the application layer, the application framework layer (Framework), the hardware abstraction layer (HAL), the kernel layer (Kernel), the Sensor Hub and the hardware layer (Hardware). Optionally, the application layer, the application framework layer (Framework), and the hardware abstraction layer (HAL) can be deployed on the AP side; optionally, the Sensor Hub can be deployed on the Modem side.

[0127] The application layer may include a series of application packages, including camera, gallery, calendar, call, map, WLAN, Bluetooth, music, video, short message and other APPs.

[0128] Among them, the application framework layer provides an application programming interface (Application Programming Interface, API) and a programming framework for the application of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, etc. The embodiment of the present application does not impose any restrictions on this. Exemplarily, the above-mentioned window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The above-mentioned content provider is used to store and obtain data, and make these data accessible to the application. The data may include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The above-mentioned view system can be used to build the display interface of the application. Each display interface can be composed of one or more controls. Generally speaking, controls can include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. The above-mentioned resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The above notification manager uses an application to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, a vibration is made, an indicator light flashes, etc. In some embodiments, the application framework layer may also include a camera service (Camera Service). The camera service (Camera Service) can interact with the hardware abstraction layer.

[0129] Among them, the hardware abstraction layer is an interface layer located between the kernel layer and the hardware, which can be used to abstract the hardware. In some embodiments, the hardware abstraction layer includes a hardware abstraction layer interface definition language (HIDL) interface. Among them, the hardware abstraction layer may include: camera HAL, display HAL, sensor HAL, etc. In some embodiments, the camera HAL includes an OIS control module, an adapter module, and a SensorHub control module. Among them, the OIS control module is used to enable the OIS function of the camera module based on the OIS business algorithm and generate a first indication information; the adapter module is used to determine whether the electronic device supports the SOIS mechanism; the SensorHub control module is used to send the control instructions (such as the first indication information) in the camera HAL layer to the SensorHub, and to receive and store data from the SensorHub.

[0130] The kernel layer is the foundation of the Android operating system, responsible for hardware drivers, network, power supply, system security, memory management and other functions. The kernel layer is an intermediate layer between hardware and software, and its function is to pass the application's request to the hardware. The kernel layer at least includes display drivers, audio drivers, sensor drivers, etc. In some embodiments, the kernel layer may also include a camera driver.

[0131] Among them, SensorHub is used to perform hardware abstraction for common sensor categories and realize centralized management of sensors. In some embodiments, SensorHub can be used to run the OIS control algorithm involved in the embodiments of the present application.

[0132] The hardware layer may include one or more hardware components. For example, the hardware layer may include a camera module, and the camera module may include an OIS driver chip, an OIS motor, a lens, and the like.

[0133] Combine the following Figure 6 The software structure diagram details the interaction between various modules involved when the electronic device executes the lens control method.

[0134] S301: A camera application receives a request to start the camera application.

[0135] It is understandable that the user can trigger the electronic device to start the camera application. Correspondingly, the camera application in the electronic device can receive the request information for starting the camera application.

[0136] For example, Fig. 7AAs shown, the user can click the camera application icon 701. Accordingly, the electronic device can receive a request message to start the camera application. In response to the request message to start the camera application, the electronic device can start the camera application and display the Figure 7B User interface 700 is shown. It is understandable that the user can also start the camera application by voice, gesture, etc., and the present application does not limit the specific method in which the user triggers the electronic device to start the camera application.

[0137] S302: The camera application sends a request message to the camera service to start the camera service.

[0138] After receiving the request information for starting the camera application, the camera application can send the request information for starting the camera service to trigger the camera service to start. Correspondingly, the camera service can receive the request information for starting the camera service sent by the camera application and start the camera service.

[0139] S303: The camera service sends a request message for starting the camera HAL to the camera HAL.

[0140] After receiving the request information for starting the camera service sent by the camera application, the camera service may send down a request information for starting the camera HAL to trigger the camera HAL to start.

[0141] S304: The camera HAL receives the request information for starting the camera HAL sent by the camera service, and starts the OIS control module.

[0142] It is understandable that the specific startup process (such as step S301 to step S304) can be referred to relevant technical documents and will not be described in detail here.

[0143] S305 . The OIS control module enables the OIS function of the camera module based on the OIS service algorithm and generates first indication information, where the first indication information indicates that the OIS function has been enabled.

[0144] Optionally, if the OIS control module determines whether the current shooting scene needs to turn on the OIS function of the camera module, then the OIS control module also needs to determine the current shooting scene. Based on this, the method also includes: the OIS sensor node module determines the scene information based on the ambient light brightness detected by the ambient light sensor, and the scene information indicates the current shooting scene. When the OIS sensor node module sends the scene information to the OIS control module, the OIS control module can determine the current shooting scene based on the scene information. The OIS sensor node module is not in Figure 6 Shown in.

[0145] S306: The OIS control module sends first indication information to the adaptation module.

[0146] Correspondingly, the adaptation module receives the first indication information from the OIS control module.

[0147] S307, the adaption module determines whether the camera module supports the SOIS mechanism. If it supports the SOIS mechanism, execute steps S308 to S312; if it supports the DOIS mechanism, execute steps S313 to S315.

[0148] In an optional implementation, the adapter module may determine whether the camera module supports the SOIS mechanism based on the capability information of the camera module. For example, if the value of the capability information is the first value, it is determined that the SOIS mechanism is supported; if the value of the capability information is the second value, it is determined that the DOIS mechanism is supported. Optionally, the capability information may be pre-configured in the camera module. When the camera module is turned on or the camera application is started, the camera module may report the capability information to the adapter module of the AP. For example, when the camera module is Figure 2 When the camera module shown in (a) in FIG. 1 is used, the capability information pre-configured in the camera module is a first value, and the camera module reports the first value to the adaptation module; when the camera module is Figure 2 When the camera module shown in (b) is used, the capability information preconfigured in the camera module is the second value, and the capability information reported by the camera module to the adaptation module is the second value. It should be noted that the adaptation module can also be replaced by other modules in the AP, such as the OIS control module, without limitation.

[0149] S308. The adaptation module sends first indication information to the SensorHub control module.

[0150] Correspondingly, the SensorHub control module receives the first indication information from the adaptation module.

[0151] Exemplarily, in a Qualcomm Snapdragon mobile platform, the SensorHub control module may be a None Camera Sensor Service Qualcomm Sensor Execution Environment (NCS QSEE) module for a non-camera sensor.

[0152] S309: The SensorHub control module sends first indication information to the SensorHub.

[0153] Correspondingly, the SensorHub receives the first indication information from the SensorHub control module.

[0154] For example, taking the NCS QSEE module of the Qualcomm Snapdragon mobile platform as the SensorHub control module, the NCSQSEE module and the SensorHub can communicate and interact through the NCS SSCConnection interface.

[0155] S310, the SensorHub generates second indication information in response to the first indication information, where the second indication information indicates a target position of the lens of the camera module.

[0156] S311, the Sensor Hub sends second indication information to the camera module. Correspondingly, the camera module receives the second indication information from the Sensor Hub.

[0157] S312: The camera module responds to the second instruction information and moves the lens from the current position to the target position.

[0158] For the relevant description of steps S308 to S312, please refer to the aforementioned embodiment and will not be repeated here.

[0159] S313: The adaptation module sends first instruction information to the camera driver.

[0160] Correspondingly, the camera driver receives the first indication information from the adaptation module.

[0161] Optionally, the adapter module and the camera driver may communicate and interact via a CSL interface.

[0162] S314: The camera driver sends first instruction information to the camera module.

[0163] Correspondingly, the camera module receives the first indication information from the camera driver.

[0164] S315: The camera module responds to the first instruction information, determines a target position of the lens, and moves the lens from a current position to the target position.

[0165] In an optional implementation, step S315 specifically includes: the OIS driver chip of the camera module responds to the first indication information, inputs the jitter data and the current position of the lens into the OIS control algorithm, runs the OIS control algorithm to obtain the target position of the lens, and generates control information through the OIS control drive, and controls the OIS motor to push the lens to the target position through the control signal.

[0166] It can be seen that the embodiment of the present application has added an adapter module. The adapter module can transmit the first indication information to the SensorHub control module when the camera module supports the SOIS mechanism, so that the SensorHub control module can control the SensorHub to run the OIS control algorithm to obtain the target position of the lens. At the same time, the adapter module can transmit the first indication information to the camera driver of the kernel layer when the camera module supports the DOIS mechanism, so that the camera driver can control the OIS driver chip in the camera module to run the OIS control algorithm to obtain the target position of the lens. It can be seen that the embodiment of the present application can be compatible with the DOIS mechanism and the SOIS mechanism.

[0167] It should be noted that the above embodiments describe the implementation of the control path of the OIS system under two mechanisms. Similarly, the specific implementation of the data path can be referred to Figure 6 The control path is not described in detail.

[0168] In some embodiments, when the camera module refers to N cameras, the SensorHub or the camera module can manage the information (such as the first indication information) and data information (such as the position information of the lens in the camera, etc.) of the N cameras through N OIS hardware instances, and the AP manages the OIS services of the N cameras through N OIS software instances. The OIS hardware instance is used to abstractly describe the characteristics of the camera based on a structured data structure (including but not limited to the OIS mode of the camera, the trajectory of the lens of the camera, etc.); the OIS software instance is used to abstractly describe the behavior implementation of the OIS service based on a structured data structure (including but not limited to turning on or off the OIS function of the camera). It should be noted that the OIS hardware instance here can also be understood as a hardware module in the camera module. When the camera module supports the SOIS mechanism, the OIS hardware instance can be scheduled by the SensorHub based on a unified device usage interface. When the camera module supports the DOIS mechanism, the OIS hardware instance can be scheduled by the AP based on a unified device usage interface. The OIS software instance can also be understood as an OIS thread in the AP. When the camera module supports the SOIS mechanism, the OIS software instance can be scheduled by the SensorHub control module on the AP side. When the camera module supports the DOIS mechanism, the OIS software instance can be scheduled by the camera driver on the AP side. It should be noted that the native SensorHub control module may not have an OIS software instance (for example, the native NCS QSEE module does not have an OIS software instance). The SensorHub control module involved in the embodiment of the present application can create a virtual OIS software instance based on the native SensorHub control module.

[0169] The camera, OIS hardware instance, and OIS software instance are interrelated. Exemplarily, the camera has a corresponding camera ID, the OIS hardware instance has a corresponding hardware instance ID (such as sensor uid, which can be abbreviated as suid), and the OIS software instance has a corresponding OIS software instance ID (such as OIS ID). Take the camera module including a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera, the wide-angle camera is identified as camera ID1, the ultra-wide-angle camera is identified as camera ID2, and the telephoto camera is identified as camera ID1, the three OIS hardware instances are identified as suid1, suid2, and suid3, and the three OIS software instances are identified as OIS ID1, OIS ID2, and OIS ID3, as an example, combined with Table 1, the association relationship between the camera, the OIS hardware instance, and the OIS software instance is exemplified. From the association relationship in Table 1, it can be seen that the wide-angle camera identified as camera ID1, the OIS software instance identified as OIS ID1, and the OIS hardware instance identified as suid1 are interrelated; the ultra-wide-angle camera identified as camera ID2, the OIS software instance identified as OIS ID2, and the OIS hardware instance identified as suid2 are interrelated; the telephoto camera identified as camera ID3, the OIS software instance identified as OIS ID3, and the OIS hardware instance identified as suid3 are interrelated.

[0170] Table 1

[0171] Camera logo OIS hardware instance identifier OIS software instance identifier camera ID1 suid1 OIS ID1 camera ID2 suid2 OIS ID2 camera ID3 suid3 OIS ID3

[0172] Optionally, the association relationship between the OIS hardware instance and the camera is preconfigured and can be obtained by the electronic device when it is powered on. Optionally, the association relationship between the OIS software instance and the camera can be established when the camera application is turned on through the non-camera sensor service (NCS Service). Then, the electronic device can associate the camera, the OIS hardware instance, and the OIS software instance with each other based on the association relationship between the OIS hardware instance and the camera, and the association relationship between the OIS software instance and the camera. For example, N cameras include a first camera, N OIS software instances include a first OIS software instance, and N OIS hardware instances include a first OIS hardware instance, and the first camera, the first OIS software instance, and the first OIS hardware instance are associated with each other. The first camera can be a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, etc.

[0173] For example, Figure 4 The step S101 may specifically include: the AP turns on the OIS function of each camera, determines the OIS software instance associated with each camera based on the association relationship between the OIS software instance identifier and the camera identifier, and generates first indication information for each camera through the associated OIS software instance. That is, for the first camera, the first OIS software instance generates the first indication information, and the first indication information is used to indicate that the OIS function of the first camera has been turned on.

[0174] For example, Figure 4 The step S102 may specifically include: the AP determines the OIS software instance associated with the camera based on the association relationship between the camera identifier and the OIS software instance identifier, and sends the first indication information of each camera through the OIS software instance associated with each camera. That is, for the first camera, the first OIS software instance sends the first indication information.

[0175] For example, Figure 4 The step S103 may specifically include: the SensorHub may respond to the first indication information of each camera, determine the OIS hardware instance associated with each OIS software instance based on the association relationship between the OIS hardware instance identifier and the OIS software instance identifier, and generate the second indication information of each camera by running the OIS control algorithm through the associated OIS hardware instance. That is, the first OIS hardware instance generates the second indication information, and the second indication information is used to indicate the target position of the lens of the first camera.

[0176] For example, Figure 4 The step S104 may specifically include: the SensorHub determines the OIS hardware instance associated with the camera based on the association relationship between the camera identifier and the OIS hardware instance identifier, and sends the second indication information of the camera to the camera module through the OIS hardware instance associated with the camera. That is, for the first camera, the first OIS hardware instance sends the second indication information.

[0177] For example, Figure 4 The step S105 may specifically include: the camera module may receive the second indication information of each camera, and at this time, the camera module may move each camera to a corresponding target position based on the second indication information of each camera. In other words, the first camera may receive the target position of the lens of the first camera from the first OIS hardware instance.

[0178] For example, Figure 4The step S106 may specifically include: the camera module determines the OIS hardware instance associated with each camera based on the association relationship between the OIS hardware instance identifier and the camera identifier, and sends the location information of each camera to the SensorHub through the OIS hardware instance associated with each camera. That is, for the first camera, the first OIS hardware instance may send the location information of the lens of the first camera to the SensorHub.

[0179] For example, Figure 4 The step S107 may specifically include: the SensorHub determines the OIS software instance associated with each camera based on the association relationship between the OIS software instance identifier and the camera identifier, and sends the location information of each camera to the AP through the OIS software instance associated with each camera. That is, for the first camera, the first OIS software instance may send the location information of the lens of the first camera to the AP.

[0180] It can be seen that in the embodiment of the present application, an association relationship can be established among the camera, the OIS hardware instance and N OIS software instances, and the OIS hardware instance and OIS software instance of each camera can be bound and distinguished to realize the control path and data path of the corresponding camera.

Claims

1. A lens control method, characterized in that: The method is applied to an electronic device, the electronic device includes an application processor AP, a smart sensor hub SensorHub and a camera module, and the method includes: The AP turns on the optical image stabilization (OIS) function of the camera module and generates first indication information, where the first indication information indicates that the OIS function has been turned on; When the AP determines that the camera module supports the running of the OIS control algorithm by the SensorHub, the AP sends the first indication information to the SensorHub. The SensorHub responds to the first indication information, runs the OIS control algorithm to generate second indication information, and sends the second indication information to the camera module, where the second indication information indicates the target position of the lens of the camera module. The camera module receives and responds to the second indication information, and moves the lens from the current position to the target position. When the AP determines that the camera module supports the operation of the OIS control algorithm by the camera module, it sends the first indication information to the camera module. The camera module responds to the first indication information, operates the OIS control algorithm to determine the target position of the lens, and moves the lens from the current position to the target position.

2. The method according to claim 1, characterized in that The method further comprises: The SensorHub obtains the position information of the lens; The SensorHub sends the location information of the lens to the AP; The position information of the lens includes the current position of the lens and / or the target position of the lens.

3. The method according to claim 2, characterized in that The SensorHub writes the position information of the lens into the first memory; The AP reads the position information of the lens from the first memory; The first memory is a shared memory of the AP and the SensorHub.

4. The method according to any one of claims 1 to 3, characterized in that The camera module includes N cameras, the AP includes N OIS software instances, and the OIS software instances are used to abstractly describe the behavior implementation of the OIS service based on a structured data structure; The SensorHub includes N OIS hardware instances, and the OIS hardware instances are used to abstractly describe the features of the camera based on a structured data structure, where N is a positive integer; The N cameras include a first camera, the N OIS software instances include a first OIS software instance, the N OIS hardware instances include a first OIS hardware instance, and the first camera, the first OIS software instance, and the first OIS hardware instance are associated with each other; The first OIS software instance generates the first indication information, where the first indication information is used to indicate that the OIS function of the first camera has been turned on; The first OIS hardware instance generates the second indication information, where the second indication information is used to indicate a target position of a lens of the first camera.

5. The method according to claim 4, characterized in that The first OIS hardware instance sends the position information of the lens of the first camera to the SensorHub.

6. The method according to claim 4, characterized in that The first OIS software instance sends the location information of the lens of the first camera to the AP.

7. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 6.

Images