Lens control method, electronic equipment and storage medium

Abstract

The invention provides a lens control method, electronic equipment and a storage medium. The method comprises the steps that an AP starts OIS functions of a plurality of target cameras in N cameras, first indication information corresponding to all the target cameras is generated through OIS software examples associated with all the target cameras, SensorHub responds to the first indication information corresponding to all the target cameras, the OIS hardware examples associated with all the target cameras are used for executing OIS software examples associated with all the target cameras, and the OIS software examples associated with all the target cameras are used for executing OIS software examples associated with all the target cameras. And generating second indication information corresponding to each target camera, and moving the lens from the current position to the target position by each target camera based on the corresponding second indication information. Therefore, based on the OIS hardware instances and the OIS software instances associated with the plurality of target cameras, the control access of the plurality of cameras can be realized, so that the lenses of the plurality of target cameras are moved to the corresponding target positions.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a lens control method, an electronic device, and a storage medium. Background Art

[0002] With the wide use of electronic devices, it has become increasingly common to use electronic devices for shooting. When people use electronic devices for shooting, due to their own hand tremors, device movement, etc., it is easy to cause the captured images to be blurred. To improve the clarity of the images and reduce the impact of motion blur, Optical Image Stabilization (OIS) has been introduced. OIS is a hardware-based anti-shake solution. The movable camera module installed in the electronic device can compensate for the displaced optical path, thereby reducing or eliminating the motion blur effect of the images. Summary of the Invention

[0003] The embodiments of this application provide a lens control method, an electronic device, and a storage medium. The lens control method can simplify the composition of the camera module, save the hardware cost of the camera module, and reduce the occupied space of the camera module.

[0004] In a first aspect, the embodiments of this application provide a lens control method. The method is applied to an electronic device, which includes a Smart Sensing Hub (SensorHub) and a camera module. The method includes: enabling the OIS function of the camera module and generating a first indication message. SensorHub responds to the first indication message to generate a second indication message, and the second indication message indicates the target position of the lens of the camera module; in response to the second indication message, moving the lens from the current position to the target position.

[0005] The lens control method described in the first aspect can use the computing power on the SensorHub side to determine the target position of the lens. The camera module does not need to be embedded with an MCU unit, the composition of the camera module is simpler, the hardware cost of the camera module can be effectively saved, and the occupied space of the camera module can be reduced. Moreover, the lens control method can provide a corresponding control path, so that the lens can be controlled to move based on this control path to achieve optical anti-shake.

[0006] In combination with the first aspect, in an alternative embodiment, the electronic device further includes an Application Processor (AP). The SensorHub includes an OIS control algorithm. The SensorHub responds to the first indication information and generates second indication information, including: the AP determines whether the camera module supports the SOIS mechanism based on the capability information of the camera module. If the camera module supports the SOIS mechanism, the SensorHub responds to the first indication information and runs the OIS control algorithm to generate the second indication information. Herein, the SOIS mechanism is the OIS mechanism in which the SensorHub runs the OIS control algorithm.

[0007] Optionally, the method further includes: if the camera module supports the DOIS mechanism, the camera module responds to the first indication information and runs the OIS control algorithm to generate the second indication information. Herein, the DOIS mechanism is the OIS mechanism in which the camera module runs the OIS control algorithm. The camera module includes an OIS control algorithm.

[0008] In this embodiment, when the AP determines that the camera module supports the SOIS mechanism, it runs the OIS control algorithm through the SensorHub; when the camera module supports the DOIS mechanism, it runs the OIS control algorithm through the camera module, which can be compatible with the SOIS mechanism and the DOIS mechanism.

[0009] In combination with the first aspect, in an alternative embodiment, the method further includes: the SensorHub obtains the position information of the lens; the SensorHub sends the position information of the lens to the AP. Herein, the position information of the lens includes the current position of the lens and / or the target position of the lens. In this embodiment, the lens control method can provide a corresponding data path, enabling the position information of the camera module to be uploaded to the AP based on this data path, which is beneficial for the AP to perform operations such as Electronic Image Stabilization (EIS) and Automatic Focus (AF) based on the position information of the lens.

[0010] In combination with the first aspect, in an alternative embodiment, the SensorHub writes the position information of the lens into the first memory; the AP reads the position information of the lens from the first memory; the first memory is a shared memory between the AP and the SensorHub. In this embodiment, the method of communication between the AP and the SensorHub through the shared memory avoids problems such as increased data transmission delay or data loss caused by the QMI communication mechanism, and can effectively ensure the stability and real-time nature of data transmission.

[0011] In combination with the first aspect, in an alternative embodiment, the camera module includes N cameras, the AP includes N OIS software instances, and the OIS software instances are used to abstract and describe the behavior implementation of the OIS service based on a structured data structure; the SensorHub includes N OIS hardware instances, and the OIS hardware instances are used to abstract and describe the characteristics of the camera based on a structured data structure, where N is a positive integer; the N cameras include a first camera, the N OIS software instances include a first OIS software instance, and the N OIS hardware instances include a first OIS hardware instance, and the first camera, the first OIS software instance, and the first OIS hardware instance are interrelated.

[0012] In one implementation, the first OIS software instance generates first indication information, and the first indication information is used to indicate that the first camera has enabled the OIS function. In this implementation, for the first camera, a control path between the AP and the SensorHub can be implemented based on the first OIS software instance.

[0013] In one implementation, the first OIS hardware instance generates second indication information, and the second indication information is used to indicate the target position of the lens of the first camera. In this implementation, for the first camera, a control path between the SensorHub and the camera can be implemented based on the first OIS hardware instance.

[0014] In one implementation, the first OIS hardware instance sends the position information of the lens of the first camera to the SensorHub. In this implementation, for the first camera, a data path between the SensorHub and the camera can be implemented based on the first OIS hardware instance.

[0015] In one implementation, the first OIS software instance sends the position information of the lens of the first camera to the AP. In this implementation, for the first camera, a data path between the AP and the SensorHub can be implemented based on the first OIS software instance.

[0016] In a second aspect, an embodiment of the present application further provides an electronic device, which includes a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in the first aspect.

[0017] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, the computer is caused to execute the method described in the first aspect.

[0018] Fourthly, an embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program runs on a computer, it causes the computer to execute the method described in the first aspect. Description of the Drawings

[0019] To describe the technical solutions in the embodiments of the present application more clearly, the drawings required to be used in the embodiments of the present application will be described below.

[0020] Figure 1 Shows a schematic diagram of an optical image stabilization principle;

[0021] Figure 2 Shows a schematic diagram of the architecture of an OIS system under two mechanisms;

[0022] Figure 3 Is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0023] Figure 4 Is a schematic diagram of the interaction process of a lens control method provided by an embodiment of the present application;

[0024] Figure 5 Is a schematic diagram of the software structure of an electronic device provided by an embodiment of the present application;

[0025] Figure 6 Is a schematic diagram of the interaction process of a shared memory provided by an embodiment of the present application;

[0026] Figure 7A and Figure 7B Is a schematic diagram of a user interface provided by an embodiment of the present application. Detailed Embodiments

[0027] Some terms in the present application will be explained below to facilitate understanding by those skilled in the art.

[0028] The application (Application, App) involved in the embodiments of the present application can also be called an application program, which is a software program that can implement one or more specific functions. Usually, multiple applications can be installed in an electronic device. For example, instant messaging applications, audio applications, image capture applications, and so on. Among them, instant messaging applications, for example, can include text message applications, WeChat, DingTalk, etc. Audio applications, for example, can include KuGou Music, Xiami Music, and so on. Image capture applications, for example, can include camera applications (factory-set system cameras or third-party camera applications). The applications mentioned in the following embodiments can be applications pre-installed in the electronic device when it leaves the factory, or applications downloaded from the network or obtained from other electronic devices by the user during the use of the electronic device.

[0029] It should be understood that unless otherwise specified in this application, " / " means "or". For example, A / B may represent A or B. "And / or" in this application is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. "At least one" means one or more, and "a plurality" means two or more.

[0030] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0031] With the widespread use of electronic devices, it has become increasingly common to use electronic devices for photographing. During the photographing process of an electronic device, the generation of each frame of image relies on accumulating photons within the exposure time, converting them into electrons through photoelectric conversion, and further transforming them into images recognizable by the human eye. During this time, if the electronic device has a large amplitude of movement, the movement information will also be accumulated and recorded, and the generated image will be accompanied by strong motion blur.

[0032] In order to improve the clarity of the image and reduce the influence of motion blur, an OIS system constructed based on OIS technology is introduced into the electronic device. The OIS system can detect the jitter data of the electronic device based on motion sensors (such as gyroscopes, acceleration sensors) during the photographing exposure, calculate the displacement amount that needs to be compensated according to the detected jitter data, and then move the lens or image sensor in the camera module according to this displacement amount to offset the displacement generated by the jitter, so as to obtain a clearly exposed image. Figure 1 shows a schematic diagram of an optical image stabilization principle. As Figure 1 shown in (a) of, when the electronic device has no jitter, the ambient light reflected by the object to be photographed can be transmitted to the lens through the optical path, and the lens focuses on the ambient light, so that the light converges and forms an image on the image sensor. However, when the electronic device jitters, the lens and the image sensor will generate displacement, and the imaging on the image sensor will shift during the photographing exposure, resulting in a blurred image, Figure 1 shown in (b) of. If the lens is moved downward according to the displacement amount, when the ambient light reflected by the object to be photographed is transmitted to the lens, the lens can deflect the optical path to the image sensor, so that the position of the imaging on the image sensor remains unchanged during the photographing exposure, so that the photographed image is a clear image, that is, by compensating and offsetting the displacement generated by the jitter, a clearly exposed image is obtained, as Figure 1As shown in (c) thereof. It should be noted that, in other embodiments, the image sensor can also be moved to compensate for displacement, which will not be elaborated herein.

[0033] Exemplarily, the OIS system may include: a DOIS mechanism and a SOIS mechanism. Among them, the DOIS mechanism refers to the OIS mechanism in which the camera module runs the OIS control algorithm, and the SOIS mechanism refers to the OIS mechanism in which the intelligent sensing hub (SensorHub) runs the OIS control algorithm.

[0034] Optionally, the software functional modules of the OIS system may include: an OIS service algorithm, an OIS control algorithm, and an OIS control driver.

[0035] I. OIS Service Algorithm

[0036] The OIS service algorithm is used to manage the OIS service functions of the camera module, including but not limited to: turning on or off the OIS function of the camera module, adjusting the OIS mode of the OIS function, etc.

[0037] (1) Turning on or off the OIS function of the camera module based on the OIS service algorithm.

[0038] Among them, turning on the OIS function of the camera module means using the OIS system to reduce or eliminate the motion blur effect of the image when the camera application uses the camera module for shooting.

[0039] For example, the shooting scenarios of the electronic device include a night shooting scenario and a day shooting scenario. The OIS service algorithm includes: when the electronic device is in the night shooting scenario, turning on the OIS function of the camera module; when the electronic device is in the day shooting scenario, turning off the OIS function of the camera module.

[0040] Another example, the shooting scenarios of the electronic device include a video recording scenario and a photo taking scenario. The OIS service algorithm includes: when the electronic device is in the video recording scenario, turning on the OIS function of the camera module; when the electronic device is in the photo taking scenario, turning off the OIS function of the camera module.

[0041] Furthermore, the camera module may refer to N cameras, where N is a positive integer. For example, the camera module includes at least one of a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera. Based on the OIS service algorithm, the OIS functions of some or all of the cameras can be turned on or off. For example, the OIS service algorithm may include: when shooting a night scene in an open grassland, turning on the OIS function of the camera module means turning on the OIS function of the wide-angle camera. Another example, when shooting a night scene in a narrow room, turning on the OIS function of the camera module means turning on the OIS functions of the wide-angle camera and the ultra-wide-angle camera at the same time. For the convenience of description, hereinafter, the camera module is taken as an example of a single camera for elaboration.

[0042] (2) The OIS mode that adjusts the OIS function based on the OIS service algorithm.

[0043] Among them, different OIS modes are used to implement the OIS functions for different service requirements. For example, the first OIS mode is used to implement the OIS function that focuses on smoothness, and the second OIS mode is used to implement the OIS function that focuses on real-time performance.

[0044] Optionally, the moving methods of the lenses in different OIS modes are different. Taking the first OIS mode and the second OIS mode as examples, the lens in the first OIS mode can move slowly in small steps, which is beneficial to improving the smoothness of the OIS function, and the lens in the second OIS mode can move quickly, which is beneficial to ensuring the real-time performance of the OIS function.

[0045] For the convenience of elaboration, hereinafter, the OIS service algorithm for turning on or off the OIS function of the camera module will be used for exemplary illustration.

[0046] II. OIS Control Algorithm

[0047] The OIS control algorithm is used to determine the target position of the lens in the camera module. Specifically, when the jitter data and the current position of the lens are input into the OIS control algorithm, the OIS control algorithm can be run to obtain the target position of the lens.

[0048] III. OIS Control Drive

[0049] The OIS control drive is used to output control information, and this control information is used to control the OIS motor to push the lens to move.

[0050] Optionally, for the DOIS mechanism and the SOIS mechanism, the hardware structures of the OIS system are different, and the deployment methods of the software function modules are also different. The following will elaborate on the schematic diagrams of the OIS system architectures under the DOIS mechanism and the SOIS mechanism in combination with Figure 2 Detail the schematic diagrams of the OIS system architectures under the DOIS mechanism and the SOIS mechanism.

[0051] Such as Figure 2As shown in (a) of [Figure / Illustration], the OIS system under the DOIS mechanism may include an Application Processor (AP) 201 and a camera module 202. The AP 201 is connected to the camera module 202. Moreover, the OIS service algorithm is deployed within the AP 201, and the OIS control algorithm and the OIS control driver are deployed within the camera module 202 (for example, the camera module 202 includes an OIS driver chip 2021, an OIS motor 2022, and a lens 2023, and the OIS control algorithm and the OIS control driver may be deployed within the OIS driver chip 2021). When the OIS function is enabled through the OIS service algorithm within the AP 201, the target position of the lens can be determined through the OIS control algorithm within the OIS driver chip 2021, and control information can be generated through the OIS control driver within the OIS driver chip 2021. This control information is used to control the OIS motor 2022 to push the lens 2023 to move to the target position.

[0052] As shown in Figure 2 (b) of [Figure / Illustration], the OIS system under the SOIS mechanism may include an AP 211, a SensorHub 212, and a camera module 213. The AP 211 is connected to the SensorHub 212, and the SensorHub 212 is connected to the camera module 213. Moreover, the OIS service algorithm is deployed within the AP 211, the OIS control algorithm is deployed within the SensorHub 212, and the OIS control driver is deployed within the camera module 213 (for example, the camera module 213 includes an OIS driver chip 2131, an OIS motor 2132, and a lens 2133, and the OIS control driver may be deployed within the OIS driver chip 2131). When the OIS function is enabled through the OIS service algorithm within the AP 211, the target position of the lens can be determined through the OIS control algorithm within the SensorHub 212, and control information can be generated through the OIS control driver within the OIS driver chip 2131. This control information is used to control the OIS motor 2132 to push the lens 2133 to move to the target position.

[0053] It can be seen that by comparing the DOIS mechanism and the SOIS mechanism, the OIS system under the SOIS mechanism can use the SensorHub to run the OIS control algorithm to determine the target position of the lens, and can achieve optical image stabilization when the OIS driver chip in the camera module does not embed a Microcontroller Unit (MCU) unit. The OIS driver chip without the embedded MCU unit has a lower cost and occupies less space. Thus, the hardware cost of the camera module is also lower and it occupies less space. The structure of the camera module in the OIS system under the SOIS mechanism is simpler, the cost of the camera module is lower, and it occupies less space.

[0054] Exemplarily, the above-mentioned OIS systems (including the OIS system under the DOIS mechanism and the OIS system under the SOIS mechanism) can operate in an electronic device. The electronic device provided by the embodiments of the present application can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, etc. The embodiments of the present application do not impose special restrictions on the specific form of the electronic device.

[0055] As Figure 3 shown, it is a schematic structural diagram of an electronic device. Among them, the electronic device may include: a processor, an external memory interface, an internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, antenna 1, antenna 2, a mobile communication module, a wireless communication module, an audio module, a sensor module, a camera module, a motor, an indicator, a display screen, a button, and a subscriber identification module (SIM) card slot, etc. Among them, the audio module may include a speaker, a receiver, a microphone, a headphone interface, etc., the sensor module may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., and the camera module may include an OIS driving chip, an OIS motor, a lens, etc.

[0056] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0057] Among them, the processor may include one or more processing units. For example, the processor may include an AP, a modem (which may also be referred to as a baseband processor), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. In some embodiments, the SensorHub involved in this application is deployed within the Modem.

[0058] Among them, different processing units may be independent devices or integrated in one or more processors. The processor is the nerve center and command center of the terminal. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0059] Among them, the wireless communication function of the electronic device can be implemented through Antenna 1, Antenna 2, a mobile communication module, a wireless communication module, and a Modem, etc. In some embodiments, Antenna 1 and the mobile communication module of the electronic device are coupled, and Antenna 2 and the wireless communication module are coupled, enabling the electronic device to communicate with network-side devices and other electronic devices through wireless communication technologies.

[0060] Among them, the gyroscope sensor can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., the x, y, and z axes) can be determined through the gyroscope sensor. In the embodiments of this application, the jitter data may include the angular velocities of the three axes collected by the gyroscope sensor.

[0061] Among them, the acceleration sensor can detect the acceleration of the electronic device in various directions (i.e., the x, y, and z axes). When the electronic device is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers. In the embodiments of this application, the jitter data may include the accelerations in the three directions collected by the acceleration sensor.

[0062] Among them, the magnetic sensor includes a Hall sensor. In the embodiments of this application, the electronic device can use the Hall sensor to detect the position information of the lens. For example, the Hall sensor includes a Hall sensor for the X-axis and a Hall sensor for the Y-axis. The Hall sensor for the X-axis is used to detect the position of the lens in the X-axis direction, and the Hall sensor for the Y-axis is used to detect the position of the lens in the Y-axis direction.

[0063] Among them, a touch sensor, also known as a "touch control device". The touch sensor can be disposed on the display screen, and a touch screen, also known as a "touch control screen", is formed by the touch sensor and the display screen. The touch sensor is used to detect a touch operation acting thereon or nearby. The touch sensor can transmit the detected touch operation to the AP to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen. In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device, at a different position from that of the display screen. In the embodiments of the present application, whether there is a user operation acting on the display screen of the electronic device can be completed by the touch sensor. After the touch sensor detects a user operation acting on the camera application, the electronic device can execute a lens control method.

[0064] Optionally, the above-mentioned electronic device can be equipped with various mobile platforms, including but not limited to Qualcomm Snapdragon mobile platforms, HiSilicon Kirin mobile platforms, or MediaTek Dimensity mobile platforms, etc.

[0065] Next, the interaction process of the lens control method will be introduced in detail.

[0066] Please refer to Figure 4 , Figure 4 which exemplarily shows a schematic diagram of the interaction process of a lens control method.

[0067] As Figure 4 shown, the electronic device may include an AP, a SensorHub, and a camera module. The AP includes an OIS service algorithm, the SensorHub includes an OIS control algorithm, and the camera module includes an OIS control driver. Optionally, the camera module includes an OIS driver chip, an OIS motor, and a lens. The OIS control driver of the camera module can be specifically deployed in the OIS driver chip of the camera module.

[0068] S101. The AP enables the OIS function of the camera module and generates a first indication message, where the first indication message indicates that the OIS function has been enabled.

[0069] In one implementation, the AP enables the OIS function of the camera module by default when the camera application is started and generates a first indication message. In this case, enabling the OIS function of the camera module while starting the camera application is simpler and more convenient.

[0070] In another implementation, the AP determines whether to enable the OIS function of the camera module based on the OIS service algorithm for the current shooting scenario. If the OIS function of the camera module needs to be enabled for the current shooting scenario, the AP enables the OIS function of the camera module and generates a first indication message; if the OIS function of the camera module needs to be disabled for the current shooting scenario, step S101 is not executed. For example, if the current shooting scenario is a night shooting scenario, it is determined based on the OIS service algorithm that the OIS function of the camera module needs to be enabled. If the current shooting scenario is a daytime shooting scenario, it is determined based on the OIS service algorithm that the OIS function of the camera module needs to be disabled. In this case, it is determined whether to enable the OIS function of the camera module based on the current shooting scenario, and the OIS function has a higher matching degree with the current shooting scenario.

[0071] Optionally, the AP can determine the current shooting scenario according to the ambient light brightness detected by the ambient light sensor. For example, if the current ambient light brightness is greater than the brightness threshold, it is determined that the current shooting scenario is a daytime shooting scenario; if the current ambient light brightness is less than or equal to the brightness threshold, it is determined that the current shooting scenario is a night shooting scenario. It should be noted that the detection of the current shooting scenario includes but is not limited to the above methods. In actual use, other methods can be adopted to obtain the current shooting scenario according to actual needs, which is not limited.

[0072] S102. The AP sends the first indication message to the SensorHub.

[0073] Correspondingly, the SensorHub receives the first indication message from the AP.

[0074] S103. The SensorHub responds to the first indication message and generates a second indication message, where the second indication message indicates the target position of the lens of the camera module.

[0075] In one embodiment, the SensorHub responds to the first indication message, runs the OIS control algorithm to generate a second indication message, and the second indication message indicates the target position of the lens of the camera module. Specifically, the SensorHub responds to the first indication message, inputs the jitter data and the current position of the lens into the OIS control algorithm in the SensorHub, runs the OIS control algorithm to obtain the target position of the lens, and generates a second indication message.

[0076] Optionally, the jitter data may include the angular velocity of three axes (i.e., x, y, and z axes) collected in real time by the gyroscope sensor and / or the acceleration of three axes (i.e., x, y, and z axes) collected in real time by the acceleration sensor.

[0077] In an alternative embodiment, sensors (such as gyro sensors, acceleration sensors, Hall sensors, etc.) can collect data at preset time intervals (such as 1 millisecond, 2 milliseconds, etc.). It can be seen that in this embodiment, the interval for the sensors to collect data is a fixed value, which is simpler and more convenient. In another alternative embodiment, the time interval for the sensors to collect data can be adjusted in real time according to the motion scenario where the user is located. For example, if the current motion scenario is a running scenario, the jitter frequency is relatively high, and the time interval for the sensors to collect data can be adjusted smaller, such as 1 millisecond, 2 milliseconds, etc.; if the current motion scenario is a walking scenario, the jitter frequency is relatively low, and the time interval for the sensors to collect data can be adjusted larger, such as 5 milliseconds, 10 milliseconds, etc. It can be seen that in this embodiment, the interval for the sensors to collect data can change with the motion scenario, and there is no need to frequently collect jitter data in scenarios with a low jitter frequency, which can effectively save power.

[0078] Optionally, the gyro sensor is connected to the SensorHub through a Serial Peripheral Interface (SPI). The angular velocities of the three axes collected by the gyro sensor can be transmitted to the SensorHub through this SPI interface. Similarly, the acceleration sensor is also connected to the SensorHub through an SPI interface, and the accelerations of the three axes collected by the acceleration sensor can also be transmitted to the SensorHub through the corresponding SPI interface.

[0079] Optionally, the SensorHub may include a driving module and an OIS control algorithm module. The driving module is used to implement data communication between the SensorHub and other modules, such as data communication between the SensorHub and sensors (such as gyro sensors, acceleration sensors, Hall sensors, etc.), or data communication between the SensorHub and the AP and the camera module; the OIS control algorithm module is used to run the OIS control algorithm.

[0080] S104. The SensorHub sends a second indication message to the camera module.

[0081] Correspondingly, the camera module receives the second indication message from the SensorHub.

[0082] In an alternative embodiment, the interaction between the SensorHub and the camera module can be through an Inter-Integrated Circuit (I2C) interface. The SensorHub can send the second indication message to the camera module through the I2C interface.

[0083] S105. The camera module responds to the second indication message and moves the lens from the current position to the target position.

[0084] Optionally, the camera module receives second indication information from the SensorHub, including: the OIS driver chip of the camera module receives the second indication information from the SensorHub. The OIS driver chip can process the second indication information based on OIS control to generate control information, and control the OIS motor to push the lens to the target position through the control signal. For example, the X-axis OIS motor is controlled to push the lens in the x-axis direction through the control information, and the Y-axis OIS motor is controlled to push the lens in the y-axis direction, so that the lens finally moves to the target position, thereby compensating for the displacement generated by jitter.

[0085] In some alternative embodiments, the camera module can also send the real-time position information of the lens (which can also be referred to as the "current position") to the SensorHub. The SensorHub can determine the real-time second indication information based on the real-time position information and the real-time jitter data, so as to achieve real-time jitter compensation and keep the optical path stable during the entire shooting period.

[0086] It can be seen that in the OIS system, the OIS service algorithm can be deployed on the AP side, the OIS control algorithm can be deployed on the SensorHub side, and the OIS control driver can be deployed on the camera module side. The OIS control algorithm can be moved up to the SensorHub side to utilize the computing power on the SensorHub side to run the OIS control algorithm. Optical image stabilization can be achieved even when the MCU unit is removed, which can effectively save the cost of the camera module and reduce the occupied space of the camera module.

[0087] Moreover, the lens control solution can provide a corresponding control path, enabling the AP to control the movement of the lens of the camera module based on the control path. That is, when the AP enables the OIS function of the camera module and generates the first indication information, the SensorHub can receive the first indication information from the AP and run the OIS control algorithm in response to the first indication information to generate the second indication information, which indicates the target position of the lens. When the camera module receives the second indication information from the SensorHub, it can respond to the second indication information and move the lens to achieve optical image stabilization.

[0088] In some embodiments, the camera module can also transmit data information to the upper layer through a data path. Taking the position information of the lens as an example, the camera module can send the position information of the lens to the SensorHub. After the SensorHub receives the position information of the lens, the SensorHub can upload the position information of the lens to the AP side so that the AP side can perform related operations based on the position information of the lens. For example, perform an electronic image stabilization (EIS) service based on the position information of the lens. Optionally, the position information of the lens may include the current position of the lens and / or the target position of the lens. It should be noted that during the movement, the current position of the lens refers to the position where the lens is located after moving.

[0089] The following will be elaborated in detail in combination with steps S106 to S108.

[0090] S106. The camera module sends the position information of the lens to the SensorHub.

[0091] Correspondingly, the SensorHub obtains the position information of the lens, that is, the SensorHub receives the position information of the lens from the camera module.

[0092] In an alternative implementation, similar to step S104, the camera module can send the position information of the lens to the SensorHub through the I2C interface.

[0093] S107. The SensorHub sends the position information of the lens to the AP.

[0094] Correspondingly, the AP receives the position information of the lens from the SensorHub.

[0095] Exemplarily, if the AP also performs an EIS service based on the position information of the lens, then the method may further include step S108: The AP performs an EIS service based on the EIS control algorithm. Specifically, the AP can use the EIS control algorithm to perform jitter compensation on the image sequence collected by the image sensor using the position information of the lens and the jitter data, which can improve the stability of the image.

[0096] It can be seen that in this implementation, the camera module can report the position information of the lens to the AP via the SensorHub through the data path, which is beneficial for the AP to perform EIS services, AF services, etc. based on the position information of the lens.

[0097] Among them, the communication between the AP and the SensorHub is the communication between multiple processors. The communication method between the AP and the SensorHub is a communication method based on the CS architecture, that is, the communication between the AP and the SensorHub uses the communication method of the CS architecture. Among them, the communication method of the CS architecture refers to a communication method in which one party is used as the client and the other party is used as the server.

[0098] Exemplarily, taking the AP sending the first indication information to the SensorHub as an example, the AP can be used as the client and the SensorHub can be used as the server. The AP can first send a request message to the SensorHub, and this request message is used to request the establishment of a communication connection. When the SensorHub receives this request message, it can return a response message corresponding to this request message to the AP, and this response message indicates the result of the establishment of the communication connection. When the AP receives the response message indicating the successful establishment of the communication connection, the AP can send the first indication information to the SensorHub.

[0099] In one embodiment, since the interaction between the AP and the SensorHub is carried out in a communication method based on the CS architecture, the information transmitted between the AP and the SensorHub (such as the aforementioned request message, response message, and first indication information) can be encapsulated into Qualcomm Messaging Interface (QMI) messages. Based on this, QMI messages can be encapsulated through the QMI interface.

[0100] Exemplarily, for Figure 4 the step S102, when the AP sends the first indication information to the SensorHub, the AP can use the QMI interface to encapsulate the first indication information into a QMI message and send this QMI message to the SensorHub, and the SensorHub can de-encapsulate and obtain the first indication information from this QMI message.

[0101] Exemplarily, for Figure 4 the step S107, when the SensorHub sends the position information of the lens to the AP, the SensorHub can use the QMI interface to encapsulate the position information of the lens into a QMI message and send this QMI message to the AP, and the AP can de-encapsulate and obtain the position information of the lens from this QMI message.

[0102] It can be seen that in this embodiment, the communication interaction between the AP and the SensorHub can be realized based on QMI messages.

[0103] In another embodiment, considering that the essence of QMI messages is inter-core communication and is greatly affected by inter-core scheduling, problems such as increased data transmission latency or data loss may occur. For example, for Hall data such as location information, using the QMI communication mechanism to transmit this data easily leads to problems such as increased data transmission latency or data loss.

[0104] To address this problem, in this embodiment, the data interaction between the AP and the SensorHub can be performed in the form of shared memory. Taking the shared memory between the AP and the SensorHub as the first memory as an example, the first memory refers to the memory that can be accessed by both the AP and the SensorHub. The first memory can be used as the memory of the AP and store at least the first indication information generated by the AP. Also, the first memory can be used as the memory of the SensorHub and store at least the position information of the lens received by the SensorHub. Then, Figure 4 The step S102 may include: the AP writes the first indication information into the first memory, and the SensorHub reads the first indication information from the first memory. Figure 4 The step S107 may include: the SensorHub writes the position information of the lens into the first memory; the AP reads the position information of the lens from the first memory.

[0105] In one implementation, the shared memory can be implemented based on FastRPC. FastRPC is a Remote Procedure Call (RPC) protocol based on XML-RPC. Like XML-RPC, it uses the Hypertext Transfer Protocol (HTTP) as the transport protocol. The FastRPC framework allows users to transparently make remote calls between the AP and the SensorHub.

[0106] Taking the AP as the client and the SensorHub as the server, and the AP reads the position information of the lens stored by the SensorHub from the shared memory as an example, the method for implementing the shared memory based on FastRPC is described exemplarily. As Figure 5 shown, taking the AP and the SensorHub of the Qualcomm Snapdragon mobile platform as an example, the AP may include a None Camera Sensor (NCS) module, a FastRPC Stub module, a FastRPC User module, and a FastRPC Kernel driver module. The SensorHub may include a Sensor Implementation, a FastRPC Stel module, a FastRPC User module, and a FastRPC Kernel driver module.

[0107] S201. The NCS module of the AP calls the service in a local call manner, and passes the method to be called, parameter types, and parameters to the FastRPC Stub module of the AP.

[0108] S202. After receiving the call service, the FastRPC Stub module is responsible for assembling data such as methods, parameter types, and parameters into a message body that can be transmitted over the network. At the same time, it serializes the message body object into binary data and transmits the message body to the FastRPC User module of the AP.

[0109] S203. The FastRPC User module of the AP sends the message body to the SensorHub through the FastRPC Kernel driver module, and it is received by the FastRPC User module of the SensorHub.

[0110] S204. The FastRPC User module of the SensorHub passes the received message body to the FastRPC Stel module.

[0111] S205. The FastRPC Stel module of the SensorHub deserializes the received message body.

[0112] S206. The FastRPC Stel module of the SensorHub parses information such as the method to be called, parameter types, and parameters from the deserialization result, and calls the sensor instance of the SensorHub.

[0113] S207. After the SensorHub executes the corresponding method, it returns the execution result data to the FastRPC Stel module of the SensorHub.

[0114] S208. The FastRPC Stel module of the SensorHub serializes the result data and packages it into a message body that can be transmitted.

[0115] S209. The FastRPC Stel module of the SensorHub passes the message body to the FastRPC User module of the SensorHub.

[0116] S210. The FastRPC User module of the SensorHub sends the message body to the AP through the FastRPC Kernel driver module, and it is received by the FastRPC User module of the AP.

[0117] S211. The FastRPC User module of the AP sends the message body to the FastRPC Stub module of the AP.

[0118] S212. The FastRPC Stub module of the AP deserializes the message body.

[0119] Among them, the deserialized result is the result data, that is, the position information of the lens.

[0120] The above is described by taking the AP as the client and the SensorHub as the server as an example. For the related embodiments with the SensorHub as the client and the AP as the server, please refer to Figure 5 the relevant steps and will not be elaborated here.

[0121] It should be noted that Figure 5 the described embodiments are described by taking the implementation of shared memory in the Qualcomm Snapdragon mobile platform as an example. Other mobile platforms can refer to Figure 5 the described embodiments and will not be elaborated here.

[0122] It should be noted that Figure 5 the described embodiments are only for illustrative purposes. In other ways, shared memory can also be implemented in other ways and will not be elaborated here.

[0123] It can be seen that in this embodiment, the method of communication between the AP and the SensorHub through shared memory avoids problems such as increased data transmission delay or data loss caused by the QMI communication mechanism, and can effectively ensure the stability and real-time performance of data transmission.

[0124] In addition, an operating system runs on the above components. For example, the iOS operating system developed by Apple Inc., the Android open-source operating system developed by Google Inc., the Windows operating system developed by Microsoft Corporation, etc.

[0125] The operating system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In this application embodiment, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device. It should be noted that although this application embodiment is described by taking the Android system as an example, its basic principle also applies to electronic devices based on operating systems such as iOS or Windows.

[0126] Figure 6It is a schematic diagram of the software structure of an electronic device. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. Taking the Android system as an example, in some embodiments, the Android system is divided into six layers, from top to bottom are the application layer, the application framework layer (Framework), the hardware abstraction layer (HAL), the kernel layer (Kernel), SensorHub, and the hardware layer (Hardware). Optionally, the application layer, the application framework layer (Framework), and the hardware abstraction layer (HAL) can be deployed on the AP side; optionally, SensorHub can be deployed on the Modem side.

[0127] Among them, the application layer may include a series of application packages. The application packages may include APPs such as camera, gallery, calendar, call, map, WLAN, Bluetooth, music, video, short message, etc.

[0128] Among them, the application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, etc. The embodiments of the present application do not impose any restrictions on this. Exemplarily, the above window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The above content provider is used to store and obtain data, and enable these data to be accessed by applications. The data may include videos, images, audio, dialed and answered calls, browsing history and bookmarks, phone books, etc. The above view system can be used to build the display interface of an application. Each display interface can be composed of one or more controls. Generally speaking, controls may include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. The above resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and so on. The above notification manager enables applications to display notification information in the status bar, can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify that the download is complete, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of a background-running application, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompt text information in the status bar, emit a prompt tone, vibrate, blink the indicator light, etc. In some embodiments, the application framework layer may further include a Camera Service. The Camera Service can interact with the hardware abstraction layer.

[0129] Among them, the hardware abstraction layer is an interface layer located between the kernel layer and the hardware, which can be used to abstract the hardware. In some embodiments, the hardware abstraction layer includes an interface of the hardware abstraction layer interface definition language (HIDL). Among them, the hardware abstraction layer may include: camera HAL, display HAL, sensor HAL, etc. In some embodiments, the camera HAL includes an OIS control module, an adaptation module, and a SensorHub control module. Among them, the OIS control module is used to turn on the OIS function of the camera module based on the OIS service algorithm and generate a first indication message; the adaptation module is used to determine whether the electronic device supports the SOIS mechanism; the SensorHub control module is used to send control instructions (such as the first indication message) within the camera HAL layer to the SensorHub, and is used to receive and store data from the SensorHub.

[0130] Among them, the kernel layer is the foundation of the Android operating system, which is responsible for functions such as hardware driver programs, networks, power supplies, system security, and memory management. The kernel layer is an intermediate layer between the hardware and the software, and its function is to pass the requests of application programs to the hardware. The kernel layer at least includes a display driver, an audio driver, a sensor driver, etc. In some embodiments, the kernel layer may further include a camera driver.

[0131] Among them, SensorHub is used to perform hardware abstraction on common sensor categories and realizes centralized management of sensors. In some embodiments, SensorHub can be used to run the OIS control algorithm involved in the embodiments of the present application.

[0132] Among them, the hardware layer may include one or more hardware components. For example, the hardware layer may include a camera module, and the camera module may include an OIS driver chip, an OIS motor, a lens, etc.

[0133] The following combines Figure 6 the software structure schematic diagram to elaborate in detail the interaction schematic diagram between the various modules involved when the electronic device executes the lens control method.

[0134] S301. The camera application receives a request message for starting the camera application.

[0135] It can be understood that the user can trigger the electronic device to start the camera application. Correspondingly, the camera application in the electronic device can receive the request message for starting the camera application.

[0136] Exemplarily, such as Figure 7AAs shown, the user can click on the camera application icon 701. Correspondingly, the electronic device can receive a request message to start the camera application. In response to the request message to start the camera application, the electronic device can start the camera application and display Figure 7B the user interface 700 shown. It can be understood that the user can also start the camera application by means of voice, gesture, etc. This application does not limit the specific way for the user to trigger the electronic device to start the camera application.

[0137] S302. The camera application sends a request message to start the camera service to the camera service.

[0138] After receiving the request message to start the camera application, the camera application can send a request message to start the camera service to trigger the start of the camera service. Correspondingly, the camera service can receive the request message to start the camera service sent by the camera application and start the camera service.

[0139] S303. The camera service sends a request message to start the camera HAL to the camera HAL.

[0140] After receiving the request message to start the camera service sent by the camera application, the camera service can send a request message to start the camera HAL to trigger the start of the camera HAL.

[0141] S304. The camera HAL receives the request message to start the camera HAL sent by the camera service and starts the OIS control module.

[0142] It can be understood that the specific startup process (such as steps S301 to S304) can refer to relevant technical documents and will not be elaborated here.

[0143] S305. The OIS control module enables the OIS function of the camera module based on the OIS service algorithm and generates a first indication message, which indicates that the OIS function has been enabled.

[0144] Optionally, if the OIS control module is to determine whether the OIS function of the camera module needs to be enabled for the current shooting scene, then the OIS control module also needs to determine the current shooting scene. Based on this, the method further includes: the OIS sensor node module determines the scene information based on the ambient light brightness detected by the ambient light sensor, and the scene information indicates the current shooting scene. When the OIS sensor node module sends the scene information to the OIS control module, the OIS control module can determine the current shooting scene based on the scene information. The OIS sensor node module is not shown in Figure 6 it.

[0145] S306. The OIS control module sends the first indication message to the adaptation module.

[0146] Correspondingly, the adaptation module receives the first indication information from the OIS control module.

[0147] S307. The adaptation module determines whether the camera module supports the SOIS mechanism. If it supports the SOIS mechanism, steps S308 to S312 are executed; if it supports the DOIS mechanism, steps S313 to S315 are executed.

[0148] In an optional implementation, the adaptation module can determine whether the camera module supports the SOIS mechanism based on the capability information of the camera module. For example, if the value of the capability information is the first value, it is determined that the SOIS mechanism is supported; if the value of the capability information is the second value, it is determined that the DOIS mechanism is supported. Optionally, the capability information can be pre-configured in the camera module. When the device is powered on or the camera application is launched, the camera module can report the capability information to the adaptation module of the AP. For example, when the camera module is the Figure 2 camera module shown in (a) in, the capability information pre-configured in the camera module is the first value, and the camera module reports the first value to the adaptation module; when the camera module is the Figure 2 camera module shown in (b) in, the capability information pre-configured in the camera module is the second value, and the capability information reported by the camera module to the adaptation module is the second value. It should be noted that the adaptation module can also be replaced by other modules in the AP, such as the OIS control module, without limitation.

[0149] S308. The adaptation module sends the first indication information to the SensorHub control module.

[0150] Correspondingly, the SensorHub control module receives the first indication information from the adaptation module.

[0151] Exemplarily, in the Qualcomm Snapdragon mobile platform, the SensorHub control module can be the None Camera Sensor Service Qualcomm Sensor Execution Environment (NCS QSEE) module for non-camera sensors.

[0152] S309. The SensorHub control module sends the first indication information to the SensorHub.

[0153] Correspondingly, the SensorHub receives the first indication information from the SensorHub control module.

[0154] Exemplarily, taking the NCS QSEE module of the Qualcomm Snapdragon mobile platform as the SensorHub control module as an example, communication and interaction can be achieved between the NCS QSEE module and the SensorHub through the NCS SSCConnection interface.

[0155] S310. The SensorHub responds to the first indication information and generates second indication information, and the second indication information indicates the target position of the lens of the camera module.

[0156] S311. The SensorHub sends the second indication information to the camera module. Correspondingly, the camera module receives the second indication information from the SensorHub.

[0157] S312. The camera module responds to the second indication information and moves the lens from the current position to the target position.

[0158] For the related elaboration on steps S308 to S312, please refer to the foregoing embodiments and will not be elaborated herein.

[0159] S313. The adaptation module sends the first indication information to the camera driver.

[0160] Correspondingly, the camera driver receives the first indication information from the adaptation module.

[0161] Optionally, communication and interaction between the adaptation module and the camera driver can be achieved through the CSL interface.

[0162] S314. The camera driver sends the first indication information to the camera module.

[0163] Correspondingly, the camera module receives the first indication information from the camera driver.

[0164] S315. The camera module responds to the first indication information, determines the target position of the lens, and moves the lens from the current position to the target position.

[0165] In an optional implementation manner, step S315 specifically includes: the OIS driver chip of the camera module responds to the first indication information, inputs the jitter data and the current position of the lens into the OIS control algorithm, runs the OIS control algorithm to obtain the target position of the lens, and generates control information through the OIS control drive, and controls the OIS motor to push the lens to the target position through this control signal.

[0166] As can be seen, an adaptation module is newly added in the embodiments of the present application. When the camera module supports the SOIS mechanism, the adaptation module can transmit the first indication information to the SensorHub control module, so that the SensorHub control module can control the SensorHub to run the OIS control algorithm to obtain the target position of the lens. At the same time, when the camera module supports the DOIS mechanism, the adaptation module can transmit the first indication information to the camera driver in the kernel layer, so that the camera driver can control the OIS driver chip in the camera module to run the OIS control algorithm to obtain the target position of the lens. As can be seen, the embodiments of the present application can perform compatibility adaptation for the DOIS mechanism and the SOIS mechanism.

[0167] It should be noted that the above embodiments illustrate the implementation of the control path of the OIS system under two mechanisms. Similarly, for the specific implementation manner of the data path, reference can be made to Figure 6 the control path and will not be elaborated here.

[0168] In some embodiments, when the camera module refers to N cameras, SensorHub or the camera module can manage the information of N cameras (such as the first indication information) and data information (such as the position information of the lens in the camera, etc.) through N OIS hardware instances respectively, and the AP can manage the OIS services of N cameras through N OIS software instances respectively. The OIS hardware instance is used to abstractly describe the characteristics of the camera (including but not limited to the OIS mode of the camera, the trajectory of the lens of the camera, etc.) based on a structured data structure; the OIS software instance is used to abstractly describe the behavior implementation of the OIS service (including but not limited to turning on or off the OIS function of the camera) based on a structured data structure. It should be noted that the OIS hardware instance here can also be understood as a hardware module in the camera module. When the camera module supports the SOIS mechanism, the OIS hardware instance can be scheduled by SensorHub based on a unified device usage interface. When the camera module supports the DOIS mechanism, the OIS hardware instance can be scheduled by the AP based on a unified device usage interface. The OIS software instance can also be understood as an OIS thread in the AP. When the camera module supports the SOIS mechanism, the OIS software instance can be scheduled by the SensorHub control module on the AP side. When the camera module supports the DOIS mechanism, the OIS software instance can be scheduled by the camera driver on the AP side. It should be noted that the native SensorHub control module may not have an OIS software instance (for example, the native NCS QSEE module does not have an OIS software instance). The SensorHub control module involved in the embodiments of the present application can create a virtual OIS software instance on the basis of the native SensorHub control module.

[0169] The camera, the OIS hardware instance, and the OIS software instance are interrelated. Exemplarily, the camera has a corresponding camera identifier (camera ID), the OIS hardware instance has a corresponding hardware instance identifier (such as sensor uid, which can be abbreviated as suid), and the OIS software instance has a corresponding OIS software instance identifier (such as OIS ID). Taking a camera module that includes a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera, where the identifier of the wide-angle camera is camera ID1, the identifier of the ultra-wide-angle camera is camera ID2, and the identifier of the telephoto camera is camera ID3, and the identifiers of the 3 OIS hardware instances are suid1, suid2, and suid3 respectively, and the identifiers of the 3 OIS software instances are OIS ID1, OIS ID2, and OIS ID3 respectively as an example, Table 1 exemplarily shows the association relationship among the camera, the OIS hardware instance, and the OIS software instance. From the association relationship in Table 1, it can be seen that the wide-angle camera with the identifier camera ID1, the OIS software instance with the identifier OIS ID1, and the OIS hardware instance with the identifier suid1 are interrelated; the ultra-wide-angle camera with the identifier camera ID2, the OIS software instance with the identifier OIS ID2, and the OIS hardware instance with the identifier suid2 are interrelated; the telephoto camera with the identifier camera ID3, the OIS software instance with the identifier OIS ID3, and the OIS hardware instance with the identifier suid3 are interrelated.

[0170] Table 1

[0171] Camera identifier OIS hardware instance identifier OIS software instance identifier camera ID1 suid1 OIS ID1 camera ID2 suid2 OIS ID2 camera ID3 suid3 OIS ID3

[0172] Optionally, the association relationship between the OIS hardware instance and the camera is pre-configured and can be obtained by the electronic device when it is powered on. Optionally, the association relationship between the OIS software instance and the camera can be established when the camera application is launched through the None Camera Sensor Service (NCS Service). Then, the electronic device can interrelate the camera, the OIS hardware instance, and the OIS software instance based on the association relationship between the OIS hardware instance and the camera and the association relationship between the OIS software instance and the camera. For example, among N cameras including the first camera, N OIS software instances including the first OIS software instance, and N OIS hardware instances including the first OIS hardware instance, the first camera, the first OIS software instance, and the first OIS hardware instance are interrelated. The first camera can be a wide-angle camera, an ultra-wide-angle camera, or a telephoto camera, etc.

[0173] Exemplarily, forFigure 4 The described step S101 may specifically include: The AP enables the OIS function of each camera, determines the OIS software instance associated with each camera based on the association relationship between the OIS software instance identifier and the camera identifier, and generates the first indication information for each camera through the associated OIS software instance. That is to say, for the first camera, the first OIS software instance generates the first indication information, and the first indication information is used to indicate that the OIS function of the first camera has been enabled.

[0174] Exemplarily, for Figure 4 The described step S102 may specifically include: The AP determines the OIS software instance associated with the camera based on the association relationship between the camera identifier and the OIS software instance identifier, and sends the first indication information of each camera through the OIS software instance associated with each camera. That is to say, for the first camera, the first OIS software instance sends the first indication information.

[0175] Exemplarily, for Figure 4 The described step S103 may specifically include: The SensorHub can respond to the first indication information of each camera, determine the OIS hardware instance associated with each OIS software instance based on the association relationship between the OIS hardware instance identifier and the OIS software instance identifier, and run the OIS control algorithm through the associated OIS hardware instance to generate the second indication information for each camera. That is to say, the first OIS hardware instance generates the second indication information, and the second indication information is used to indicate the target position of the lens of the first camera.

[0176] Exemplarily, for Figure 4 The described step S104 may specifically include: The SensorHub determines the OIS hardware instance associated with the camera based on the association relationship between the camera identifier and the OIS hardware instance identifier, and sends the second indication information of the camera to the camera module through the OIS hardware instance associated with the camera. That is to say, for the first camera, the first OIS hardware instance sends the second indication information.

[0177] Exemplarily, for Figure 4 The described step S105 may specifically include: The camera module can receive the second indication information of each camera. At this time, the camera module can move each camera to the corresponding target position based on the second indication information of each camera. That is to say, the first camera can receive the target position of the lens of the first camera from the first OIS hardware instance.

[0178] Exemplarily, for Figure 4The specific steps of step S106 may include: the camera module determines the OIS hardware instance associated with each camera based on the association relationship between the OIS hardware instance identifier and the camera identifier, and sends the position information of each camera to the SensorHub through the OIS hardware instance associated with each camera. That is to say, for the first camera, the first OIS hardware instance may send the position information of the lens of the first camera to the SensorHub.

[0179] Exemplarily, for Figure 4 The specific steps of step S107 may include: the SensorHub determines the OIS software instance associated with each camera based on the association relationship between the OIS software instance identifier and the camera identifier, and sends the position information of each camera to the AP through the OIS software instance associated with each camera. That is to say, for the first camera, the first OIS software instance may send the position information of the lens of the first camera to the AP.

[0180] It can be seen that in the embodiment of the present application, an association relationship can be established among the camera, the OIS hardware instance, and the N OIS software instances, and the OIS hardware instance and the OIS software instance of each camera are bound and distinguished to implement the control path and data path of the corresponding camera.

Claims

1. A lens control method, characterized in that, the method is applied to an electronic device, the electronic device includes an application processor AP, a smart sensing hub SensorHub and N cameras, N is a positive integer greater than 1, the AP includes N optical image stabilization OIS software instances, the N OIS software instances are respectively associated with the N cameras, the SensorHub includes N OIS hardware instances, and the N OIS hardware instances are respectively associated with the N cameras; the AP enables the OIS function of multiple target cameras among the N cameras, and generates first indication information corresponding to each of the target cameras through the OIS software instance associated with each target camera, and each first indication information respectively indicates that the OIS function of each target camera has been enabled; the SensorHub responds to the first indication information corresponding to each target camera, and generates second indication information corresponding to each target camera through the OIS hardware instance associated with each target camera, and each second indication information respectively indicates the target position of the lens of each target camera; each target camera moves the lens from the current position to the target position based on the corresponding second indication information.

2. The method according to claim 1, characterized in that, the multiple target cameras are target cameras that support the SensorHub to run the OIS control algorithm; the SensorHub responds to the first indication information corresponding to each target camera, and generates second indication information corresponding to each target camera through the OIS hardware instance associated with each target camera, including: the SensorHub responds to the first indication information corresponding to each target camera, and runs the OIS control algorithm through the OIS hardware instance associated with each target camera to generate second indication information corresponding to each target camera.

3. The method according to claim 2, characterized in that, before the SensorHub responds to the first indication information corresponding to each target camera, and runs the OIS control algorithm through the OIS hardware instance associated with each target camera to generate second indication information corresponding to each target camera, the method further includes: the AP sends the first indication information corresponding to each target camera to the SensorHub through the OIS software instance associated with each target camera.

4. The method according to claim 3, characterized in that, before each target camera moves the lens from the current position to the target position based on the corresponding second indication information, the method further includes: the SensorHub sends the corresponding second indication information to each target camera respectively through the OIS hardware instance associated with each target camera.

5. The method according to any one of claims 2 to 4, characterized in that, the method further includes: The target camera sends the position information of the lens of the target camera to the SensorHub through the OIS hardware instance associated with itself.

6. The method according to claim 5, wherein, the method further includes: The SensorHub sends the position information of the lens of each target camera to the AP through the OIS software instance associated with each target camera.

7. The method according to claim 6, wherein, The SensorHub writes the position information of the lens of each target camera into the first memory through the OIS software instance associated with each target camera; The AP reads the position information of the lens of each target camera from the first memory; The first memory is a shared memory of the AP and the SensorHub.

8. The method according to any one of claims 1-6, wherein, The multiple target cameras include a first target camera that supports the OIS control algorithm to be run by the SensorHub, and a second target camera that supports the OIS control algorithm to be run by the camera; After the AP enables the OIS function of multiple target cameras among the N cameras and generates first indication information corresponding to each target camera through the OIS software instance associated with each target camera, the method further includes: The SensorHub responds to the first indication information corresponding to each first target camera, runs the OIS control algorithm through the OIS hardware instance associated with each first target camera to generate second indication information corresponding to each first target camera, so that each first target camera moves the lens from the current position to the target position indicated by the corresponding second indication information based on the corresponding second indication information; Each second target camera responds to the corresponding first indication information, runs the OIS control algorithm to determine the target position of the lens of each second target camera, and moves the lens of each second target camera from the current position to the target position.

9. An electronic device, wherein, The electronic device includes a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, wherein, A computer program is stored in the computer-readable storage medium. When it runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

11. A computer program product, wherein, The computer program product includes a computer program. When it runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images