Optical image stabilization control method and electronic device

Abstract

Embodiments of the present application provide an optical anti-shake control method and an electronic device, and relate to the technical field of terminals. The embodiments improve the problem of picture drift after switching of a camera mode. The specific scheme is as follows: a first preview interface of a camera application is displayed, wherein the first preview interface includes image data collected by a first camera module; at a first time, a first operation of switching a camera mode is received, and before the first time, the camera mode of the camera application is a first mode; between the first time and a second time, the position of a lens in the first camera module is adjusted in the case that the electronic device is shaken; and at the second time, a second preview interface is displayed, wherein the second preview interface includes image data collected by the first camera module, and after the second time, the camera mode of the camera application is a second mode.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an optical image stabilization control method and electronic device. Background Technology

[0002] With the widespread use of electronic devices, taking photos with them is becoming increasingly common. When people use electronic devices to take pictures, image blur is easily caused by hand tremors, device movement, and other factors. To improve image sharpness and reduce the effects of motion blur, optical image stabilization (OIS) has been introduced. OIS is a hardware-based image stabilization solution where a movable camera module installed in the electronic device compensates for the displacement of the light path, thereby reducing or eliminating motion blur. Summary of the Invention

[0003] This application provides an optical image stabilization control method and electronic device, which improves the image drift problem after switching camera modes when optical image stabilization is enabled.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] Firstly, embodiments of this application provide an optical image stabilization control method, which can be applied to electronic devices. The electronic device includes a camera application. This camera application can be an application that provides image capture services. Exemplarily, the camera application includes one or more camera modes. Different camera modes correspond to different shooting functions. In different camera modes, the electronic device can respond to user operations and capture image data of different types and visual effects, such as photos, videos, moving images, night scene images, portrait images, panoramic images, etc.

[0006] In an exemplary scenario, when a camera application is running in the foreground on an electronic device, a first preview interface can be displayed. This first preview interface includes image data captured by the first camera module. The first preview interface is a shooting preview interface for a first mode, which is a camera mode provided by the camera application. The first camera module can be a camera module that is enabled in the first mode. At a first moment after displaying the first preview interface, the electronic device receives a first operation to switch the camera mode. This first operation instructs the camera application to switch the camera mode from the first mode to a second mode. At a second moment, the electronic device displays a second preview interface, which is a shooting preview interface corresponding to the second mode. The first camera module is a camera module that is enabled in the second mode, and the second preview interface includes image data captured by the first camera module. After the second moment, the camera mode of the camera application is the second mode.

[0007] Between the first and second moments, the camera application switches the camera mode from the first mode to the second mode. Simultaneously, between the first and second moments, if the electronic device shakes, the position of the lens in the first camera module will be adjusted.

[0008] That is, in the above embodiments, the camera modules that can be enabled in the first mode and the second mode are the same, such as the first camera module in both cases. During the process of switching the camera mode from the first mode to the second mode, the electronic device can continue to perform optical image stabilization on the first camera module. For example, if the electronic device shakes during the camera mode switch, compensation is made by moving the lens position to ensure that the change in the image within the lens's field of view is minimal before and after the camera mode switch. In this way, there will be no image drift problem before / after the camera mode switch.

[0009] In some embodiments, if the camera modules that can be enabled in the camera mode before and after the switch are the same, it is not necessary to initialize optical image stabilization for that camera module after the camera mode switch. Initializing optical image stabilization may include moving the lens of the camera module to a preset position. The preset position can be a fixed position, such as a lens-centered position.

[0010] In an exemplary scenario, at a third time point following the second time point, the electronic device receives a second operation to switch camera modes. This second operation instructs the camera application to switch the camera mode from the second mode to the first mode. At a fourth time point, a first preview interface is displayed. After the fourth time point, the camera application's camera mode is the first mode. Before the third time point, the camera application's camera mode is the second mode.

[0011] Between the third and fourth moments, the camera mode switches from the second to the third mode. During this period, the electronic equipment remains stationary, and the position of the lens in the first camera module remains unchanged.

[0012] That is, in the above embodiments, during the camera mode switching process, the lens in the first camera module is not moved to a preset position, nor is the optical image stabilization device (e.g., OIS motor) in the first camera module initialized. After enabling the first mode, the first camera module does not need to start optical image stabilization from scratch, thus avoiding the problem of image drift after switching from the second mode.

[0013] In some embodiments, the electronic device further includes a Sensorhub. The electronic device controls the first camera module for optical image stabilization via the Sensorhub.

[0014] For example, when the electronic device is shaken, the position of the lens in the first camera module can be adjusted by the following method: transmitting an electrical signal to the first camera module via Sensorhub to adjust the position of the lens in the first camera module.

[0015] In the above embodiments, the Sensorhub controls the first camera module to perform optical image stabilization. Consequently, the first camera module does not need to be equipped with a microprocessor for optical image stabilization, reducing the cost and thickness of the first camera module. In addition, the Sensorhub has lower power consumption than the microprocessor used for optical image stabilization, resulting in lower energy consumption for performing the same optical image stabilization tasks, thereby effectively reducing the power consumption of the electronic device.

[0016] In some embodiments, if the camera modules that can be enabled in the camera mode before and after the switch are different, the electronic device does not control the camera module to perform optical image stabilization during the camera mode switching process.

[0017] In an exemplary scenario, the electronic device further includes a second camera module. After displaying the second preview interface, the electronic device receives a third operation to switch camera modes. This third operation instructs the camera application to switch the camera mode from the second mode to the third mode. The camera module available in the third mode is the second camera module. The camera modules available in the second and third modes are different. Based on the third operation, the electronic device can power down the first camera module. Understandably, after the first camera module is powered down, the OIS motor in the first camera module will also be powered down, and the first camera module will naturally be unable to perform optical image stabilization. The electronic device can also power on the second camera module based on the third operation. The electronic device can also display a third preview interface based on the third operation. This third preview interface is the shooting preview interface corresponding to the third mode, and it includes image data acquired by the second camera module.

[0018] It should be noted that "based on a third operation" can refer to the electronic device triggering subsequent steps such as powering down the first camera module, powering on the second camera module, and displaying a third preview interface after a third operation is detected. However, "based on a third operation" is not limited to only detecting the third operation before triggering subsequent steps. For example, the electronic device may detect other operations before or after the third operation is detected, and these other operations, along with the third operation, may jointly trigger the execution of actions such as powering down the first camera module, powering on the second camera module, and displaying a third preview interface.

[0019] Before the first camera module is powered off, the camera is in the second camera mode. After the second camera module is powered on, the camera is in the third camera mode. While controlling the second camera module to be powered on, adjust the position of the lens in the second camera module if the electronic device is shaken.

[0020] In the above embodiments, during the process of switching the camera mode from the second mode to the third mode, the electronic device does not control any camera module to perform optical image stabilization, so as to avoid unnecessary power consumption.

[0021] In some embodiments, the electronic device also includes a Sensorhub. The electronic device controls the second camera module for optical image stabilization via the Sensorhub.

[0022] For example, when an electronic device is shaken, the position of the lens in the second camera module can be adjusted by transmitting an electrical signal to the second camera module via Sensorhub.

[0023] In the above embodiments, the Sensorhub can control multiple camera modules to perform optical image stabilization. One Sensorhub can replace the microprocessor used to control optical image stabilization in multiple camera modules, which reduces the number of hardware components in electronic devices and also reduces power consumption while ensuring the optical image stabilization effect.

[0024] In some embodiments, before the electronic device controls the first camera module to power down in response to the third operation, the electronic device may also control the lens in the first camera module to move to a preset position.

[0025] After the lens in the first camera module is moved to the preset position, and before the first camera module is powered off, the electronic device shakes, and the position of the lens in the first camera module remains unchanged. That is, the electronic device pauses the control of the first camera module to perform optical image stabilization, thereby reducing power consumption.

[0026] In some embodiments, when the camera mode is the second mode, the enabled camera modules include a third camera module in addition to the first camera module. After displaying the second preview interface, the electronic device receives a fourth operation to switch lenses. For example, zooming can trigger the switching of camera modules, and correspondingly, the fourth operation can be an operation indicating zoom. In response to the fourth operation, at a fifth moment, the electronic device displays image data acquired by the third camera module in the second preview interface. While displaying the image data acquired by the third camera module in the second preview interface, the first camera module remains powered on. Similarly, while displaying the image data acquired by the first camera module in the second preview interface, the third camera module also remains powered on. After the fifth moment, the electronic device shakes, transmitting electrical signals to the third camera module via Sensorhub to adjust the position of the lens in the third camera module, while keeping the position of the lens in the first camera module unchanged.

[0027] In the above embodiments, both the first camera module and the third camera module are camera modules available in the second mode. When the first camera module is displaying data, the Sensorhub can control the first camera module to perform optical image stabilization; when the third camera module is displaying data, the Sensorhub can control the third camera module to perform optical image stabilization. In the same camera mode, the same Sensorhub can control different camera modules in a time-division manner, reducing power consumption and thus reducing the number of hardware components in the electronic device.

[0028] In some embodiments, the electronic device includes an application processor (AP), a first sensor, and a sensorhub. The first sensor may be a gyroscope, an accelerometer, or a sensor device integrating a gyroscope and an accelerometer. When the camera mode of the camera application is in a first mode, in response to a first operation, the AP sends a first control command to the sensorhub, instructing the sensorhub to continue controlling the first camera module for optical image stabilization during camera mode switching. Accordingly, in response to the first control command, the sensorhub continues to receive gyroscope data and / or acceleration data collected by the first sensor. The sensorhub can adjust the position of the lens in the first camera module based on the gyroscope data and / or acceleration data, i.e., to control the first camera module for optical image stabilization.

[0029] In the above embodiments, by continuing to control the first camera module to perform optical image stabilization during camera mode switching, the problem of image drift caused by the shaking of electronic devices during camera mode switching is avoided.

[0030] In some embodiments, when the camera mode of the camera application is the second mode, in response to a third operation, the AP sends second control information to Sensorhub, instructing Sensorhub to stop controlling the camera module to perform optical image stabilization during camera mode switching. After receiving the second control information, Sensorhub controls the lens of the first camera module to move to a preset position. Afterward, even if the electronic device shakes, the position of the lens in the first camera module will no longer be adjusted. That is, Sensorhub stops controlling the first camera module to perform optical image stabilization, avoiding unnecessary optical image stabilization during camera mode switching and reducing device power consumption.

[0031] In some embodiments, when the camera mode of the camera application is the second mode, in response to the third operation, after the second camera module is powered on, the Sensorhub controls the lens of the second camera module to move to a preset position, and then receives the gyroscope data and / or acceleration data collected by the first sensor, and adjusts the position of the lens in the second camera module based on the gyroscope data and / or acceleration data, that is, to realize the control of the second camera module to perform optical image stabilization.

[0032] In some embodiments, in response to opening the camera application, the electronic device runs the camera application in the foreground and displays a first preview interface. Additionally, in response to opening the camera application, the AP sends a first data packet to Sensorhub. The first data packet includes initialization configuration parameters for the optical image stabilization motor in the first camera module. After receiving and buffering the first data packet, Sensorhub sends an initialization success message to the AP.

[0033] Subsequently, when optical image stabilization is enabled in the first mode, Sensorhub receives gyroscope data and / or acceleration data collected by the first sensor, and adjusts the position of the lens in the first camera module based on the gyroscope data and acceleration data, thereby controlling the camera module to perform optical image stabilization.

[0034] In some embodiments, the electronic device further includes a fourth camera module, and the electronic device receives a fifth operation to switch camera modes. The fifth operation indicates that the enabled camera mode is the fourth mode. Before receiving the fifth operation, the camera mode applied by the camera is different from the fourth mode, for example, it could be the first mode, the second mode, or the third mode, etc., without specific limitation. In response to the fifth operation, the electronic device controls the fourth camera module to power on.

[0035] If optical image stabilization is not supported in the fourth mode, Sensorhub can initialize the optical image stabilization components in the fourth camera module, such as initializing the OIS motor to move the lens of the fourth camera module to a preset position. Then, the AP controls Sensorhub to release the system resources used for handling optical image stabilization.

[0036] The optical image stabilization process may include: processing gyroscope data and / or acceleration data from the first sensor; calculating the displacement offset of the optical image stabilization motor in the camera module based on the gyroscope data and / or acceleration data; sending control information carrying the displacement offset to the camera module; and processing control information from the AP for optical image stabilization of the camera module. In this way, when the electronic device shakes, Sensorhub will no longer adjust the position of the lens in the fourth camera module, that is, it will also stop controlling the fourth camera mode for optical image stabilization.

[0037] In the above embodiments, even without enabling optical image stabilization, the lens of the fourth camera module can be centered to ensure the shooting effect.

[0038] In some embodiments, the AP can determine whether the fourth mode supports enabling optical image stabilization. Specifically, when the camera mode is a single-camera mode, it can be determined that the camera mode supports enabling optical image stabilization. When the camera mode is a multi-camera mode and the multi-camera algorithm configured in the electronic device does not support optical image stabilization, it can be determined that the camera mode does not support enabling optical image stabilization. Conversely, when the camera mode is a multi-camera mode and the multi-camera algorithm configured in the electronic device supports optical image stabilization, it can be determined that the camera mode supports enabling optical image stabilization.

[0039] Secondly, embodiments of this application provide an optical image stabilization control method applied to an electronic device. The electronic device includes a first camera module, a second camera module, and a Sensorhub. The electronic device displays a first preview interface, which includes image data acquired by the first camera module. When the electronic device shakes, an electrical signal is transmitted to the first camera module via the Sensorhub to adjust the position of the lens in the first camera module. In response to a third operation, a third preview interface is displayed, which includes image data acquired by the second camera module. When the electronic device shakes, an electrical signal is transmitted to the second camera module via the Sensorhub to adjust the position of the lens in the second camera module.

[0040] In the above embodiments, the Sensorhub can control multiple camera modules to perform optical image stabilization. One Sensorhub can replace the microprocessor used to control optical image stabilization in multiple camera modules, which reduces the number of hardware components in electronic devices and also reduces power consumption while ensuring the optical image stabilization effect.

[0041] Thirdly, embodiments of this application provide an electronic device including a Sensorhub, one or more camera modules, a processor, and a memory, wherein the memory is used to store code instructions; and the processor is used to run the code instructions, causing the electronic device to perform the methods described in the first aspect, the second aspect, and any implementation thereof.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed, cause a computer to perform the methods described in the first aspect, the second aspect, and any implementation thereof.

[0043] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when run, causes a computer to perform the methods described in the first aspect, the second aspect, and any of their implementations.

[0044] It should be understood that the second to fifth aspects of the embodiments of this application correspond to the technical solutions of the first aspect of the embodiments of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be described again. Attached Figure Description

[0045] Figure 1 These are example diagrams illustrating a portion of the structure of an electronic device in some embodiments;

[0046] Figure 2 Example diagram of an on-chip system in an electronic device provided in this application embodiment;

[0047] Figure 3 Example diagrams of a portion of the structure of an electronic device provided in the embodiments of this application;

[0048] Figure 4 A software system architecture diagram of an electronic device provided in the embodiments of this application;

[0049] Figure 5 Example diagram of NCS initialization provided in the embodiments of this application;

[0050] Figure 6 A flowchart of an electronic device provided in an embodiment of this application;

[0051] Figure 7 One of the interaction diagrams between the software and hardware modules of the electronic device provided in the embodiments of this application;

[0052] Figure 8 The second interaction diagram between the software and hardware modules of the electronic device provided in the embodiments of this application;

[0053] Figure 9An interaction diagram between software modules of the electronic device provided in the embodiments of this application;

[0054] Figure 10 Example diagrams of part of the hardware structure of the electronic device provided in the embodiments of this application;

[0055] Figure 11A The third diagram illustrating the interaction between the software and hardware modules of the electronic device provided in this application embodiment;

[0056] Figure 11B The fourth diagram illustrating the interaction between the software and hardware modules of the electronic device provided in this application embodiment;

[0057] Figure 12 A timing diagram illustrating the execution of the method provided in the embodiments of this application;

[0058] Figure 13 Example diagram of a scenario for launching a camera application provided in an embodiment of this application;

[0059] Figure 14 One of the signaling interaction diagrams for an optical image stabilization control method provided in this application embodiment;

[0060] Figure 15 This is an example diagram of a scene for switching camera modes provided in an embodiment of this application;

[0061] Figure 16 The second signaling interaction diagram of an optical image stabilization control method provided in the embodiments of this application;

[0062] Figure 17 The third signaling interaction diagram of an optical image stabilization control method provided in the embodiments of this application;

[0063] Figure 18 This is an example diagram illustrating the Sensorhub control of a camera module during the operation of an electronic device, as provided in an embodiment of this application. Detailed Implementation

[0064] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0065] To make the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is given first:

[0066] OIS (Optical Image Sensor) function: During image exposure, it detects shake data from electronic devices based on motion sensors (such as gyroscopes and accelerometers). For example, gyroscope data (deflection angle, deflection speed, and deflection time) and accelerometer data. Based on the detected gyroscope and accelerometer data, it calculates the amount of displacement that needs to be compensated, and then moves the lens or image sensor in the camera module according to this displacement to counteract the shake, thus obtaining a clear, exposed image.

[0067] like Figure 1 As shown, the gyroscope and accelerometer in an electronic device can also be referred to as A+G devices, where A represents the accelerometer and G represents the gyroscope. Optionally, an A+G device represents two separate devices, or alternatively, an A+G device represents a composite device that combines the functions of an accelerometer and a gyroscope. The A+G device may include a high-frequency sampling channel (also known as a high-precision sampling channel) and a low-frequency sampling channel (low-precision sampling channel). For example, the high-frequency sampling channel can collect gyroscope and acceleration data required for OIS functionality; for instance, the high-frequency sampling channel could be a Gyro OIS Filter. The low-frequency sampling channel can collect gyroscope and acceleration data required for system functions (e.g., shake-to-wake function); for instance, the low-frequency sampling channel could be a Gyro UI Filter. The sampling frequency of the high-frequency sampling channel is higher than that of the low-frequency sampling channel, and the accuracy of the gyroscope and acceleration data collected by the high-frequency sampling channel is also higher than that of the low-frequency sampling channel.

[0068] like Figure 1 As shown, the camera module of the electronic device is equipped with an OIS controller, which includes a microcontroller unit (MCU) processor.

[0069] The MCU processor communicates with the A+G devices via an inter-integrated circuit (I2C) bus. The MCU processor can obtain high-precision gyroscope and acceleration data from the A+G devices' Gyro OIS filter. Additionally, the MCU processor is equipped with OIS-related algorithms (referred to as OIS algorithms). The MCU processor can run the OIS algorithms to process the gyroscope and acceleration data, calculate the displacement used to counteract jitter, control the OIS motor in the camera module based on the displacement, and obtain the Hall IC position corresponding to the OIS motor. The OIS motor adjusts the physical position of the lens or image sensor in the camera module to counteract the displacement caused by jitter. The Hall IC is a chip that detects the position parameters of the motor control axis; the data detected by this chip is called the Hall IC position.

[0070] The MCU processor can also control the AF motor and obtain the position of the Hall IC corresponding to the AF motor. The AF motor adjusts the relative position between the lens and the image sensor in the camera module, changing the focal length of the camera module to achieve focusing.

[0071] For example, the OIS controller also stores the voice coil motor (VCM) driver corresponding to the OIS motor and the VCM driver corresponding to the AF motor. The MCU can drive the OIS motor through the VCM driver corresponding to the OIS motor, or it can drive the AF motor through the VCM driver corresponding to the AF motor. This application embodiment does not specifically limit this.

[0072] In addition, electronic devices are equipped with intelligent sensor hubs, which are integrated onto a separate processor. The sensor hub can be a part of the separate processor (e.g., an MCU / little core) or the entire processor. Optionally, a sensor hub can refer to the separate processor, the program running on the separate processor, or a combination of both.

[0073] Optionally, the aforementioned independent processor can be a digital signal processor (DSP), an analog devices signal processor (ADSP), or a system control processor (SCP).

[0074] Optionally, the aforementioned independent processor can be a low-power processor, where a low-power processor is a processor with lower power consumption than an application processor (AP).

[0075] Understandably, Sensorhub is integrated on a processor that is relatively independent and has lower power consumption compared to AP, and Sensorhub runs a lightweight RTOS operating system. When Sensorhub and AP perform the same task, Sensorhub generates less power consumption.

[0076] Optionally, Sensorhub can control various sensors configured in electronic devices and process data from these sensors. For example, Sensorhub can communicate with A+G devices via a Serial Peripheral interface (SPI) bus. Sensorhub can obtain low-precision gyroscope and acceleration data from the Gyro UI Filter of the A+G device via the SPI bus.

[0077] like Figure 2 As shown, the application processor (AP) and DSP of an electronic device can be two independent chips integrated in a system-on-chip (SOC). During AP sleep, the Sensorhub in the DSP can continue to work, such as continuing to receive and process data acquired by various sensors.

[0078] In an exemplary scenario, an electronic device is equipped with a touchscreen and a display screen. The display screen has the ability to display images, and the touchscreen generates corresponding touch data when touched. Optionally, the touchscreen and display screen can be integrated into a single screen. Alternatively, the touchscreen and display screen can be two separate screens, with the touchscreen covering the display screen. After the electronic device's screen is turned off, the AP enters sleep mode, but the Sensorhub can continue to receive touch data sent by the touchscreen. When a user taps the touchscreen, the Sensorhub wakes up the AP based on the touch data from the touchscreen and reports the touch event to the AP. High-power APs do not need to be constantly active, reducing device power consumption and maintaining the normal operation of basic functions.

[0079] like Figure 2 As shown, the SOC also integrates other chips, such as a baseband processor (BP), etc. This application embodiment does not specifically limit this.

[0080] Currently, electronic devices are equipped with numerous camera modules, all of which are required to have OIS (Optical Image Sensor) functionality. For example... Figure 1 As shown, each camera module's OIS controller needs to include an MCU with an OIS algorithm deployed. Configuring an MCU with an OIS algorithm in all camera modules will undoubtedly increase the manufacturing cost of electronic devices. In addition, the MCU integrated in the OIS controller will increase the thickness of the camera module, thereby increasing the thickness of the electronic device, making it difficult to meet users' requirements for a thin and light electronic device.

[0081] Low-cost software-based optical image stabilization (SOIS) refers to replacing the MCU that deploys the OIS algorithm in the OIS controller with a Sensorhub. Compared to OIS, SOIS eliminates the need for an MCU to deploy the OIS algorithm in any camera module, reducing the cost and thickness of the camera module. Furthermore, the Sensorhub consumes less power than the MCU that deploys the OIS algorithm; the Sensorhub generates less power during the process of controlling the camera module for optical image stabilization.

[0082] like Figure 3 As shown, Sensorhub is configured with an SOIS driver, which includes the OIS algorithm. Sensorhub can communicate with A+G devices via I2C, SPI, and I3C (improved inter-integrated circuit) buses. Specifically, Sensorhub can obtain low-precision gyroscope and acceleration data from the A+G device's Gyro UI Filter via the I2C bus. Sensorhub can obtain high-precision gyroscope and acceleration data from the A+G device's Gyro UI Filter via either the SPI or I3C bus. For example, Sensorhub can select an idle bus from the SPI and I3C buses and use this idle bus to obtain high-precision gyroscope and acceleration data from the A+G device's Gyro UI Filter.

[0083] and Figure 1 The difference is, Figure 3 A Sensorhub with an SOIS driver can handle OIS-related matters, allowing it to replace the MCU with an OIS algorithm in the OIS controller. This eliminates the need for an MCU with an OIS algorithm in the camera module of an electronic device, reducing both manufacturing costs and the thickness of the camera module.

[0084] This application provides an optical image stabilization control method applied to electronic devices. These electronic devices can achieve optical image stabilization based on SOIS. The optical image stabilization control method provided in this application is described below with reference to the accompanying drawings.

[0085] In some embodiments, the aforementioned electronic device may be a mobile phone, tablet computer, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, or other Android device, etc. The embodiments of this application do not impose special limitations on the specific form of the electronic device.

[0086] It is understood that the software system of electronic devices can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses a layered architecture system as an example to exemplify the software structure of an electronic device.

[0087] Figure 4 The diagram illustrates the software system architecture of an electronic device. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the electronic device may include, for example, an application layer, a framework layer, a hardware abstraction layer (HAL), and a kernel layer. Of course, it may also include... Figure 4 Layers not shown in the diagram include, for example, runtime and system libraries.

[0088] For example, the application layer may include a series of applications.

[0089] like Figure 4 As shown, the application layer may include applications such as camera apps. In some embodiments, the application layer may also include... Figure 4 Applications not shown include, for example, other applications with shooting functions (live streaming applications, beauty camera applications, etc.), as well as applications for calling, camera, maps, navigation, etc.

[0090] For example, the framework layer provides the application programming interface (API) and programming framework for the application layer. The framework layer includes some predefined functions. For example... Figure 4As shown, the application framework layer can include a camera service. This camera service provides various services that implement the interface functionality of the Camera API. Applications with shooting capabilities can obtain relevant services from the camera service through the Camera API.

[0091] The aforementioned camera services may include functional modules such as Connect device, configure stream, set repeating request, and camera output stream.

[0092] The aforementioned functional module of the connecting device (connecting device module): When the camera application (or other application with shooting capabilities) starts, the camera application can view the available logical cameras and related configuration parameters by calling the connecting device module in the camera service. Here, a logical camera can be an abstract device composed of one or more physical cameras. The logical camera performs certain camera functions, such as large aperture and zoom, by controlling one or more physical cameras. A physical camera can refer to a physical camera module configured in an electronic device, and each physical camera has a unique identifier, such as cameraindex information.

[0093] The aforementioned flow distribution module (flow distribution module): After the camera application runs, the camera application can trigger the creation of various functional nodes through the flow distribution module. Different functional nodes can correspond to different hardware, and these functional nodes can control the corresponding hardware to implement various functions. Taking the sensor node as an example, the flow distribution module can create a sensor node, which is a node created for the camera module. The sensor node can send control commands to the camera module. For example, the control commands can be commands to configure camera parameters or commands to control image acquisition. In addition, the sensor node can also create an OIS node, which is a node created for OIS-related devices (such as OIS motors and Hall ICs). The OIS node can send control commands for OIS-related devices to the Sensorhub. For details, please refer to the subsequent embodiments, which will not be elaborated here.

[0094] The above-mentioned repeat request setting module (repeat request setting module): The camera application can configure the image acquisition frequency of the camera module through the repeat request setting module.

[0095] The aforementioned camera output stream functional module (camera output stream module): The camera output stream module can determine the currently enabled camera mode. For example, the camera mode can be, but is not limited to, normal photo mode, bokeh mode (e.g., portrait mode), or video recording mode.

[0096] For example, the hardware abstraction layer (HAL) can encapsulate drivers in the kernel layer and provide an interface for calling the framework layer, shielding the implementation details of the low-level hardware.

[0097] like Figure 4 As shown, the HAL of an electronic device includes at least a camera HAL. The camera HAL includes a SWL (Software Node Library). The SWL is a set of nodes that do not have independent computing capabilities and must rely on the CPU to run, such as sensor nodes and OIS nodes. When the electronic device starts a camera application, it establishes the aforementioned sensor nodes and OIS nodes corresponding to the camera modules required by the camera application.

[0098] The aforementioned camera HAL can also include a non-camera service (NCS). The NCS enables communication between the non-camera component and the camera component. For example, it establishes a communication channel between the camera HAL (camera component) and Sensorhub (non-camera component), supporting data transmission between the camera HAL and Sensorhub.

[0099] Additionally, the electronic device can create an NCS during the power-on process. Furthermore, the electronic device can establish a communication channel between the camera HAL and Sensorhub via the NCS; specific implementation details are described in subsequent embodiments. The aforementioned camera HAL may also include a Core and a CHI-CDK. The Core is used to interact with the CHI-CDK. The CHI-CDK is a collection of code implementations for customized requirements. This collection of code implementations includes XML file components and algorithm module components. The XML file component includes various types of XML files.

[0100] For example, the aforementioned XML files may include XML files corresponding to products, sensors, modules, EEPROMs, actuators, flash, and OIS files.

[0101] The XML files for each component are as follows: `product` contains the product-specific content of the electronic device; `sensor` contains private information and register configuration parameters for various sensors within the electronic device; `module` contains initialization configuration parameters for various sensors; `Eeprom` contains configuration parameters for the EEPROM external storage module; `actuator` contains configuration parameters for different focusing modules (e.g., AF motor); and `flash` contains configuration parameters for the flash module.

[0102] The aforementioned OIS file includes configuration parameters for OIS-related devices (OIS motor, Hall IC), such as the model of the OIS motor, the parameters used to calibrate the OIS motor, the parameters used to calibrate the Hall IC, and the sampling frequency of the Hall IC.

[0103] For example, the aforementioned algorithm module may include a platform algorithm module and platform metadata. The platform algorithm module can be used to determine whether the camera mode enabled by the camera application can enable the SOIS function. The aforementioned platform metadata is used to store the determination result of the platform algorithm module, facilitating other modules in the camera HAL to read the determination result. Specific implementation details can be found in subsequent embodiments.

[0104] The kernel layer must at least include the Qualcomm Message Interface (QMI) bus service. The QMI bus service is a functional interface service for inter-process communication among multiple processors. For example, the QMI bus service schedules the QMI bus, enabling communication between the application processor (AP) and the sensor hub via the QMI bus. Additionally, the kernel layer may also include... Figure 4 Drivers not shown, such as camera sensor drivers, display drivers, sensor drivers, etc.

[0105] The aforementioned application layer, framework layer, HAL, and kernel layer run on the AP. The electronic device also includes a Sensorhub and multiple camera modules, such as a rear main camera module, a rear telephoto camera module, and a rear wide-angle camera module.

[0106] For example, the Sensorhub mentioned above is configured with an SOIS driver. The SOIS driver may include functional modules such as an initialization service module, a jitter data acquisition module, a stabilization module identification module, and an OIS control module.

[0107] The initialization service module executes the initialization service process. The jitter data acquisition module acquires high-precision gyroscope and acceleration data from the A+G device. The image stabilization module identification module identifies the camera module requiring image stabilization. The OIS control module controls the operating status of OIS-related devices in the camera module. For example, the OIS control module powers on or off the OIS motor and Hall IC. It also controls the movement of the OIS motor's control axis to adjust the physical position of the lens or image sensor in the camera module. Furthermore, it controls the Hall IC to initiate position detection. In some embodiments, the electronic device initiates the camera service in response to a user opening the camera application. The camera service can send a driver registration command to the NCS. The NCS, in response to the driver registration command, executes the driver registration process.

[0108] As one implementation method, the process of performing driver registration is as follows:

[0109] First, after receiving the driver registration command, NCS can perform initialization. For example... Figure 5 As shown, NCS creates the NCSIntfQSEE class. The NCSIntfQSEE class performs one or more of the following initialization steps: creates the NCSSensorData data structure buffer linked list; creates the QMI bus connection service; registers the QMI bus heartbeat signal detection service; and registers the Qualcomm Secure Execution Environment (QSEE) and CHI-CDK scheduling API.

[0110] like Figure 5 As shown, the NCSIntfQSEE class can request Sensorhub driver identifiers (SUIDs) corresponding to multiple sensor types from Sensorhub. These multiple sensor types include OIS-related devices in camera modules. Understandably, when a sensor is mounted on Sensorhub, Sensorhub can include the drivers corresponding to that sensor type. In Sensorhub, each sensor type driver has a corresponding SUID, which uniquely identifies the driver. Furthermore, the SUIDs and identification information of each sensor can be stored correspondingly in Sensorhub.

[0111] As one implementation, NCS can query the XML file corresponding to the sensor through the CHI-CDK scheduling API. Based on the XML file corresponding to the sensor, NCS can determine the identification information of all types of sensors configured in the electronic device. NCS can then iterate through each identified identification and request the SUID of the sensor indicated by that identification from Sensorhub.

[0112] For example, electronic devices may contain gyroscopes, accelerometers, flashlights, and OIS-related devices. The NCS can retrieve the identification information of these sensors from their corresponding XML files. The NCS can then sequentially request the SUIDs corresponding to the gyroscope, accelerometer, flashlight, and OIS-related devices from Sensorhub.

[0113] like Figure 5 As shown, the process by which the NCS sequentially requests SUIDs from Sensorhub for the gyroscope, accelerometer, flash, and OIS-related devices is as follows:

[0114] If the NCS finds the gyroscope's identification information, it creates a `Suidlookup` class for the gyroscope and sends a request to Sensorhub to find the gyroscope's SUID. This request includes the gyroscope's identification information. If Sensorhub contains the gyroscope's driver, it responds to the SUID request, obtains the gyroscope's SUID1, and sends it back to the `NCSIntfQSEE` class via a callback. Upon receiving SUID1, the NCS records the correspondence between SUID1 and the gyroscope's identification information. If Sensorhub does not contain the gyroscope's driver, it sends a "fail" message to the `NCSIntfQSEE` class via a callback, indicating that the gyroscope's SUID retrieval failed.

[0115] After traversing the identification information of the accelerometer in the NCS, a Suidlookup class for the accelerometer is created. This Suidlookup class sends a request to Sensorhub to look up the accelerometer's SUID. This request includes the accelerometer's identification information. If Sensorhub contains the accelerometer driver, it responds to the SUID lookup request, obtains the accelerometer's SUID2, and sends it back to the NCSintfQSEE class via a callback. Upon receiving SUID2, the NCS records the correspondence between SUID2 and the accelerometer's identification information. If Sensorhub does not contain the accelerometer driver, it sends a "fail" message to the NCSintfQSEE class via a callback, indicating that the accelerometer SUID acquisition failed. This process is repeated for the flare's SUID3 and the SUID4 of OIS-related devices, etc.

[0116] In some embodiments, NCS can generate a sensor list based on the sensor's identification information and corresponding SUID.

[0117] Secondly, such as Figure 5 As shown, after generating the Sensor list, NCS can also obtain the initialization configuration parameters for various types of sensors from CHI-CDK via the CHI-CDK scheduling API. These initialization configuration parameters can be parameters required for sensor initialization, such as calibration parameters. Based on the initialization configuration parameters, NCS sequentially checks whether the data interaction channels (also known as Handles) indicated by the Suid in the Sensor list are valid.

[0118] As one implementation, NCS uses the data interaction channel corresponding to Suid1 to send the gyroscope's initialization configuration parameters to Sensorhub. For example, the gyroscope's Suid1 is encapsulated in the initialization configuration parameters. Then, using the communication channel between NCS and Sensorhub, the initialization configuration parameters encapsulated with Suid1 are sent to Sensorhub. Understandably, after Sensorhub parses Suid1 from the received initial configuration parameters, it passes the initial configuration parameters to the gyroscope driver for storage. Sensorhub can also send a registration success message to NCS. After receiving the registration success message, NCS records that the gyroscope's data interaction channel is valid. If Sensorhub fails to parse a valid Suid from the received initial configuration parameters, or if data transmission to the gyroscope driver fails, Sensorhub can send a registration failure message to NCS. NCS can then record that the data interaction channel corresponding to Suid1 is invalid. When the data interaction channel corresponding to Suid1 is invalid, the functional modules in the camera HAL cannot control the gyroscope through Sensorhub.

[0119] The NCS uses the data interaction channel corresponding to Suid2 to send the initial configuration parameters of the accelerometer to the Sensorhub. Similarly, after parsing Suid2 from the received initial configuration parameters, the Sensorhub passes the initial configuration parameters to the accelerometer driver for storage. The Sensorhub can also send a success message to the NCS. Upon receiving the success message, the NCS records that the accelerometer's data interaction channel is valid. If the Sensorhub fails to parse a valid Suid from the received initial configuration parameters, or if data transmission to the accelerometer driver fails, the Sensorhub can send a failure message to the NCS, and the NCS can record that the data interaction channel corresponding to Suid2 is invalid. This process continues until the validity of the data interaction channel corresponding to Suid3 is determined and recorded.

[0120] Understandably, NCS checks the validity of the data interaction channels corresponding to various sensors by instructing Sensorhub to initialize the drivers for each type of sensor.

[0121] In addition, OIS-related devices belong to the camera module. When the camera application is opened on the electronic device, the camera service can control the camera module to initialize. NCS does not need to perform initialization processing for OIS-related devices, nor does it check whether the data interaction channel corresponding to Suid4 is valid, thus avoiding repeated initialization.

[0122] In some embodiments, after an electronic device opens the camera application and starts the camera service, the camera application can initiate initialization processing for the camera module through the camera service.

[0123] As one implementation method, such as Figure 6 As shown, after opening the camera application, it can call the Connect device in the camera service. The Connect device can then call the open function in the camera HAL to power on the target camera module. The target camera module mentioned above is the camera module that the camera application can power on and operate in the current camera mode; it can also be referred to as the logical camera corresponding to the current camera mode. The camera HAL can query information about the logical cameras that the camera application can enable (halDevice) and obtain the camera parameters required by the camera module corresponding to each logical camera (getCameraParams).

[0124] After the camera's HAL executes the open function, as follows Figure 7 As shown, the camera application can also call the configurestream in the camera service. The configurestream can instruct the camera HAL to create the sensor node corresponding to the target camera module. The sensor node can create the OIS node.

[0125] The OIS node includes an SOIS client, which can be used to interact with the NCS.

[0126] In some embodiments, after an OIS node is created, it can be determined whether the current electronic device is in production line mode. If it is not in production line mode, it can be determined whether SOIS can be enabled.

[0127] Understandably, the platform algorithm module in CHI-CDK can determine whether SOIS is currently supported based on the camera mode currently enabled by the camera application, and write the determination result into the platform metadata. Details of CHI-CDK's determination of whether SOIS is supported can be found in subsequent embodiments, and will not be elaborated here. The OIS node can determine whether SOIS is currently supported by reading the above determination result from the platform metadata.

[0128] If the OIS node determines that it currently supports SOIS, it can call the setupSOISlink function to establish a data interaction channel between the OIS node and the SOIS driver in Sensorhub, which can also be called establishing an SOIS connection.

[0129] For example, after the OIS node calls the setupSOISlink function, the SOIS client in the OIS node can control the NCS to send channel establishment information to Sensorhub, where the channel establishment information includes the Suid4 of the OIS-related device.

[0130] As one implementation, NCS includes the NCS sensor, the NCS intf qsee module, and the QSEE (snapdragon sensor core, SSC) connection.

[0131] The NCS sensor can access the sensor list to find the SUID corresponding to the sensor mounted under Sensorhub. For example, the NCS sensor can find the SUID4 corresponding to the OIS-related device. The NCS intf qsee module can generate channel establishment information containing SUID4 according to pre-configured rules and pass it to the QSEE SSC connection. The QSEESSC connection, through the QMI server in the camera HAL, calls the kernel-level QMI bus service to pass the above channel establishment information to Sensorhub.

[0132] After receiving the channel establishment information, Sensorhub parses it. If Sensorhub parses Suid4 from the channel establishment information, it can pass the channel establishment information to the SOIS driver. In response to this channel establishment information, the SOIS driver allocates memory resources and QMI bus port resources for the data interaction channel corresponding to the OIS node. Then, the SOIS driver sends SOIS connection establishment information to the OIS node via NCS.

[0133] In some embodiments, after a data exchange channel is established between the OIS node and the SOIS driver in Sensorhub, the OIS node can initialize the SOIS driver. For example... Figure 8 As shown, the OIS node can obtain the OIS file from CHI-CDK via the core. The OIS file includes the configuration parameters corresponding to the OIS-related devices. It is understandable that CHI-CDK includes an XML file portion and an algorithm module portion. For ease of understanding, Figure 7 and Figure 8 The CHI-CDK shown is different.

[0134] The OIS node can obtain the target configuration parameters from the OIS file. These target configuration parameters, also known as the first data packet, can be initialization parameters for OIS-related devices in the target camera module, such as the sampling frequency of the Hall IC, the calibration parameters of the Hall IC, and the calibration parameters of the OIS motor.

[0135] like Figure 8 As shown, the process by which an OIS node initializes the SOIS driver is as follows:

[0136] S1, the OIS node can send the target configuration parameters and target cameraindex information to the NCS. Each camera module should have a unique cameraindex. The target cameraindex information can be the cameraindex information corresponding to the target camera module. For example, in the current camera mode, only one target camera module is acquiring images, and the target cameraindex information is the cameraindex information of that target camera module. Optionally, in the same camera mode, the target camera module actually acquiring images may change, and correspondingly, the target cameraindex information will also change. For example, if the target camera module actually acquiring images changes from the rear main camera to the rear wide-angle camera, the target cameraindex information will also change from the rear main camera's cameraindex information to the rear wide-angle camera's cameraindex information.

[0137] For example, in the current camera mode, multiple target camera modules acquire images simultaneously, and the target cameraindex information is the cameraindex information of the main camera module in the target camera module.

[0138] As one implementation, the platform algorithm module can determine the actual camera module requiring image stabilization when deciding whether SOIS is supported. After identifying the camera module requiring stabilization, the platform algorithm module can also write the target camera index information of that camera module into the platform metadata. Before sending the target configuration parameters, the OIS node can obtain the target camera index information from the platform metadata. In this way, the OIS node can send both the target configuration parameters and the target camera index information to the NCS, which then forwards them to Sensorhub.

[0139] For example, an OIS node can pass target configuration parameters and target cameraindex information to the NCS through an SOIS client.

[0140] S2, NCS passes the target configuration parameters and target cameraindex information to the SOIS driver in Sensorhub.

[0141] For example, the NCS sensor in the NCS queries the SUID4 corresponding to the OIS-related device. The NCS intf qsee module encapsulates the target configuration parameters and target camera index information based on the SUID4, and then passes these parameters and information to the QMI bus service in the kernel layer via the QSEE SSCconnection. The QMI bus service then passes these parameters and information to Sensorhub. After Sensorhub parses the SUID4, it passes the parameters and information to the initialization service module in the SOIS driver.

[0142] S3, the initialization service module in the SOIS driver caches target configuration parameters and target cameraindex information.

[0143] S4, after caching the target configuration parameters and target cameraindex information, the initialization service module can send the initialization completion information back to the OIS node through the QMI bus service and NCS.

[0144] After receiving the initialization completion information (also known as initialization success information), the OIS node can determine that the SOIS driver has completed initialization. Furthermore, it can send various control commands to the SOIS driver via the NCS and QMI bus services according to pre-configured rules. For example, the pre-configured rules include sending an exit command to the SOIS driver when both the camera mode and the target camera module change. This exit command may instruct that when the OIS node on the AP side is destroyed, the corresponding SOIS connection on the Sensorhub side is also destroyed, and the SOIS driver stops processing OIS-related matters. As another example, the pre-configured rules include sending a non-exit command to the SOIS driver when the camera mode changes but the target camera module remains unchanged. This non-exit command may instruct that when the OIS node on the AP side is destroyed, the corresponding SOIS connection on the Sensorhub side is retained, and the SOIS driver continues processing OIS-related matters. Specific details are described in subsequent embodiments.

[0145] In some embodiments, after initializing the service module to cache target configuration parameters and target cameraindex information, the SOIS driver can enable the jitter data acquisition module to execute the process of acquiring high-precision gyroscope data and acceleration data from the A+G device. Understandably, after the SOIS driver completes the initialization process, when enabling the jitter data acquisition module for the first time, it is necessary to initialize the channel interface corresponding to the high-frequency sampling channel of the A+G device to enable the SOIS driver to acquire high-precision gyroscope data and acceleration data from the A+G device.

[0146] In some embodiments, after the initialization service module stores the target configuration parameters and target cameraindex information, the SOIS driver can also enable the image stabilization module identification module. This module determines the camera module requiring image stabilization based on the target cameraindex information from the OIS node. It is understood that the camera module requiring image stabilization is one of the target camera modules. For example, in the current camera mode, when only one target camera module is acquiring images, the camera module requiring image stabilization is that target camera module. As another example, in the current camera mode, when multiple target camera modules are acquiring images simultaneously, the camera module requiring image stabilization is the main camera module among the target camera modules.

[0147] In an exemplary scenario, the rear main camera module corresponds to cameraindex1, the rear telephoto camera module corresponds to cameraindex2, and the rear wide-angle camera module corresponds to cameraindex3. When the target cameraindex information is cameraindex1, the image stabilization module identification module can determine that the rear main camera module is the one requiring image stabilization.

[0148] After identifying the camera module requiring image stabilization, the SOIS driver initializes the OIS-related components (such as the OIS motor and Hall IC) in the camera sensor based on the target configuration parameters, and controls the camera module to center the lens. Then, the shake data acquisition module in the SOIS driver receives high-precision gyroscope and acceleration data (also known as high-frequency sampled shake data) from the A+G device. The OIS control module calculates the offset coordinates of the OIS motor based on this high-precision gyroscope and acceleration data. The OIS control module sends the OIS motor's offset coordinates to the camera module requiring stabilization, and the VCM Driver corresponding to the OIS motor in the camera module controls the movement of the OIS motor's motor control axis. During the movement of the motor control axis, the Hall IC can detect the Hall IC position of the motor control axis in real time and feed it back to the VCM Driver. Thus, after the motor control axis reaches the Hall IC position indicated by the offset coordinates, the VCM Driver controls the OIS motor's motor control axis to stop moving. Through the above process, the physical position of the lens or image sensor is adjusted by the OIS motor to counteract the displacement caused by shaking.

[0149] During the above process, the OIS control module can control the OIS-related devices in the camera module requiring image stabilization (e.g., the rear main camera module) via the I2C bus. Understandably, if the camera module requiring image stabilization changes from the rear main camera module to the rear telephoto (or rear wide-angle) camera module during operation, the OIS control module can also control the OIS-related devices in the rear telephoto (or rear wide-angle) camera module via the I2C bus.

[0150] In other embodiments, if the OIS node determines that SOIS is not currently supported, it can also call the `setupSOISlink` function to establish a SOIS connection, initialize the SOIS driver, and control lens centering. Unlike the scenario where SOIS is currently supported, after lens centering, a process to destroy the OIS node is initiated, also known as initiating a destruction process. Optionally, in the above destruction process, the OIS node on the AP side can be destroyed, for example, by releasing the system resources occupied by the OIS node. Optionally, in the above destruction process, the SOIS driver can control OIS-related devices to stop working, for example, by controlling the OIS motor to stop moving and controlling the Hall IC to stop detection. Optionally, in the above destruction process, the SOIS driver can also destroy the SOIS connection. For example, releasing the memory resources occupied by the data interaction channel corresponding to the SOIS connection in Sensorhub and releasing the port resources of the QMI bus allocated to the data interaction channel. Afterwards, the SOIS driver stops receiving high-precision gyroscope data and acceleration data from the A+G device and also stops controlling the OIS motor in the camera module. In other embodiments, the SOIS driver may continue to receive high-precision gyroscope data and acceleration data, but the SOIS driver will not continue to process the gyroscope data and acceleration data, nor will it control the camera module to perform optical image stabilization based on the gyroscope data and acceleration data. In other words, it will not handle matters related to optical image stabilization.

[0151] In some embodiments, after an electronic device opens a camera application and starts a camera service, the camera application can also configure the acquisition frequency of preview images.

[0152] As one implementation, the camera application can call `setrepeatingrequanest` in the camera service to set the preview image acquisition frequency. Then, `setrepeatingrequanest` can periodically send preview image acquisition requests to the target camera module that actually performs image acquisition according to the acquisition frequency. In some embodiments, after the electronic device opens the camera application and starts the camera service, the camera application can also send the identifier of the current camera mode to the camera service. The camera mode enabled after opening the camera application can be, but is not limited to, normal photo mode, bokeh mode (e.g., portrait mode), or video recording mode, etc. This application embodiment does not specifically limit this. Furthermore, normal photo mode can also be called photo mode, which is a camera mode used to capture photos or moving images. In the following embodiments, the description mainly takes enabling normal photo mode after opening the camera application as an example.

[0153] As one implementation method, such as Figure 9As shown, the camera application can send the identifier of the current camera mode to the camera3Outputstream in the camera service. The camera3Outputstream then writes the received camera mode into the scene metadata. For example, after receiving the identifier of the normal shooting mode, the camera3Outputstream can write the normal shooting mode identifier into the scene metadata.

[0154] Additionally, when the enabled camera mode changes during camera application operation, the camera application can send the identifier of the changed camera mode to camera3Outputstream, which then writes it into the scene metadata. For example, in response to a user action, the camera application switches the enabled camera mode from normal shooting mode to bokeh mode. The camera application sends the bokeh mode identifier to camera3Outputstream, which then writes the bokeh mode identifier into the scene metadata.

[0155] In some embodiments, the platform algorithm module in CHI-CDK can periodically read the camera mode identifier from the scene metadata. After reading the camera mode identifier, the platform algorithm module can determine whether SOIS is currently supported based on the camera mode identifier.

[0156] Understandably, supporting SOIS means that in the current camera mode, the camera module can be controlled for optical image stabilization via the SOIS driver in Sensorhub. Not supporting SOIS means that the camera module cannot be controlled for optical image stabilization in the current camera mode.

[0157] For example, the platform algorithm module can determine whether a camera mode supports enabling SOIS each time it reads the camera mode identifier. As another example, when the platform algorithm module reads the camera mode identifier for the first time after the camera application is opened, it determines whether the initially read camera mode supports enabling SOIS. When the platform algorithm module reads the camera mode identifier for the Nth time after the camera application is opened, it compares whether the Nth read camera mode is the same as the (N-1)th read camera mode. If they are different, it determines whether the Nth read camera mode supports enabling SOIS. If they are the same, no further determination is made.

[0158] As one implementation method, in camera mode, when only one target camera module is acquiring images, it is determined that SOIS is supported, and the determination result is written into the platform metadata.

[0159] For example, in a scenario where single-camera mode is enabled, the platform algorithm module can determine that SOIS is currently supported and write an identifier indicating that SOIS can be enabled, such as ENABLE, OIS_ON_EIS_ON, or OIS_ON_EIS_OFF, into the platform metadata.

[0160] Here, (electronic anti-shake, EIS) refers to electronic image stabilization, and OIS_ON_EIS_ON indicates that either SOIS or EIS can be enabled. For example, when the camera is in video recording mode, the corresponding result could be OIS_ON_EIS_ON.

[0161] OIS_ON_EIS_OFF indicates that SOIS can be enabled, but EIS cannot. For example, when the camera mode is normal shooting mode, the corresponding result can be OIS_ON_EIS_OFF.

[0162] For example, in camera mode, when multiple target camera modules are acquiring images simultaneously, it is determined whether SOIS is currently supported based on whether the multi-camera algorithm configured in the electronic device supports dual-camera OIS.

[0163] Taking a scenario where dual-camera mode is enabled as an example, the target camera module includes a main camera module and an auxiliary camera module, both of which acquire images simultaneously. When the dual-camera algorithm meets condition 1, the platform algorithm module can determine that the dual-camera algorithm configured in the electronic device does not support dual-camera OIS, and therefore, it can determine that enabling SOIS is currently not supported.

[0164] Among them, condition 1: the distance between the lenses of the main camera module and the auxiliary camera module is calibrated in the dual-camera algorithm, and the center distance between the lenses of the main camera module and the auxiliary camera module is restricted to not change.

[0165] After determining that enabling SOIS is not supported, the platform algorithm module writes a flag indicating that SOIS cannot be enabled, such as "DISBALE" or OIS_OFF, into the platform metadata. For example, when the camera mode is a bokeh mode (e.g., portrait mode), the corresponding determination result could be OIS_OFF.

[0166] Taking the scenario of enabling dual-camera mode as an example, when the dual-camera algorithm meets condition 2, the judgment platform algorithm module can determine that the dual-camera algorithm configured in the electronic device supports dual-camera OIS, and can determine that SOIS is currently supported.

[0167] Among them, condition 2 includes: (1) The dual-camera algorithm calibrates that the main camera module supports OIS. When the dual-camera algorithm fuses the images acquired by the main camera module and the auxiliary camera module, it is necessary to obtain the HallIC position corresponding to the OIS motor in the main camera module. (2) The dual-camera algorithm also calibrates the maximum and minimum center distance between the lenses of the main camera module and the auxiliary camera module, that is, the distance between the lenses of the main camera module and the auxiliary camera module is variable.

[0168] After confirming that SOIS is enabled, the platform algorithm module writes an identifier indicating that SOIS can be enabled, such as "ENABLE", into the platform metadata.

[0169] As another implementation, electronic devices can also pre-configure a type mapping relationship between camera mode types and whether SOIS is enabled. Supporting SOIS means that in this type of camera mode, the camera module can be controlled for optical image stabilization via the SOIS driver in Sensorhub. Not supporting SOIS means that in this type of camera mode, the camera module cannot be controlled for optical image stabilization.

[0170] For example, in the type mapping relationship, "normal shooting mode" corresponds to "supports SOIS enabled", "bokeh mode" corresponds to "does not support SOIS enabled", and "video recording mode" corresponds to "supports SOIS enabled". When the current camera mode is normal shooting mode and bokeh mode is not enabled, it can be determined that SOIS is currently supported; when the current camera mode is bokeh mode enabled, it can be determined that SOIS is currently not supported; when the current camera mode is video recording mode, it can be determined that SOIS is currently supported. Understandably, after an electronic device opens the camera application and starts the camera service, the camera application can asynchronously or synchronously call configurestream, setrepeatingrequanest, and camera3Outputstream. The asynchronous calls mentioned above refer to calling multiple functions in non-blocking mode, that is, when calling the next function, there is no need to wait for the result of the previously called function. In the case of asynchronous calls to configurestream, setrepeatingrequanest, and camera3Outputstream, stream allocation, configuring the acquisition frequency of the preview stream image, and recording the current camera mode can be performed in parallel. The above synchronous call refers to calling multiple functions in blocking mode. That is, when calling the next function, it is necessary to wait for the result of the previously called function. When synchronously calling configurestream, setrepeatingrequanest and camera3Outputstream, the stream allocation, the acquisition frequency of the preview stream image is configured, and the current camera mode is recorded in sequence.

[0171] Understandably, only after the camera application writes the current camera mode to the camera3Outputstream can the platform algorithm module in CHI-CDK determine whether the current camera mode supports enabling SOIS and write the determination result to the platform metadata. After the camera application calls configurestream, as in the aforementioned embodiment... Figure 7 As shown, the sensor node can create the OIS node, which reads the platform metadata to obtain the judgment results written in the platform metadata.

[0172] When the camera application simultaneously executes the call to configurestream and writes the current camera mode identifier to camera3Outputstream, there is no necessary order between the time when the platform algorithm module writes the judgment result to the platform metadata and the time when the OIS node reads the platform metadata.

[0173] If the judgment result has not yet been written into the platform metadata, that is, if the platform algorithm module has not yet determined whether the current scenario supports enabling SOIS, the platform metadata read by the OIS node will not contain the judgment result. Accordingly, the OIS node can read the platform metadata again after a preset interval.

[0174] If the platform metadata read by the OIS node includes the identifier "DISBALE" (an example of a judgment result), the OIS node can determine that enabling SOIS is not currently supported.

[0175] If the platform metadata read by the OIS node includes the identifier "ENABLE" (another example of a judgment result), the OIS node can determine that SOIS is currently supported. Figure 10 As shown, after the target camera module is initialized and the lens is centered, Sensorhub can receive high-precision gyroscope and acceleration data from the A+G device. Additionally, Sensorhub can also receive low-precision gyroscope and acceleration data from the low-frequency sampling channel of the A+G device via other corresponding driver channel interfaces.

[0176] like Figure 10 As shown, the OIS control module in the SOIS driver can calculate the displacement deviation based on high-precision gyroscope and acceleration data. Then, the OIS control module can control the OIS motor in the camera module requiring image stabilization to move according to the displacement deviation, achieving image stabilization for the target camera module. Detailed process can be found in the aforementioned embodiments. Optionally, the displacement deviation can be an offset in an initial coordinate system, which can be a three-dimensional coordinate system established with the center of the lens after the camera module's lens is centered. For example, the displacement deviation includes the deviation in the X-axis direction, the deviation in the Y-axis direction, and the deviation in the Z-axis direction in the initial coordinate system.

[0177] Additionally, during the SOIS-driven control of the camera module requiring image stabilization, the camera HAL on the AP side can send control commands to the camera module requiring image stabilization via Sensorhub. For example... Figure 11A and Figure 11BAs shown, the enabled camera mode can change during camera application operation. After the enabled camera mode changes, the camera application can send the identifier of the changed camera mode to camera3Outputstream, which then writes it into the scene metadata. The platform algorithm module can read the camera mode identifier from the scene metadata. If the camera mode read this time is different from the previously read camera mode, it can determine that the camera mode has changed. In the case of a camera mode change, the platform algorithm module can determine whether the target camera module is the same before and after the camera mode change.

[0178] For example, in normal photo mode, the target camera module includes the rear main camera module; in portrait mode, the target camera module includes the rear main camera module and the rear wide-angle camera module; and in video mode, the target camera module includes the rear main camera module.

[0179] Continuing the previous example, in response to user input, the camera app changes its camera mode from normal photo mode to portrait mode. In this scenario, before and after the camera mode change, the target camera module changes from the rear main camera module to both the rear main camera module and the rear wide-angle camera module. That is, the platform's algorithm module can determine that the target camera module differs before and after the camera mode change.

[0180] like Figure 11A As shown, when the platform algorithm module determines that the target camera module is different before and after the camera mode change, the platform algorithm module can write a power-down flag into the platform metadata. Optionally, the OIS node can periodically read the platform metadata. If the read platform metadata contains a power-down flag, the SOIS client in the OIS node can send an exit command to the SOIS driver in Sensorhub through the NCS and QMI bus services. Optionally, in response to the exit command, the SOIS driver controls the OIS motor to stop working after centering the lens, and controls the Hall IC to stop detecting the position parameters of the motor control axis. Optionally, the SOIS driver can also destroy the SOIS connection, for example, by releasing the memory resources occupied by the data interaction path between the OIS node and the SOIS driver in Sensorhub and the port resources of the QMI bus. Optionally, the SOIS driver stops processing OIS-related matters, such as stopping the calculation of the displacement offset corresponding to the OIS motor.

[0181] Continuing the previous example, in response to a user action, the camera app changes its camera mode from normal photo mode to video mode. In this scenario, the target camera module before and after the camera mode change is the rear main camera module. That is, the platform's algorithm module can determine that the target camera module remains the same before and after the camera mode change.

[0182] like Figure 11B As shown, if the platform algorithm module determines that the target camera module is the same before and after the camera mode switch, it can write a non-power-loss flag into the platform metadata. The OIS node can periodically read the platform metadata. If the read platform metadata contains a non-power-loss flag, the SOIS client in the OIS node can send a non-exit command to Sensorhub via the NCS and QMI bus services. After receiving the non-exit command, the SOIS driver maintains its current working state and continues to process OIS-related matters, such as receiving high-precision gyroscope and acceleration data from the A+G device, calculating the displacement offset based on the high-precision gyroscope and acceleration data, and sending it to the OIS motor in the camera module requiring image stabilization. Additionally, the OIS motor will continue to operate; for example, after receiving the displacement offset, the OIS motor can adjust the physical position of the lens or image sensor according to the displacement offset. HallIC can also continue position detection, etc.

[0183] In short, such as Figure 12 As shown, after the Sensor node is created, initial camera parameters can be configured and added to the target camera module. Additionally, one or more child nodes can be created, such as AF motor nodes, OIS nodes, and flash nodes. The AF motor node controls the AF motor in the target camera module, the OIS node controls the OIS-related devices in the target camera module, and the flash node controls the flash in the target camera module.

[0184] For example, after the OIS node is created, it can be initialized. The OIS node initialization process is as follows: The OIS node establishes an NCS connection and calls the NCS to send data indicating the registration of the SOIS channel to Sensorhub, such as the channel establishment information mentioned in the previous embodiment. The OIS node packages the initialization data (e.g., target configuration parameters) and sends it to Sensorhub through the NCS. The OIS node checks whether the current version is a burn-in version. If so, it proceeds with the production line process. If not, the OIS node checks whether SOIS is currently supported. As one implementation, the method for checking whether SOIS is currently supported can be to read the judgment result from the platform metadata of CHI-CDK, and determine whether SOIS is supported based on the judgment result. For example, the platform algorithm module in CHI-CDK can determine whether SOIS is currently supported based on the camera mode used by the camera application, and write the judgment result to the platform metadata. The judgment result is OIS_OFF(0), indicating that SOIS is not supported. For example, when the camera mode is a bokeh mode, the judgment result is OIS_OFF(0). The result of the judgment is OIS_ON_EIS_OFF(1), which means that SOIS is supported, but electronic image stabilization (EIS) is not supported. For example, when the camera mode is normal shooting mode, the result of the judgment is OIS_ON_EIS_OFF(1). The result of the judgment is OIS_ON_EIS_ON(3), which means that SOIS is supported and EIS is also supported. For example, when the camera mode is video recording mode, the result of the judgment is OIS_ON_EIS_ON(3). When the result of the judgment in the platform metadata is OIS_ON_EIS_OFF(1) or OIS_ON_EIS_ON(3), the OIS node determines that SOIS is currently supported.

[0185] Additionally, during OIS initialization, if the judgment result recorded in the platform metadata indicates that SOIS is currently supported, the judgment result is fed back to the OIS node through the OIS node pointer. If the judgment result recorded in the platform metadata indicates that SOIS is currently not supported, the destruction process is initiated to destroy the OIS node.

[0186] like Figure 12 As shown, after detecting whether SOIS is currently supported, the OIS node can also configure the calculation factors corresponding to the OIS-related devices into the NCS. These calculation factors are used to process the data collected by the Hall IC.

[0187] After creating various child nodes, the Sensor node can also configure parameters in the target camera module. The Sensor node can also package the Sensor node frame processing flow, the OIS node frame processing flow, and information configuring OIS functionality, and then pass this data to Sensorhub via NCS. Sensorhub can process the data from the Sensor node according to the Sensor node frame processing flow. Sensorhub can also process the data from the OIS node according to the OIS node frame processing flow.

[0188] After the OIS function is enabled, such as Figure 12 As shown, Sensorhub performs driver initialization. For example, it initializes the driver for the A+G device, and also initializes the SOIS driver. During SOIS driver initialization, the SOIS driver can initialize the channel interfaces corresponding to the high-frequency sampling channels of the A+G device. Afterwards, the SOIS driver can receive high-precision gyroscope and acceleration data from the A+G device. The SOIS driver can identify the camera module requiring image stabilization and, based on the high-precision gyroscope and acceleration data, control the OIS-related devices in that camera module to perform image stabilization.

[0189] The implementation details of the method provided in the embodiments of this application are described below in conjunction with practical application scenarios.

[0190] In some embodiments, such as Figure 13 As shown, after the electronic device is unlocked, the main interface 1301 can be displayed. This main interface 1301 includes an application icon 1302 for the camera application. While the electronic device displays the main interface 1301, it can receive user actions on the application icon 1302. In response to user actions on the application icon 1302, the electronic device can display a shooting preview interface 1303. Understandably, when the camera application is running in the foreground, a default camera mode can be enabled. For example, if the default camera mode is normal shooting mode, the electronic device will enable normal shooting mode by default after opening the camera application. The mode indicator 1304 in the shooting preview interface 1303, indicating normal shooting mode, is selected.

[0191] After the electronic device detects that the user has clicked the application icon 1302, the electronic device can execute the method provided in the embodiments of this application. For example... Figure 14 As shown, the electronic device includes a camera application, camera services, camera HAL, and Sensorhub. It is understood that the electronic device may also include other... Figure 14 Software modules not shown in the diagram. For example... Figure 14 As shown, when the electronic device executes the method provided in the embodiments of this application, the interaction between the various software and hardware modules is as follows:

[0192] S101 launches the camera service during the camera application startup process.

[0193] In some embodiments, the electronic device can launch the camera application based on a user instruction to open the camera application. This instruction to open the camera application may be an action that triggers the camera application to run in the foreground. For example, the instruction to open the camera application may be an action by the user clicking the application icon 1302. For instance, if the desktop manager receives a first reporting event, it can trigger the launch of the camera application, wherein the first reporting event indicates that the display location of the application icon 1302 has been clicked.

[0194] As another example, the above-mentioned instruction to open the camera application can also be a voice command from the user to open the camera application. This application does not specifically limit the form of the instruction to open the camera application.

[0195] S102, the camera service sends a driver registration instruction to the NCS in the camera HAL.

[0196] S103, NCS responds to the driver registration command by obtaining the SUID of various sensors from Sensorhub and generating the corresponding Sensor list.

[0197] In some embodiments, S104 is executed after the Sensor list is generated.

[0198] S104, NCS sends the initialization parameters corresponding to each type of sensor to Sensorhub in sequence according to the Sensor list.

[0199] The initialization parameters sent do not include initialization parameters for OIS-related devices.

[0200] S105, Sensorhub stores the received initialization parameters.

[0201] S106, Sensorhub sends a message to NCS indicating successful registration.

[0202] Understandably, steps S104 to S106 are executed cyclically. After the NCS sends the initialization parameters of one sensor to Sensorhub, it waits for Sensorhub to return a message indicating successful registration. Upon receiving the message, it sends the initialization parameters of the next sensor to Sensorhub, and so on, until the initialization parameters for all sensors in the Sensor list (excluding OIS-related devices) have been sent to Sensorhub.

[0203] In some embodiments, the implementation details of S102 to S106 described above can be found in [reference]. Figure 5 The specific details of the NCS initialization process shown will not be elaborated here.

[0204] S107, the camera service calls the open function in the camera HAL.

[0205] In some embodiments, S107 and S102 are steps following S101, and there is no necessary order between them.

[0206] S108, the camera HAL is turned on and connected to the first module.

[0207] Understandably, the first module is the target camera module corresponding to the current camera mode. The current camera mode is the camera mode enabled after opening the camera application, also known as the first camera mode. In different operating scenarios, the first camera module can use different camera modes, such as any one of normal photo mode, portrait mode (blur mode), and video recording mode. This application embodiment does not specifically limit this.

[0208] For example, such as Figure 13 As shown, after opening the camera application, the normal shooting mode is enabled. The target camera module corresponding to the normal shooting mode is the rear main camera module. The first module can be the rear main camera module.

[0209] S109, Camera HAL acquires camera parameters from the first module.

[0210] In some embodiments, S109 described above may involve acquiring camera parameters of all logical cameras in the electronic device. It is understood that a logical camera may consist of one or more physical camera modules, and all logical cameras include logical cameras composed of a first module. Correspondingly, the camera parameters of all logical cameras include the camera parameters corresponding to the first module.

[0211] S110, the camera HAL returns the camera parameters of the first module to the camera service.

[0212] In some embodiments, S110 described above may involve sending the camera parameters of all logical cameras to the camera service. During this process, the camera parameters of the first module may also be sent to the camera service.

[0213] As one implementation method, the implementation details of S107-S110 above can be found in [reference]. Figure 6 This will not be elaborated upon here.

[0214] S111, the camera service triggers the camera HAL to create the Sensor node corresponding to the first module.

[0215] In some embodiments, the camera service can send the camera parameters corresponding to the first module to the camera HAL, triggering the camera HAL to create a Sensor node. After the Sensor node is created, in addition to executing S112, the Sensor node can also pass the camera parameters of the first module to the target camera module to achieve initialization for the first module.

[0216] S112, the camera HAL creates the Sensor node corresponding to the first module.

[0217] S113, the Sensor node creates the OIS node corresponding to the first module.

[0218] As one implementation method, the implementation details of S111-S113 above can be found in [reference]. Figure 7 This will not be elaborated upon here.

[0219] S114, the camera application sends the identifier of the first camera mode to the camera service, instructing the camera service to write the scene metadata.

[0220] The first camera mode is the camera mode enabled after opening the camera application, as detailed in the aforementioned embodiments, and will not be repeated here.

[0221] In some embodiments, S114 is a step following S111, executed asynchronously with S112, and there is no necessary order between them. After S112 and S113 are executed, S117 is executed.

[0222] S115, the platform algorithm module in CHI-CDK reads the identifier of the first camera mode from the scene metadata.

[0223] In some embodiments, the above-described S115 may be executed periodically, and the platform algorithm module may obtain the first camera mode enabled by the camera application from the scene metadata, and detect whether the camera mode enabled by the camera application has changed.

[0224] S116, the platform algorithm module in CHI-CDK identifies whether SOIS is currently supported based on the first camera mode, and writes the judgment result into the platform metadata.

[0225] In some embodiments, the platform algorithm module in CHI-CDK can determine whether SOIS is supported in the first camera mode based on the number of target camera modules and the camera algorithm type for acquiring images in the first camera mode, and write the determination result into the platform metadata.

[0226] In other embodiments, the platform algorithm module in CHI-CDK can determine whether SOIS is supported in the first camera mode based on the type of the first camera mode and in combination with the pre-configured type mapping relationship.

[0227] For example, a judgment result of OIS_OFF(0) indicates that SOIS can be enabled. For instance, when the first camera mode is bokeh mode, the judgment result is OIS_OFF(0). A judgment result of OIS_ON_EIS_OFF(1) indicates that SOIS can be enabled but electronic image stabilization (EIS) is disabled. For instance, when the first camera mode is normal shooting mode, the judgment result is OIS_ON_EIS_OFF(1). A judgment result of OIS_ON_EIS_ON(3) indicates that either SOIS or EIS can be enabled. For instance, when the first camera mode is video recording mode, the judgment result is OIS_ON_EIS_ON(3).

[0228] In addition, the platform algorithm module can also determine the camera modules in the first module that need image stabilization based on the first camera mode, and write the first target cameraindex information of the camera modules that need image stabilization into the platform metadata.

[0229] As one implementation method, the implementation details of S114 to S116 can be found in [reference]. Figure 9 This will not be elaborated upon here.

[0230] S117, the OIS node reads the judgment result recorded in the platform's metadata.

[0231] In some embodiments, S116 and S117 are executed asynchronously. If S117 is executed before S116, the OIS node cannot read the judgment result from the platform metadata. In this case, S117 can be executed again after a preset time interval until the judgment result is read from the platform metadata, and then S118 is executed.

[0232] S118, the OIS node controls the NCS to start and establish an SOIS connection.

[0233] In some embodiments, the OIS node can call the setupSOISlink function to control the NCS to start and establish an SOIS connection.

[0234] S119, NCS queries the sensor list for the corresponding Suid4 for the OIS-related device.

[0235] S120, NCS sends channel establishment information to Sensorhub, which includes Suid4.

[0236] S121, the SOIS driver in Sensorhub responds to the channel establishment information and establishes an SOIS connection.

[0237] S122, the SOIS driver in Sensorhub sends feedback information that the SOIS connection has been established to the OIS node through NCS.

[0238] As one implementation method, the implementation details of S117-S122 above can be found in [reference]. Figure 7 This will not be elaborated upon here.

[0239] S123, the OIS node obtains the first configuration parameters and the first target cameraindex information from CHI-CDK.

[0240] The first configuration parameter is the target configuration parameter corresponding to the first module, which includes the initialization parameters of the OIS-related devices (OIS motor and Hall IC) in the first module.

[0241] In some embodiments, the OIS node can obtain the first target cameraindex information based on the platform metadata from CHI-CDK.

[0242] S124, the OIS node sends the first configuration parameters and the first target camera index information to the SOIS driver in Sensorhub through NCS.

[0243] In some embodiments, the OIS node can call the sendSOIScmd function to send the first configuration parameters and the first target cameraindex information to Sensorhub via the NCS. The process of sending data to Sensorhub via the NCS can be found in S119 and S120, and will not be elaborated here. Similarly, the process of Sensorhub sending data to the OIS node via the NCS can be found in S122, and will not be elaborated here either.

[0244] S125, the SOIS driver in Sensorhub stores the first configuration parameters and the first target cameraindex information.

[0245] In some embodiments, Sensorhub can pass the first configuration parameters and the first target camera index information to the SOIS driver, and the SOIS driver stores the first configuration parameters and the first target camera index information.

[0246] After storing the first configuration parameters and the first target cameraindex information, in addition to executing S126, the SOIS driver can also receive gyroscope data and acceleration data. Based on the gyroscope data and acceleration data, it controls the OIS-related devices of the camera module that requires image stabilization. For specific implementation details, please refer to [link / reference needed]. Figure 8 This will not be elaborated upon here.

[0247] S126, the SOIS driver in Sensorhub sends feedback information indicating that initialization is complete to the OIS node through NCS.

[0248] As one implementation method, the implementation details of S123-S126 above can be found in [reference]. Figure 8 This will not be elaborated upon here.

[0249] S127, the OIS node sends a lens centering command to the SOIS driver in Sensorhub via NCS.

[0250] S128, the SOIS drive in Sensorhub controls the lens in the camera module indicated by the first target cameraindex information to move to the pre-configured centering position.

[0251] S129, the SOIS driver in Sensorhub feeds back centering completion information to the OIS node through NCS.

[0252] S130, if the judgment result recorded in the platform metadata indicates that SOIS is not supported, the destruction of OIS nodes will begin.

[0253] In some embodiments, the implementation details of destroying the OIS node can be referred to the destruction process in the foregoing embodiments, and will not be repeated here. Optionally, in the destruction process triggered by the OIS node based on the judgment result, in addition to destroying the OIS node on the AP side, the SOIS connection on the Sensorhub side can also be destroyed. At the same time, the SOIS driver will also stop processing OIS-related matters.

[0254] Additionally, S130 is the step executed when the platform metadata indicates that SOIS is not currently supported. If the platform metadata indicates that SOIS is currently supported, then S130 is not executed.

[0255] S131, the camera service sends a stream start command to the sensor node.

[0256] S132, the sensor node controls the first module to acquire images.

[0257] In some embodiments, S131, S111, and S114 are all steps following S101. There is no necessary order between S111, S114, and S131, and they can be executed asynchronously. This application does not specifically limit this.

[0258] In some embodiments, such as Figure 15 As shown, the shooting preview interface 1303 includes mode icons indicating various camera modes, such as mode icon 1304 indicating normal photo mode, mode icon 1501 indicating video recording mode, and mode icon 1502 indicating portrait mode. Mode icon 1304 in the shooting preview interface 1303 is selected.

[0259] During the display of the shooting preview interface 1303, the shooting preview interface 1503 is displayed in response to the user clicking the mode identifier 1501. In the shooting preview interface 1503, the mode identifier 1501 is selected. During this process, the camera mode activated by the camera application changes from normal photo mode to video mode.

[0260] During the display of the shooting preview interface 1303, in response to the user clicking the mode identifier 1502, the shooting preview interface 1504 is displayed. In the shooting preview interface 1504, the mode identifier 1502 is selected. During this process, the camera mode activated by the camera application changes from normal shooting mode to portrait mode.

[0261] The following example illustrates the method provided in this application embodiment for scenarios where the camera mode changes from the second camera mode (e.g., normal photo mode) to the third camera mode (e.g., portrait mode or video recording mode).

[0262] Understandably, in second camera mode, the target camera module to be enabled is the second module, and in third camera mode, the target camera module to be enabled is the third module. If the third camera mode is video recording mode, the second and third modules are the same, both being the rear main camera module. If the third camera mode is portrait mode, the second and third modules are different; the second module is the rear main camera module, and the third module includes both the rear main camera module and the rear wide-angle camera module.

[0263] like Figure 16 As shown, when the third camera mode is recording mode, the interaction between the various software and hardware modules is as follows when the electronic device executes the method provided in the embodiments of this application:

[0264] S201, In the scenario where the camera mode changes from the second camera mode to the third camera mode, the camera application sends the identifier of the third camera mode to the camera service.

[0265] S202, the camera service writes the identifier of the third camera mode into the scene metadata.

[0266] S203, the platform algorithm module in CHI-CDK reads the identifier of the third camera mode from the scene metadata.

[0267] S204, the platform algorithm module in CHI-CDK determines whether the third module corresponding to the third camera mode is the same as the second module corresponding to the second camera mode, and determines whether SOIS is supported in the third camera mode.

[0268] In some embodiments, the CHI-CDK is configured with a module correspondence between each camera mode and the target camera module. When the camera mode identifiers read from the scene metadata twice consecutively by the platform algorithm module are different—for example, the identifier of the second camera mode was read previously, and the identifier of the third camera mode is read this time—the platform algorithm module determines the second module corresponding to the second camera mode and the third module corresponding to the third camera mode based on the aforementioned module correspondence. The platform algorithm module then determines whether the second module and the third module are the same.

[0269] For example, if the number of second modules is the same as the number of third modules, and the types of second modules and third modules are also the same, then the second module and the third module are determined to be the same. For instance, if the second camera mode is a normal photo mode, and the target camera module for the normal photo mode includes the rear main camera module, and the third camera mode is a video recording mode, and the target camera module for the video recording mode includes the rear main camera module, then after the camera mode changes from the second camera mode to the third camera mode, it can be determined that the second module and the third module are the same, and S205 is executed.

[0270] S205, when the second module and the third module are the same, the platform algorithm module in CHI-CDK writes a non-power-loss flag into the platform metadata, and writes the judgment result of whether SOIS is supported into the platform metadata.

[0271] S206, the camera service sends a destroy command to the camera HAL.

[0272] In some embodiments, there is no necessary order between S206 and S202. Understandably, after a camera mode change, it is necessary to destroy the Sensor node, OIS node, etc., corresponding to the original target camera module.

[0273] The aforementioned destruction command can trigger the destruction of the sensor node corresponding to the second module. As one implementation, the camera service can call the `close` function to send a destruction command to the camera HAL.

[0274] In addition, during the process of destroying the sensor node corresponding to the second module, the process of destroying the OIS node can also be executed, that is, S207 is executed.

[0275] S207, Camera HAL begins destroying OIS nodes.

[0276] In some embodiments, the camera HAL responds to a destruction command from the camera service by calling the destroy function to destroy the Sensor node corresponding to the first module. If an OIS node still exists in the camera HAL, the destruction of the OIS node begins. After the destruction of the OIS node begins, step S208 is executed.

[0277] S208, the OIS node reads the non-power-loss flag from the CHI-CDK platform metadata.

[0278] S209, the OIS node sends a non-exit command to Sensorhub via NCS.

[0279] The aforementioned non-exit instruction indicates that during the process of destroying the OIS node on the AP side, the SOIS connection is maintained on the Sensorhub side, and the SOIS driver continues to handle OIS-related matters.

[0280] S210, the SOIS driver in Sensorhub continues to handle OIS-related matters.

[0281] In some embodiments, in response to a non-exit command, the destruction of the SOIS connection on the Sensorhub side is skipped during the destruction of the OIS node, and the SOIS driver continues to handle OIS-related matters. This way, even after the OIS node on the AP side is destroyed, the SOIS driver can continue to acquire high-precision gyroscope and acceleration data, and control the OIS-related devices of the camera modules requiring image stabilization in the second module based on the gyroscope and acceleration data. This avoids image shift after switching from the second camera mode to the third camera mode.

[0282] Optionally, after re-establishing the sensor node and OIS node corresponding to the third module, the OIS node of the third camera mode can directly use the original SOIS connection to interact with the SOIS driver on the Sensorhub side.

[0283] S211, the camera service sends a connection command to the camera HAL, which is used to indicate the connection to the third module.

[0284] In some embodiments, when the camera service calls the close function, after the sensor node and OIS node are destroyed, the camera HAL can send a destruction completion message back to the camera service. In response to the destruction completion message, the camera service calls the open function in the camera HAL to send a connection command to the camera HAL.

[0285] S212, the camera HAL is turned on and connected to the third module.

[0286] S213, the camera service triggers the camera HAL to create the Sensor node corresponding to the third module.

[0287] S214, the camera HAL creates the Sensor node corresponding to the third module.

[0288] S215, the Sensor node creates the OIS node corresponding to the third module.

[0289] S216, the OIS node reads the judgment result recorded in the platform metadata.

[0290] S217, the OIS node controls the NCS to send channel establishment information to the SOIS driver in Sensorhub.

[0291] S218, the SOIS driver in Sensorhub discards channel establishment information.

[0292] Understandably, when destroying the OIS node of the second module, the SOIS connection on the Sensorhub side is not destroyed. After creating the OIS node of the third module, the OIS node of the third module can continue to use the original SOIS connection without having to execute the process of establishing a SOIS connection. Accordingly, the SOIS driver can discard the channel establishment information and execute S219.

[0293] S219, the SOIS driver in Sensorhub sends feedback to the OIS node via NCS that the SOIS connection has been established.

[0294] S220, the OIS node obtains the second configuration parameters and the second target cameraindex information.

[0295] In some embodiments, the second configuration parameter may be the target configuration parameter corresponding to the third module, and the second configuration parameter includes the initialization parameters of the OIS-related devices in the third module. The second target cameraindex information may be the target cameraindex information corresponding to the third module.

[0296] S221, the OIS node sends the second configuration parameters and the second target camera index information to the SOIS driver in Sensorhub through the NCS.

[0297] S222, the SOIS driver in Sensorhub discards the second configuration parameter and the second target cameraindex information.

[0298] Understandably, during the destruction of the OIS node of the second module, the SOIS driver retains the target configuration parameters and target camera index information corresponding to the second module. Furthermore, since the second configuration parameters are the same as the target configuration parameters of the second module, and the second target camera index information is also the same as the target camera index information of the second module, the SOIS driver can discard the received second configuration parameters and second target camera index information to avoid duplicate storage.

[0299] In a possible embodiment, S222 may also be an SOIS driver storing the second configuration parameters and the second target camera index information, replacing the target configuration parameters and target camera index information corresponding to the original second module.

[0300] S223, the SOIS driver in Sensorhub sends initialization completion information back to the OIS node through NCS.

[0301] S224, the OIS node sends a lens centering command to the SOIS driver in Sensorhub via NCS.

[0302] S225, SOIS driver in Sensorhub, discards lens centering command.

[0303] Understandably, after switching from the second camera mode to the third camera mode, if the lens of the camera module requiring image stabilization is moved to the center position in response to the lens centering command, the image within the lens's field of view will change significantly. In other words, image drift will occur before and after the camera mode change.

[0304] In this embodiment, during the switch from the second camera mode to the third camera mode, the SOIS driver continues to control the camera module requiring image stabilization to perform optical image stabilization. Additionally, the SOIS driver discards lens centering commands to ensure minimal changes in the image within the lens's field of view before and after the camera mode change; that is, there is no image drifting issue before and after the camera mode change. S226, the SOIS driver in Sensorhub feeds back centering completion information to the OIS node via NCS.

[0305] S227, if the judgment result recorded in the platform metadata indicates that SOIS is not supported, the OIS node is destroyed. It is understood that in the above embodiments, although the second and third modules are the same, the conditions for whether SOIS is supported in the second and third camera modes may differ or be related. If SOIS is not supported in the third camera mode, the OIS node of the third module needs to be destroyed to avoid consuming system resources.

[0306] In addition, the destruction of OIS nodes in S227 is different from the destruction of OIS nodes triggered by S207: it does not require querying whether the platform metadata contains a non-power-loss flag. During the destruction of OIS nodes, the SOIS connection on the Sensorhub side needs to be destroyed accordingly, and the SOIS driver also stops processing OIS-related matters.

[0307] In other embodiments, the response is used for operation, closing the camera application, or triggering the destruction of the Sensor node and OIS node. During this destruction process, it is not necessary to query whether the platform metadata contains a non-power-loss flag. During the destruction of the OIS node, the corresponding SOIS connection on the Sensorhub side needs to be destroyed, and the SOIS driver also stops processing OIS-related matters.

[0308] If the third camera mode supports camera modules, the process can skip executing S227.

[0309] S228, the camera service sends a stream start command to the sensor node.

[0310] S229, the sensor node controls the third module to acquire images.

[0311] In some embodiments, the implementation details of S211 to S229 can be found in S107 to S132, and will not be repeated here.

[0312] like Figure 17 As shown, when the third camera mode is portrait mode, the interaction between the various software and hardware modules is as follows when the electronic device executes the method provided in the embodiments of this application:

[0313] S301, In the scenario where the camera mode changes from the second camera mode to the third camera mode, the camera application sends the identifier of the third camera mode to the camera service.

[0314] S302, the camera service writes the identifier of the third camera mode into the scene metadata.

[0315] In S303, the platform algorithm module in CHI-CDK reads the identifier of the third camera mode from the scene metadata.

[0316] S304, the platform algorithm module in CHI-CDK determines whether the third module corresponding to the third camera mode is the same as the second module corresponding to the second camera mode, and determines whether SOIS is supported in the third camera mode.

[0317] For example, if the number of the second module is different from the number of the third module, or if the types of the second module and the third module are different, it is determined that the second module and the third module are different. For instance, if the second camera mode is a normal photo mode, and the target camera module for the normal photo mode includes the rear main camera module, and the third camera mode is a video recording mode, and the target camera module for the video recording mode includes the rear main camera module, after the camera mode changes from the second camera mode to the third camera mode, it can be determined that the second module and the third module are the same, and S305 is executed.

[0318] S305, when the second module and the third module are different, the platform algorithm module in CHI-CDK writes a power failure flag into the platform metadata, and writes the judgment result of whether SOIS is supported into the platform metadata.

[0319] S306, the camera service sends a destroy command to the camera HAL.

[0320] In some embodiments, there is no necessary order between S306 and S302.

[0321] S307, Camera HAL begins destroying OIS nodes.

[0322] S308, the OIS node reads the power failure flag from the CHI-CDK platform metadata.

[0323] S309, the OIS node sends an exit command to Sensorhub via NCS.

[0324] Specifically, the aforementioned exit instruction instructs that during the destruction of the OIS node on the AP side, the corresponding SOIS connection on the Sensorhub side is also destroyed, and the SOIS driver stops processing OIS-related matters. For specific implementation details, please refer to the destruction process mentioned in the previous embodiment. S310, the SOIS driver in Sensorhub stops processing OIS-related matters.

[0325] Optionally, in response to an exit command, the SOIS driver can control the OIS-related devices to stop working, such as controlling the OIS motor to stop moving or controlling the Hall IC to stop detection. Optionally, in response to an exit command, the SOIS driver can also destroy the SOIS connection. For example, releasing the memory resources occupied by the data interaction channel corresponding to the SOIS connection in Sensorhub and releasing the port resources of the QMI bus allocated to the data interaction channel. Optionally, in response to an exit command, the SOIS driver stops processing OIS-related matters. For example, the SOIS driver stops receiving high-precision gyroscope data and acceleration data from the A+G device, and also stops controlling the OIS motor in the camera module. S311, the camera service sends a connection command to the camera HAL, the connection command being used to indicate the connection to a third module.

[0326] S312, the camera HAL is turned on and connected to the third module.

[0327] S313, the camera service triggers the camera HAL to create the Sensor node corresponding to the third module.

[0328] S314, the camera HAL creates the Sensor node corresponding to the third module.

[0329] S315, the Sensor node creates the OIS node corresponding to the third module.

[0330] S316, the OIS node reads the judgment results recorded in the platform metadata.

[0331] S317, the OIS node controls the NCS to send channel establishment information to the SOIS driver in Sensorhub.

[0332] In some embodiments, the implementation details of S311 to S317 can be referred to S211 to S217, and will not be repeated here.

[0333] S318, the SOIS driver in Sensorhub establishes an SOIS connection in response to channel establishment information.

[0334] In some embodiments, the implementation details of S318 described above can be found in S121, and will not be repeated here.

[0335] S319, the SOIS driver in Sensorhub sends feedback to the OIS node via NCS that the SOIS connection has been established.

[0336] S320, the OIS node obtains the third configuration parameters and the third target cameraindex information.

[0337] In some embodiments, the third configuration parameter may be the target configuration parameter corresponding to the third module, and the third configuration parameter includes the initialization parameters of the OIS-related devices in the third module. The third target cameraindex information may be the target cameraindex information corresponding to the third module. S321, the OIS node sends the third configuration parameter and the third target cameraindex information to the SOIS driver in Sensorhub through NCS.

[0338] S322, the SOIS driver in Sensorhub stores the third configuration parameters and the third target cameraindex information.

[0339] S323, the SOIS driver in Sensorhub sends initialization completion information back to the OIS node through NCS.

[0340] S324, the OIS node sends a lens centering command to the SOIS driver in Sensorhub via NCS.

[0341] S325, the SOIS drive control in Sensorhub moves the lens in the camera module indicated by the third target cameraindex information to a pre-configured centering position.

[0342] S326, the SOIS driver in Sensorhub feeds back centering completion information to the OIS node through NCS.

[0343] S327, if the judgment result recorded in the platform metadata indicates that SOIS is not supported, the destruction of OIS nodes will begin.

[0344] If the third camera mode supports enabling SOIS, the above S327 will not be executed.

[0345] S328, the camera service sends a stream start command to the sensor node.

[0346] S329, the sensor node controls the third module to acquire images.

[0347] In some embodiments, the implementation details of S311 to S329 can be found in S107 to S132, and will not be repeated here.

[0348] In other embodiments, where the camera mode enabled by the camera application remains unchanged, the camera module requiring image stabilization can be switched in response to user operation. For example, when the camera application enables normal shooting mode, the target camera modules in normal shooting mode include the rear main camera module, the rear telephoto camera module, and the rear wide-angle camera module. In normal shooting mode, only one target camera module needs to capture images. For example, when normal shooting mode is activated, only the rear main camera module captures images, and the camera module requiring image stabilization is the rear main camera module. In response to the user's zoom operation, the camera module that needs to capture images is switched from the rear main camera module to the rear wide-angle (or rear telephoto) camera module. Accordingly, the camera module requiring image stabilization becomes the rear wide-angle (or rear telephoto) camera module.

[0349] like Figure 18 As shown, the rear main camera module in the electronic device includes a first pin 1801, which is connected to the Sensorhub. The rear wide-angle camera module in the electronic device includes a second pin 1802, which is connected to the Sensorhub.

[0350] When the rear main camera module is one that requires image stabilization, an oscilloscope can detect the transmission of data frames (or electrical signals) between the first pin 1801 and the Sensorhub. Optionally, the transmission frequency of these data frames is relatively high, and optionally, the transmission pattern of these data frames is consistent with the pattern of control commands sent by the Sensorhub when controlling the camera module to perform OIS image stabilization.

[0351] Subsequently, the camera module requiring image stabilization becomes the rear wide-angle camera module. An oscilloscope can detect data frame transmission between the second pin 1802 and the Sensorhub. Optionally, the transmission frequency of these data frames is relatively high; alternatively, the transmission pattern of these data frames is consistent with the pattern of control commands sent by the Sensorhub when controlling the camera module to perform OIS image stabilization.

[0352] Some embodiments of this application also provide an electronic device, which may include a memory and one or more processors. The memory and processors are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the electronic device in the above method embodiments.

[0353] In an exemplary scenario, when a camera application is running in the foreground on an electronic device, the first camera mode is activated. Understandably, the camera application can provide multiple camera modes, each corresponding to different shooting functions. For example, the normal photo mode, video recording mode, and portrait mode mentioned in the preceding embodiments, and other modes not mentioned in the preceding embodiments, such as night scene mode and time-lapse mode. When different camera modes are activated, the electronic device can respond to user operations and capture different types of image data with different visual effects, such as photos, animated images, night scene photos, time-lapse photos, and video data.

[0354] After activating the first mode, the electronic device can display a first preview interface for the camera application. This first preview interface includes image data captured by the first camera module. For example, the first preview interface could be the shooting preview interface 1303 in the aforementioned embodiments, and the first camera module could be the camera module requiring image stabilization within the target camera module corresponding to the first mode (e.g., normal shooting mode), such as the rear main camera module.

[0355] In the first instant, the electronic device receives the initial instruction to switch camera modes. For example, in Figure 15 In the scenario shown, the first operation could be a user clicking mode identifier 1501, where mode identifier 1501 is the mode identifier corresponding to the recording mode, which could be an example of the second mode. The first moment mentioned above can be any time at which the first operation is received. At a second moment after the first moment, the electronic device responds to the first operation by displaying a second preview interface. This second preview interface is the shooting preview interface in the second mode, for example... Figure 15 The shooting preview interface 1503 is shown in the image. The second preview interface includes image data captured by the first camera module. Furthermore, the second moment refers to the time after the first moment when the second preview interface containing the image is displayed. The time interval between the first and second moments is related to the time consumption of switching camera modes in the camera application. It is understood that the first mode and the second mode are two different camera modes identical to the target camera module. In other examples, the first mode may also be a video recording mode, and the second mode may be a normal photo taking mode. This application embodiment does not limit this.

[0356] Before the first moment, the camera application is in camera mode 1; after the second moment, it is in camera mode 2. Between the first and second moments, the camera application switches between camera modes. The process of switching camera modes can be found in related technologies and will not be elaborated here. In this embodiment, if the electronic device shakes between the first and second moments, that is, during the camera mode switching process, the electronic device can adjust the lens position in the first camera module to perform optical image stabilization.

[0357] In an exemplary scenario, if the target camera module remains unchanged before and after the camera mode switch, and the electronic device does not move during the camera mode switch, the position of the lens in the first camera module remains unchanged. That is, after the camera mode switch, the OIS-related components of the camera module are not initialized; for example, the lens is not moved to the center position via the OIS motor.

[0358] For example, at the third moment, the electronic device receives a second operation to switch camera modes. This second operation could be the electronic device displaying... Figure 15 When the shooting preview interface 1503 is displayed, the user clicks the mode icon 1304. The electronic device responds to this second operation, switching the camera mode from the second mode to the first mode. At the fourth moment, following the third moment, the first preview interface is displayed.

[0359] Before the third moment, the camera mode is in the second mode; after the fourth moment, the camera mode is in the first mode. Between the third and fourth moments, the camera application switches between camera modes. Between the third and fourth moments, assuming the electronic device remains stationary, the position of the lens in the first camera module remains unchanged. That is, after switching camera modes, the OIS-related components in the first camera module are not initialized; for example, the lens is not moved to a centered position.

[0360] In some embodiments, the electronic device further includes an access point (AP) and a first sensor (A+G device). After receiving the first operation, the AP sends a first control command to the electronic device, such as the non-exit command mentioned in the foregoing embodiments.

[0361] After receiving the first control command, Sensorhub continues to receive gyroscope data and / or acceleration data collected by the first sensor, and, based on the gyroscope data and / or acceleration data, controls the first camera module to perform optical image stabilization. Thus, between the first and second moments, if the electronic device shakes, the position of the lens in the first camera module will be adjusted. If the electronic device does not shake, the position of the lens in the first camera module will not be adjusted.

[0362] In an exemplary scenario, the electronic device receives a third operation to switch camera modes. This third operation can be an operation that triggers the switch to a third mode, which is a camera mode that supports SOIS. However, the target camera module corresponding to the third mode is different from that of the second mode. Taking the case where the portrait mode in the electronic device supports SOIS as an example, the aforementioned third operation could be the electronic device displaying... Figure 15When the user clicks mode identifier 1502 on the shooting preview interface 1503, mode identifier 1502 is the mode identifier corresponding to portrait mode. The camera application responds to the third operation, switching the camera mode from the second mode to the third mode. The process of switching the camera mode from the second mode to the third mode is as follows: based on the third operation, the first camera module is powered down, the second camera module is powered on, and the third preview interface is displayed. The second camera module is the target camera module requiring image stabilization in the third mode; the second camera module is a different camera module from the first camera module. The third preview interface is the shooting preview interface for the third mode, and it includes image data captured by the second camera module. After displaying the third preview interface, the camera application switches to the third mode.

[0363] Additionally, after the second camera module is powered on, adjust the position of the lens in the second camera module if the electronic device shakes.

[0364] In other embodiments, the third operation may also be that when the electronic device displays the shooting preview interface 1303, the user clicks the mode identifier 1502. Accordingly, in response to the third operation, the camera application can switch the camera mode from the first mode to the third mode. The implementation details can be referred to the foregoing examples and will not be repeated here.

[0365] In some examples, before powering down the first camera module, the electronic device can control the lens in the first camera module to move to a preset position. This preset position can be a pre-specified fixed position, such as the origin of the initial coordinate system mentioned in the previous embodiments, i.e., the centering position of the lens mentioned in the previous embodiments. Alternatively, the preset position can be the position of the lens when the control axis of the OIS motor is in its initial position. Or, the preset position can be the closest or farthest position during the lens's movement stroke. In this application embodiment, no specific limitation is made. Typically, when the camera module initializes the OIS function, it moves the lens to the preset position. Only after the lens reaches the preset position does the camera module begin optical image stabilization. Additionally, some electronic devices can move the lens to the preset position even after the camera module stops performing optical image stabilization. In this way, after controlling the lens in the first camera module to move to the preset position, the position of the lens in the first camera module remains unchanged even if the electronic device shakes.

[0366] In some embodiments, before the first camera module is powered down, in response to the third operation, the AP sends second control information to the Sensorhub, such as the exit command mentioned in the foregoing embodiments. After receiving the second control information, the Sensorhub controls the lens of the first camera module to move to a preset position. After the lens of the first camera module moves to the preset position, the Sensorhub stops controlling the first camera module to perform optical image stabilization.

[0367] In some embodiments, after the second camera module is powered on, Sensorhub controls the lens of the second camera module to move to a preset position, receives the gyroscope data and / or acceleration data collected by the first sensor, and controls the second camera module to perform optical image stabilization based on the gyroscope data and / or acceleration data.

[0368] In some embodiments, the electronic device further includes a third camera module, which is different from the first camera module. After displaying the second preview interface, the electronic device receives a fourth operation to switch lenses. This fourth operation can be a user-instructed zoom operation. In response to the fourth operation, the electronic device can change the displayed camera module from the first camera module to the third camera module. At a fifth moment, image data acquired by the third camera module is displayed in the second preview interface. The first camera module remains powered on during the display of the third camera module. After the image data acquired by the third camera module is displayed, the camera module requiring image stabilization is the third camera module. After the fifth moment, the electronic device shakes, transmitting electrical signals to the third camera module via Sensorhub to adjust the position of the lens in the third camera module while keeping the position of the lens in the first camera module unchanged.

[0369] In an exemplary scenario, the electronic device can also receive a fifth operation to switch camera modes. This fifth operation could be an operation that triggers a switch to a fourth mode, where the fourth mode does not support SOIS. For example, a portrait mode on the electronic device might not support SOIS, making portrait mode an example of a fourth mode. Correspondingly, the aforementioned third operation could be the electronic device displaying... Figure 15 When the user clicks on the shooting preview interface 1503 shown, the user clicks on the mode icon 1502.

[0370] Additionally, in response to the fifth operation, the electronic device can power on the target camera module corresponding to the fourth mode (e.g., the fourth camera module). If it is determined that the fourth mode does not support optical image stabilization, the Sensorhub controls the OIS-related devices in the fourth camera module to initialize, for example, by controlling the OIS motor to initialize, causing the lens of the fourth camera module to move to a preset position. Afterwards, the AP controls the Sensorhub to release the system resources occupied by handling optical image stabilization matters (OIS-related matters mentioned in the previous embodiments), such as the memory resources allocated to the SOIS connection and the QMI bus port resources mentioned in the previous embodiments. Thus, the Sensorhub does not control the fourth camera module to perform optical image stabilization, and the lens remains in the preset position when the fourth camera module acquires images.

[0371] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the mobile phone in the above method embodiments.

[0372] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the mobile phone in the above method embodiments.

[0373] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0374] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0375] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0376] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0377] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0378] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An optical image stabilization control method characterized by comprising: Applied to an electronic device, the electronic device including: an AP, a Sensorhub, and a first camera module, the method includes: Display a first preview interface of the camera application, wherein the first preview interface includes image data acquired through the first camera module; At a first moment, a first operation to switch camera modes is received. Before the first moment, the camera mode applied by the camera is the first mode, and the first operation indicates that the second mode is enabled. When the second mode corresponds to the first camera module, the AP sends the initialization configuration parameters and cameraindex information corresponding to the first camera module to the Sensorhub. When both the first mode and the second mode correspond to the first camera module, the Sensorhub discards the initialization configuration parameters and cameraindex information from the AP, and sends a first notification to the AP, the first notification indicating that the first camera module has completed initialization; The AP sends a first instruction to the Sensorhub, the first instruction being used to instruct the lens of the first camera module to be moved to a preset position; When both the first mode and the second mode correspond to the first camera module, the Sensorhub discards the first instruction; Between the first and second moments, if the electronic device shakes, the position of the lens in the first camera module is adjusted; if the electronic device does not shake, the position of the lens in the first camera module remains unchanged. At the second moment, a second preview interface is displayed, which includes image data acquired by the first camera module. After the second moment, the camera mode applied by the camera is the second mode.

2. The method according to claim 1, characterized in that, The method includes: At the third moment, receive the second operation to switch camera modes; Between the third and fourth moments, the position of the lens in the first camera module remains unchanged if the electronic device does not shake. At the fourth moment, the first preview interface is displayed, and after the fourth moment, the camera mode of the camera application is the first mode.

3. The method according to claim 1 or 2, characterized in that, The electronic device further includes a sensor hub; adjusting the position of the lens in the first camera module includes: The Sensorhub transmits electrical signals to the first camera module to adjust the position of the lens in the first camera module.

4. The method according to claim 1, characterized in that, The electronic device further includes a second camera module, and after displaying the second preview interface, the method further includes: Receive a third operation to switch camera modes; Based on the third operation, the first camera module is powered down, the second camera module is powered on, and a third preview interface is displayed; wherein, the third preview interface includes image data acquired by the second camera module; after the second camera module is powered on, the camera mode applied by the camera is the third mode; After the control device powers on the second camera module, adjust the position of the lens in the second camera module if the electronic device shakes.

5. The method according to claim 4, characterized in that, The electronic device also includes a Sensorhub, and adjusting the position of the lens in the second camera module includes: The Sensorhub transmits electrical signals to the second camera module to adjust the position of the lens in the second camera module.

6. The method according to claim 4, characterized in that, Before powering down the first camera module, the method further includes: Control the lens in the first camera module to move to a preset position; Specifically, after the lens in the first camera module is moved to the preset position, the electronic device shakes, but the position of the lens in the first camera module remains unchanged.

7. The method according to claim 1, characterized in that, The electronic device also includes a third camera module and Sensorhub. After displaying the second preview interface, the method further includes: The fourth operation is to receive the switching of camera lenses; At the fifth moment, the image data acquired by the third camera module is displayed in the second preview interface; wherein, the first camera module remains powered on. After the fifth moment, the electronic device shakes and transmits an electrical signal to the third camera module through the Sensorhub to adjust the position of the lens in the third camera module while keeping the position of the lens in the first camera module unchanged.

8. The method according to claim 1, characterized in that, The electronic device includes an application processor (AP), a first sensor, and a Sensorhub; the method further includes: In response to the first operation, a first control command is sent to the Sensorhub via the AP; After receiving the first control command, the Sensorhub continues to receive gyroscope data and / or acceleration data collected by the first sensor. Between the first and second moments, if the electronic device shakes, adjusting the position of the lens in the first camera module includes: Using the Sensorhub, the position of the lens in the first camera module is adjusted based on the gyroscope data and / or acceleration data.

9. The method according to claim 4, characterized in that, The electronic device further includes an AP, a Sensorhub, and a first sensor; before the first camera module is powered down, the method further includes: In response to the third operation, the second control information is sent to the Sensorhub via the AP; After receiving the second control information, the Sensorhub controls the lens of the first camera module to move to a preset position. After the lens of the first camera module moves to the preset position, the electronic device shakes and no longer adjusts the position of the lens in the first camera module.

10. The method according to claim 9, characterized in that, After powering on the second camera module, the method further includes: The Sensorhub controls the lens of the second camera module to move to a preset position. The Sensorhub receives gyroscope data and / or acceleration data collected by the first sensor. Using the Sensorhub, the position of the lens in the second camera module is adjusted based on the gyroscope data and / or acceleration data.

11. The method according to claim 1, characterized in that, The electronic device includes an access point (AP) and a sensor hub. Before displaying the first preview interface, the method further includes: In response to the operation of opening the camera application, a first data packet is sent to the Sensorhub through the AP; wherein, the first data packet includes the initialization configuration parameters of the optical image stabilization motor in the first camera module; The Sensorhub sends an initialization success message to the AP.

12. The method according to claim 11, characterized in that, The electronic device further includes a first sensor; after sending initialization success information to the AP, the method further includes: When optical image stabilization is enabled in the first mode, the gyroscope data and / or acceleration data collected by the first sensor are received through the Sensorhub. Using the Sensorhub, the position of the lens in the first camera module is adjusted based on the gyroscope data and acceleration data.

13. The method according to claim 1 or 2, characterized in that, The electronic device further includes an AP, a first sensor, a Sensorhub, and a fourth camera module; the method further includes: The system receives a fifth operation to switch camera modes; wherein the fifth operation indicates that the enabled camera mode is the fourth mode. Power on the fourth camera module; When optical image stabilization is not supported in the fourth mode, the lens of the fourth camera module is moved to a preset position via the Sensorhub. The AP controls the Sensorhub to release the system resources used for processing optical image stabilization.

14. The method according to claim 13, characterized in that, Before controlling the lens of the fourth camera module to move to a preset position via the Sensorhub, the method further includes: The AP determines whether the fourth mode is a camera mode that supports optical image stabilization. When the camera mode is single-lens mode, the camera mode supports enabling optical image stabilization; When the camera mode is multi-camera mode and the multi-camera algorithm configured in the electronic device does not support optical image stabilization, the camera module does not support enabling optical image stabilization. When the camera mode is a multi-camera mode and the multi-camera algorithm configured in the electronic device supports optical image stabilization, the camera mode supports enabling optical image stabilization.

15. An optical image stabilization control method, characterized in that, Applied to an electronic device, the electronic device comprising: a first camera module, a second camera module, an access point (AP), and a Sensorhub, the method comprising: Displays a first preview interface of the camera application, wherein the first preview interface includes image data captured by the first camera module; When the electronic device is shaken, an electrical signal is transmitted to the first camera module via the Sensorhub to adjust the position of the lens in the first camera module. Receive a first operation to switch camera modes; prior to receiving the first operation, the camera application was in a first mode, and the first operation indicated that a second mode was enabled; When the second mode corresponds to the first camera module, the AP sends the initialization configuration parameters and cameraindex information corresponding to the first camera module to the Sensorhub. When both the first mode and the second mode correspond to the first camera module, the Sensorhub discards the initialization configuration parameters and cameraindex information from the AP, and sends a first notification to the AP, the first notification indicating that the first camera module has completed initialization; The AP sends a first instruction to the Sensorhub, the first instruction being used to instruct the lens of the first camera module to be moved to a preset position; When both the first mode and the second mode correspond to the first camera module, the Sensorhub discards the first instruction; Receive a third operation to switch camera modes, the third operation indicating that a third mode is enabled; When the third mode corresponds to the second camera module, the AP sends the initialization configuration parameters and cameraindex information corresponding to the second camera module to the Sensorhub; In response to receiving the cameraindex information of the second camera module, the Sensorhub initializes the second camera module according to the initialization configuration parameters corresponding to the second camera module. The Sensorhub sends a second notification to the AP, the second notification indicating that the second camera module has completed initialization; The AP sends a first instruction to the Sensorhub, the first instruction being used to instruct the lens of the first camera module to be moved to a preset position; The Sensorhub controls the movement of the lens of the first camera module to a preset position; In response to the third operation, a third preview interface is displayed, wherein the third preview interface includes image data acquired by the second camera module; When the electronic device is shaken, an electrical signal is transmitted to the second camera module via the Sensorhub to adjust the position of the lens in the second camera module.

16. An electronic device comprising a Sensorhub, one or more camera modules, a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-15.

17. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-15.

Images