Camera control method and electronic device

Abstract

Provided in the present application are a camera control method and an electronic device. In the method, an electronic device controlling, according to its own state, a switch to close a path between a camera and a low-power-consumption processor, such that the corresponding camera operates in a low-power-consumption mode, thereby realizing camera switching in a low-power-consumption scenario, and meeting the requirements of different scenarios.

Description

Camera control method and electronic device Technical Field

[0001] The present application relates to the field of terminal devices, and in particular to a camera control method and electronic device. Background Art

[0002] With the advancement of terminal technology, terminal devices are becoming increasingly versatile and applicable in a wider range of scenarios. By introducing low-power devices and modules, terminal devices can implement corresponding low-power functions. For example, they can recognize gestures in the air while the screen is on. These functions often require the use of images captured by cameras at a low power cost to perform image recognition and achieve the corresponding functions. Current low-power solutions typically use images captured by a single camera, making these single-camera control solutions unsuitable for multi-camera scenarios.

[0003] Summary of the Invention

[0004] The present application provides a camera control method and electronic device, which can control the switching of multiple cameras according to scene requirements.

[0005] In a first aspect, the present application provides a camera control method, applied to an electronic device. The electronic device includes a first camera and a second camera, the first camera and the second camera being coupled to a first switch, which is coupled to a low-power processor. The first switch is configured to selectively connect a first data path between the first camera and the low-power processor or a second data path between the second camera and the low-power processor. The method comprises: in response to a first user operation received, the electronic device switches from a first state to a second state. Upon detecting that the electronic device has switched from the first state to the second state, the electronic device controls the first switch to connect the first data path. The electronic device performs image recognition on a captured first image and obtains a first image recognition result. The first image is captured by the first camera and processed by a low-power ISP. In response to a second user operation received, the electronic device switches from the second state to the first state. Upon detecting that the electronic device has switched from the second state to the first state, the electronic device controls the first switch to connect the second data path. The electronic device performs image recognition on a captured second image and obtains a second image recognition result; the second image is captured by the second camera and processed by the low-power ISP.

[0006] In this way, by coupling multiple cameras to a low-power processor, the present application can achieve the goal of enabling the corresponding camera by controlling the corresponding path through the switch when not in use. Moreover, the camera is operated in low-power mode after being started (i.e., enabled). This allows the corresponding camera to be switched according to different scene requirements in low-power scenarios.

[0007] Exemplarily, the electronic device is a foldable screen mobile phone, a straight screen mobile phone, a tablet, a computer, a wearable electronic device, etc.

[0008] Exemplarily, the first switch may be a single-pole double-throw switch, such as switch 803 - 1 .

[0009] In one possible implementation, detecting that the electronic device switches from the first state to the second state, and the first switch switches on the first data path, includes: the electronic device determining, based on camera configuration information, that the camera corresponding to the second state is the first camera; switching on the first data path based on configuration information corresponding to the first camera in the camera configuration information; and acquiring a first image through the first data path.

[0010] In this way, the low-power processor in this application is connected to two cameras through one port. Based on the status and configuration information of the electronic device, the corresponding camera can be turned on to make the turned-on camera work in low-power mode.

[0011] In one possible implementation, the first camera and the second camera are coupled to a second switch, which is coupled to a low-power processor. The second switch is configured to selectively connect a first control path between the first camera and the low-power processor or a second control path between the second camera and the low-power processor. Upon detecting that the electronic device switches from a first state to a second state, the first switch connects a first data path. The method further includes: the electronic device controlling the second switch to connect the first control path based on configuration information corresponding to the first camera. The electronic device transmits control signals to the first camera via the first control path.

[0012] In this way, the low-power processor in this application can switch the control circuit through the switch, and can interact with the enabled camera through control signals, such as providing electrical signals to the camera.

[0013] In one possible implementation, detecting that the electronic device switches from the second state to the first state, and the first switch switches on the second data path, includes: the electronic device determining, based on camera configuration information, that the camera corresponding to the first state is the second camera; and switching on the second data path based on configuration information corresponding to the second camera in the camera configuration information.

[0014] In this way, the low-power processor in this application is connected to two cameras through one port. Based on the status and configuration information of the electronic device, the corresponding camera can be turned on to make the turned-on camera work in low-power mode.

[0015] In one possible implementation, it is detected that the electronic device switches from the second state to the first state, and the first switch turns on the second data path, including: the electronic device controls the second switch to turn on the second control path based on configuration information corresponding to the second camera; and the electronic device transmits a control signal to the second camera through the second control path.

[0016] In this way, the low-power processor in this application can switch the control circuit through the switch, and can interact with the enabled camera through control signals, such as providing electrical signals to the camera.

[0017] In a possible implementation, the first switch is coupled to a first port of the low-power processor, and the second switch is coupled to a second port of the low-power processor.

[0018] Exemplarily, the first port may also be referred to as a data port, which is used to transmit data, such as image data.

[0019] Exemplarily, the second port may also be referred to as a control port, such as an I2C, MCLK, MIPI and / or RESET interface.

[0020] In one possible implementation, the first state of the electronic device is a folded state, and the second state of the electronic device is an unfolded state. In this way, the present application can determine the camera that needs to be enabled based on the different states of the electronic device, such as the switch between the folded state and the unfolded state, and enable the corresponding camera through the control circuit, so that the camera operates in a low-power mode to meet the current scene requirements.

[0021] In one possible implementation, the first camera is set on the external screen of the electronic device, and the second camera is set on the internal screen of the electronic device. In this way, the present application can determine the camera that needs to be enabled based on the different states of the electronic device, such as the switch between the folded state and the unfolded state, and enable the corresponding camera through the control circuit, so that the camera operates in a low-power mode to meet the current scene requirements.

[0022] In one possible implementation, the first camera is a front-facing camera, and the second camera is a rear-facing camera. Thus, the solution in this application can control switching between multiple front-facing cameras, and can also control switching between the front and rear cameras, to meet the low-power consumption requirements of electronic devices in different scenarios.

[0023] In a second aspect, the present application provides a control circuit, comprising: a low-power processor; a first switch, the first switch being coupled to the low-power processor; a first camera and a second camera, the first camera and the second camera being coupled to the first switch, the first switch being used to selectively conduct a first data path between the first camera and the low-power processor or a second data path between the second camera and the low-power processor.

[0024] In a possible implementation, the first camera is coupled to a first terminal of a first switch, and the second camera is coupled to a second terminal of the first switch.

[0025] In one possible implementation, the first camera and the second camera are coupled to a second switch, the second switch is coupled to a low-power processor, and the second switch is used to selectively conduct a first control path between the first camera and the low-power processor or a second control path between the second camera and the low-power processor.

[0026] In a possible implementation, the first switch is coupled to a first port of the low-power processor; and the second switch is coupled to a second port of the low-power processor.

[0027] In one possible implementation, the low-power processor is configured to send a first control signal to the first switch to control the first switch to conduct the first data path; the low-power processor is further configured to send a second control signal to the first switch to control the first switch to conduct the second data path.

[0028] In a possible implementation, the first camera is disposed on an external screen of the electronic device, and the second camera is disposed on an internal screen of the electronic device.

[0029] In a possible implementation, the first camera is a front camera, and the second camera is a rear camera.

[0030] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0031] In a third aspect, the present application provides a camera control method, applied to an electronic device, the electronic device comprising a first camera and a second camera, the first camera and the second camera being coupled to a first switch, the first switch being coupled to a low-power processor, the first switch being configured to selectively conduct a first data path between the first camera and the low-power processor or a second data path between the second camera and the low-power processor. The method comprises: in response to a received unfolding operation, the electronic device switching from a folded state to an unfolded state; upon detecting that the electronic device has switched from the folded state to the unfolded state, the first switch conducting the first data path; performing image recognition on a captured first image and obtaining a first image recognition result; the first image is obtained by capturing the first camera and processing the image signal processor (ISP); in response to a received folding operation, the electronic device switching from the unfolded state to the folded state; upon detecting that the electronic device has switched from the unfolded state to the folded state, the first switch conducting the second data path; performing image recognition on a captured second image and obtaining a second image recognition result; the second image is obtained by capturing the second camera and processing the image signal processor (ISP).

[0032] Exemplarily, the electronic device is a foldable device such as a foldable mobile phone, a foldable tablet or a foldable computer.

[0033] In one possible implementation, when the electronic device switches from a folded state to an unfolded state, the first switch turns on the first data path, including: determining, based on camera configuration information, that the camera corresponding to the unfolded state is the first camera; turning on the first data path based on configuration information corresponding to the first camera in the camera configuration information; and acquiring a first image through the first data path.

[0034] In one possible implementation, the first camera and the second camera are coupled to a second switch, the second switch is coupled to a low-power processor, and the second switch is used to selectively turn on a first control path between the first camera and the low-power processor or a second control path between the second camera and the low-power processor; when it is detected that the electronic device switches from a folded state to an unfolded state, the first switch turns on a first data path, and also includes: based on the configuration information corresponding to the first camera, controlling the second switch to turn on the first control path; and transmitting a control signal to the first camera through the first control path.

[0035] In one possible implementation, when it is detected that the electronic device switches from an unfolded state to a folded state, the first switch turns on the second data path, including: based on the camera configuration information, determining that the camera corresponding to the folded state is the second camera; based on the configuration information corresponding to the second camera in the camera configuration information, turning on the second data path.

[0036] In one possible implementation, when it is detected that the electronic device switches from an unfolded state to a folded state, the first switch turns on the second data path, including: controlling the second switch to turn on the second control path based on configuration information corresponding to the second camera; and transmitting a control signal to the second camera through the second control path.

[0037] In a possible implementation, the first switch is coupled to a first port of the low-power processor; and the second switch is coupled to a second port of the low-power processor.

[0038] In a possible implementation, the first camera is disposed on an external screen of the electronic device, and the second camera is disposed on an internal screen of the electronic device.

[0039] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0040] In a fourth aspect, the present application provides a camera control method, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, the first camera and the second camera are coupled to a first switch, the first switch is coupled to a low-power processor, and the first switch is used to selectively turn on the first data path between the first camera and the low-power processor or the second data path between the second camera and the low-power processor. The method includes: the first switch turns on the first data path; image recognition is performed on the acquired first image, and a first image recognition result is obtained; the first image is captured by the first camera and processed by the low-power image signal processor ISP; in response to the received lifting operation, the electronic device switches from a motion state to a stationary state; when the electronic device is detected to switch from a motion state to a stationary state, the first switch turns on the second data path; image recognition is performed on the acquired second image, and a second image recognition result is obtained; the second image is captured by the second camera and processed by the low-power ISP.

[0041] For example, the electronic device may be a device such as a bar phone having front and rear cameras.

[0042] For example, this application only uses the lifting operation as an example for illustration. In other embodiments, other triggering operations can also be used to trigger a change in the state of the electronic device, so that the electronic device determines the scene requirements based on the state and switches the corresponding camera. This application does not limit this.

[0043] In one possible implementation, when it is detected that the electronic device switches from a motion state to a stationary state, the first switch turns on the second data path, including: based on the camera configuration information, determining that the camera corresponding to the first state is the second camera; based on the configuration information corresponding to the second camera in the camera configuration information, turning on the second data path.

[0044] In one possible implementation, when it is detected that the electronic device switches from a motion state to a stationary state, the first switch turns on the second data path, including: controlling the second switch to turn on the second control path based on configuration information corresponding to the second camera; and transmitting a control signal to the second camera through the second control path.

[0045] In a possible implementation, the first switch is coupled to a first port of the low-power processor; and the second switch is coupled to a second port of the low-power processor.

[0046] In a possible implementation, the first camera is a front camera, and the second camera is a rear camera.

[0047] In a fifth aspect, the present application provides an electronic device. The electronic device includes one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory and, when executed by the one or more processors, cause the electronic device to perform the method of the first aspect or any possible implementation of the first aspect.

[0048] In a sixth aspect, the present application provides an electronic device. The electronic device includes one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory and, when executed by the one or more processors, cause the electronic device to perform the method of the third aspect or any possible implementation of the third aspect.

[0049] In a seventh aspect, the present application provides an electronic device. The electronic device includes one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory and, when executed by the one or more processors, cause the electronic device to perform the method of the fourth aspect or any possible implementation of the fourth aspect.

[0050] In an eighth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0051] In a ninth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the third aspect or any possible implementation of the third aspect.

[0052] In a tenth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the fourth aspect or any possible implementation of the fourth aspect.

[0053] In an eleventh aspect, the present application provides a computer program comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0054] In a twelfth aspect, the present application provides a computer program comprising instructions for executing the method in the third aspect or any possible implementation of the third aspect.

[0055] In a thirteenth aspect, the present application provides a computer program comprising instructions for executing the method in the fourth aspect or any possible implementation of the fourth aspect.

[0056] In a fourteenth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the first aspect or any possible implementation of the first aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal.

[0057] In a fifteenth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the third aspect or any possible implementation of the third aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal.

[0058] In a sixteenth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the fourth aspect or any possible implementation of the fourth aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] FIG1 is a schematic diagram showing a hardware structure of an electronic device;

[0060] FIG2 is a schematic diagram of an exemplary module interface;

[0061] FIG3 is a schematic diagram of a camera module of an electronic device;

[0062] FIG4 is a schematic diagram illustrating a software structure of an electronic device;

[0063] 5a and 5b are schematic diagrams illustrating the connection of a low-power processor;

[0064] 6a and 6b are schematic diagrams illustrating the connection of a low-power processor;

[0065] 7a and 7b are schematic diagrams illustrating the connection of a low-power processor;

[0066] 8a to 8c are schematic diagrams illustrating exemplary circuit connections;

[0067] FIG9 is an exemplary flowchart of module interaction;

[0068] FIG10 is an exemplary flowchart of module interaction;

[0069] FIG11 is an exemplary flowchart of module interaction;

[0070] FIG12 is an exemplary flowchart of module interaction;

[0071] FIG13 is a schematic diagram showing an exemplary device structure. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0073] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0074] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0075] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0076] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0077] Optionally, in an embodiment of the present application, the terminal device may be a mobile terminal with a shooting function, such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like; or it may be a professional shooting device such as a digital camera, a SLR camera / micro-single camera, a sports camera, a gimbal camera, a drone, etc. The embodiment of the present application does not limit the specific type of the terminal device.

[0078] FIG1 shows a schematic diagram of the structure of an electronic device 100. It should be understood that the electronic device 100 shown in FIG1 is merely an example of an electronic device, and that the electronic device 100 may have more or fewer components than shown, may combine two or more components, or may have a different component configuration. The various components shown in FIG1 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0079] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0080] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), an AO (always on) ISP, a low-power processor, a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0081] The AO ISP, also known as a low-power ISP, performs ISP image signal processing on image information, generating image information in a pre-set format. Compared to a standard ISP, a low-power ISP can only perform simple noise reduction and output grayscale images. It lacks automatic white balance and autofocus processing, or may provide simple white balance and / or simple focus processing. Image clarity depends solely on module resolution. Generally, cameras in AO mode have a lower output resolution, thus consuming less power.

[0082] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0083] The low-power core, also known as a low-power processor, is used to implement the steps performed by the modules on the low-power side of the embodiment of the present application. For example, the low-power processor can be used to switch the camera in low-power mode.

[0084] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0085] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0086] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0087] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0088] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0089] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0090] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0091] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0092] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0093] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0094] Figure 2 is a schematic diagram of an exemplary module interface. Please refer to Figure 2. The camera (i.e., Camera Sensor) transmits data (such as image data) to the AP side and the low-power side through the Mipi interface. The AP side and the low-power side can send control signals to the camera through the I2C interface. Among them, the above-mentioned interface is only an illustrative example, and this application does not limit the interface type. Optionally, the AP side may include devices such as a CPU and ISP and their corresponding software or modules (such as CameraHal, etc. mentioned below). The low-power side may include devices such as AO ISP and low-power processors and their corresponding software or modules (such as the various modules of the low-power side mentioned below). In an embodiment of the present application, the AP side and the low-power side are mutually exclusive. It can be understood that during the process of data interaction (including image data and control signals) between the AP side and the camera through the interface, the low-power side cannot interact with the camera, that is, the low-power side cannot call the camera. In the embodiment of the present application, the process of the AP side and the low-power side calling the camera can also be understood as the process of registering to use the camera (or can be understood as binding with the camera), wherein the object that has registered to use the camera (such as the AP side) can use the camera, and if the other side (such as the low-power side) needs to use the camera, it needs to deregister (or can be understood as releasing) with the camera on the AP side before the other side can register (i.e. bind) with the camera and exchange data with the camera. In the embodiment of the present application, the registration and deregistration of the module with the camera (i.e., the camera) can be understood as the module calling the registration function (or registration interface) to bind with the camera, which will not be repeated below.

[0095] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0096] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0097] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0099] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0100] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0101] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0102] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0103] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0104] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0105] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0106] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0107] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0108] FIG3 is a schematic diagram of a camera module of an electronic device. Referring to FIG3 , (1), (2) and (3) of FIG3 schematically illustrate the front face 101 of the outer screen of the electronic device 100 when it is in a folded state, the back face 102 of the outer screen, and the inner screen 103 of the electronic device 100 when it is in an unfolded state. The front face of the electronic device 100 can be understood as the side facing the user when the user uses the electronic device 100, and the back face of the electronic device 100 can be understood as the side facing away from the user when the user uses the electronic device 100.

[0109] The camera module 104 is used to capture still images or videos. The camera module 104 can be set on the front (folded state), back and / or inner screen of the electronic device 100. As shown in (1) of Figure 3, when the camera module 104 is set on the front of the electronic device 100, the front camera 104-1 can be used to shoot the scene on the front side of the electronic device 100, such as for self-portraits, and in some embodiments, it can be called a front camera. As shown in (3) of Figure 3, when the camera module 104 is set on the inner screen of the electronic device 100 and the electronic device 100 is in the unfolded state, the front camera 104-2 can be used to shoot the scene on the front side of the electronic device 100, such as for self-portraits, and in some embodiments, it can be called a front camera. To distinguish between the two front cameras, in the embodiment of the present application, the front camera 104-1 in the folded state is referred to as the front camera-0, and the front camera 104-3 in the unfolded state is referred to as the front camera-1, and no further explanation will be given below. As shown in FIG3 (2), when the camera module 104 is disposed on the back of the electronic device 100, the rear camera 104-2 can be used to capture the scene on the back side of the electronic device 100, and in some embodiments, can be referred to as a rear camera. When shooting, the user can select the corresponding camera module according to the shooting requirements.

[0110] It should be noted that the embodiment of the present application does not limit the number of camera modules 104 provided, and can be one, two, four, or even more. For example, one or more camera modules 104 can be provided on the front of the electronic device 100, and / or one or more camera modules 104 can be provided on the back of the electronic device 100. When multiple camera modules 104 are provided, the multiple camera modules 104 can be completely identical or different, for example, the multiple camera modules 104 have different lens optical parameters, different lens installation positions, different lens shapes, etc. The embodiment of the present application also does not impose any restrictions on the relative positions of the multiple camera modules when they are provided.

[0111] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0112] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0113] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0114] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0115] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0116] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0117] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0118] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0119] FIG4 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0120] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other via software interfaces. In some embodiments, the Android system includes, from top to bottom, but is not limited to: an application layer, a framework layer, a hardware abstraction layer (HAL), a kernel layer, and a low-power layer (also referred to as a low-power module layer, which is not limited in this application).

[0121] The application layer can include a series of application packages.

[0122] As shown in FIG4 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0123] The framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0124] As shown in FIG4 , the application framework layer may include but is not limited to: a window manager, a view system, a resource manager, an AO service, and a display subsystem.

[0125] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0126] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0127] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0128] AO service, also known as low-power AO service, includes multiple sub-services, including but not limited to: air gesture service, eye gaze without turning off the screen service, eye annotation volume reduction service, smart scanning service, etc. Optionally, AO service can also be implemented at the application layer, which is not limited in this application.

[0129] The HAL layer is an interface layer located between the operating system kernel layer and the hardware of the electronic device (such as the camera 127). Its purpose is to abstract the hardware and provide a virtual hardware platform for the operating system.

[0130] The HAL layer includes but is not limited to: AO management service, CameraHAL and SensorHAL layer.

[0131] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, motor drivers, and low-power module drivers (also known as low-power core drivers or low-power side communication drivers, which are not limited in this application).

[0132] The low-power layer includes but is not limited to the AO component and the folding screen unfolding algorithm module. Among them, the AO component includes but is not limited to: camera selection module, camera recognition module, camera acquisition module, etc.

[0133] Among them, the Camera selection module is used to control the switching of the camera in low power mode. The Camera recognition module and the Camera acquisition module are used to process the images collected by the camera in low power mode. For example, the Camera acquisition module is used to perform image processing on the images collected by the camera. The Camer recognition module is used to perform image recognition (or image algorithm processing) on ​​the processed images. Optionally, the Camera recognition module is pre-set with at least one algorithm, and the algorithms include but are not limited to: face recognition (for identifying facial features in images) algorithm, face orientation algorithm (for detecting the orientation of the face), smart code scanning algorithm (for identifying image codes), air gesture algorithm (for detecting air gestures), etc.

[0134] The folding and unfolding algorithm module is used to identify the folding state (such as (1) and (2) in Figure 3) or the unfolding state (such as (3) in Figure 3) of the electronic device.

[0135] In the embodiments of the present application, the application layer, framework layer, HAL layer, and kernel layer can be understood as the AP side, where processors or processing units such as the AP, CPU, and GPU support the modules in these layers to execute corresponding steps. On the low-power side, a low-power processor supports the modules in this layer to execute corresponding steps.

[0136] It is understood that the layers in the software structure shown in FIG4 and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0137] 4 also illustrates some of the hardware of the electronic device 100, including but not limited to: a camera group, a CPU, a low-power processor, a display, an ISP, and an AO ISP. The camera group includes but is not limited to: a front-facing camera-0 (e.g., 104-1 in FIG. 3 ), a front-facing camera-1 (e.g., 104-3 in FIG. 3 ), and a rear-facing camera (e.g., 104-2 in FIG. 3 ).

[0138] FIG5a is a connection diagram of an exemplary low-power processor. Please refer to FIG5a. In the prior art, the low-power processor provides a low-power interface (also referred to as an AO interface, which is not limited in this application) for connecting to the front camera to interact with the front camera. However, when the electronic device includes multiple cameras, such as a front camera-0 and a front camera-1, the low-power processor can only interact with one of the front cameras. For example, assuming that the low-power processor is connected to the front camera-0 (such as 104-1 in FIG3), correspondingly, when the electronic device is in a folded state, the low-power processor can call the front camera-0 to implement some low-power functions based on the image captured by the front camera-0, such as air gesture control, human eye annotation to reduce the volume, and other functions. When the electronic device is in an unfolded state, the low-power processor cannot call the front camera-1, that is, the corresponding low-power function cannot be implemented.

[0139] Figure 5b is a connection diagram of the low-power processor provided in an embodiment of the present application. Please refer to Figure 5b. In the embodiment of the present application, one low-power interface of the low-power processor can be connected to multiple cameras, for example, connecting the front camera-0 and the front camera-1. In the embodiment of the present application, the low-power processor can switch and control AO through the circuit, thereby realizing one-to-two, that is, one low-power processor controls two front cameras.

[0140] Figure 6a is a connection diagram of an exemplary low-power processor. Please refer to Figure 6a. In the prior art, the low-power processor provides a low-power interface (also called an AO interface, which is not limited in this application) for connecting to the front camera to interact with the front camera. However, when the electronic device includes multiple cameras, such as a front camera and a rear camera, the low-power processor can only interact with one of the front cameras. For example, assuming that the low-power processor is connected to the front camera (such as 104-1 in Figure 3), when the electronic device is in a folded state, the low-power processor can call the front camera to implement some low-power functions based on the image captured by the front camera, such as air gesture control, human eye annotation to reduce the volume, etc. The low-power processor of the electronic device cannot call the rear camera, that is, it cannot implement the corresponding low-power function. For example, when executing the air code scanning function, the low-power processor can only recognize the image code through the image captured by the front camera.

[0141] Figure 6b is a connection diagram of the low-power processor provided in an embodiment of the present application. Please refer to Figure 6b. In the embodiment of the present application, one low-power interface of the low-power processor can be connected to multiple cameras, such as a front camera and a rear camera. In the embodiment of the present application, the low-power processor can switch and control AO through the circuit, thereby realizing one-to-two, that is, one low-power processor controls the front camera and the rear camera.

[0142] In the embodiment of the present application, the switching and control circuit of AO can also realize a one-to-three scenario of a low-power processor. Figures 7a and 7b are exemplary connection diagrams of a low-power processor. Please refer to Figure 7a. The low-power processor can control two front cameras (including front camera-0 and front camera-1) through a low-power interface. The specific description can be referred to above and will not be repeated here. The low-power processor can be connected to the rear camera through another low-power interface to realize low-power functions based on the image captured by the rear camera, such as the smart code scanning function. Please refer to Figure 7b. The low-power processor can control the front camera (such as front camera-0) and the rear camera through a low-power interface. The specific description can be referred to above and will not be repeated here. The low-power processor can be connected to another front camera (such as front camera-1) through another low-power interface to realize low-power functions based on the image captured by the front camera.

[0143] It should be noted that the embodiments of this application are described using an example of a low-power processor controlling the switching of two cameras (which can be two front-facing cameras, or one front-facing camera and one rear-facing camera) through a single interface. In other embodiments, the circuits and control methods in the embodiments of this application can also be applied to scenarios where a low-power processor controls more than two cameras through a single interface. The specific implementation is the same as that in the embodiments of this application, and this application will not further illustrate each example.

[0144] FIG8a is a schematic diagram of an exemplary circuit connection. Referring to FIG8a , the circuit structure includes but is not limited to a low-power processor 801, a front camera 802-1, and a front camera 802-2. The front camera 802-1 is coupled to the L end (also referred to as the first terminal) of the switch 803-1, and the front camera 802-2 is coupled to the H end (also referred to as the second terminal) of the switch 803-1. The switch 803-1 is coupled to the O1 interface (also referred to as the AO interface or data interface) of the low-power processor 801. Optionally, the O1 interface of the low-power processor 801 can be a PHY interface for receiving data input by the camera, such as image data captured by the camera.

[0145] Still referring to Figure 8a, the switch 803-2 is coupled to the O3 interface of the low-power processor 801. The front camera 802-1 is coupled to the L terminal of the switch 803-2, and the front camera 803-2 is coupled to the H terminal of the switch 803-2.

[0146] Optionally, the 03 interface of the low-power processor 801 can be an I2C, MCLK, MIPI and / or RESET interface, which is used for data interaction with the front camera 802-1 or the front camera 802-2, wherein the data or signal output and input by the interface can be called a control signal or control data. For example, the low-power processor 801 can output power signals, power-on timing, I2C parameters and other control data (also referred to as configuration parameters) or control signals to the front camera 802-1 or the front camera 802-2 through the 03 interface to enable the front camera 802-1 or the front camera 802-2.

[0147] Continuing with FIG8 a , the O2 interface of the low-power processor is a GPIO interface, which is coupled to switch 803-1 and switch 803-2, respectively, and is used to output control signals to switch 803-1 and switch 803-2 to control switches 803-1 and 803-2. Optionally, the O2 interface of the low-power processor is connected to the control terminal of switch 803-1 and the control terminal of switch 803-2.

[0148] It should be noted that the types of interfaces for outputting control signals and configuration parameters involved in the embodiments of the present application are merely illustrative examples, and are not limited in this application and will not be repeated below.

[0149] In an embodiment of the present application, the low-power processor 801 can output a control signal through the GPIO interface (i.e., interface 02) to control the opening and closing of switches 803-1 and 803-2, thereby connecting the circuit between the low-power processor 801 and the front camera 802-1 or the circuit between the low-power processor 801 and the front camera 802-2.

[0150] For example, Figure 8b is an exemplary circuit connection diagram. Referring to Figure 8b, the low-power processor 801 outputs a first control signal (also referred to as a first GPIO signal) through interface 02, controls switch 803-1 to be placed at the L end, and controls switch 803-2 to be placed at the L end. Accordingly, the path between the front camera 802-1 and interface 01 of the low-power processor 801 is connected, and the path between the front camera 802-1 and interface 03 of the low-power processor 801 is connected. The low-power processor 801 can exchange data with the front camera 802-1 through the interface between interface 01 and the front camera 802-1, such as receiving images captured by the front camera 802-1. The low-power processor 801 can exchange control data with the front camera 802-1 through the interface between interface 03 and the front camera 802-1, such as outputting an electrical signal to the front camera 802-1.

[0151] To illustrate another example, FIG8c is a schematic diagram illustrating an exemplary circuit connection. Referring to FIG8c, the low-power processor 801 outputs a second control signal (also referred to as a second GPIO signal) through interface 02, controls switch 803-1 to be at terminal H, and controls switch 803-2 to be at terminal H. Accordingly, the path between the front camera 802-2 and interface 01 of the low-power processor 801 is connected, and the path between the front camera 802-2 and interface 03 of the low-power processor 801 is connected. At the same time, the path between the front camera 802-1 and interface 01 of the low-power processor 801 is disconnected, and the path between the front camera 802-1 and interface 03 of the low-power processor 801 is disconnected.

[0152] Still referring to Figure 8c, the low-power processor 801 can exchange data with the front camera 802-2 through the interface between interface 01 and the front camera 802-2, such as receiving images captured by the front camera 802-2. The low-power processor 801 can exchange control data with the front camera 802-2 through the interface between interface 03 and the front camera 802-2, such as outputting electrical signals to the front camera 802-2.

[0153] To illustrate this further, low-power processor 801 can output a first control signal via interface 03, thereby enabling front camera 802-1. For a circuit description, see Figure 8b and will not be repeated here. Simultaneously, the path between front camera 802-2 and interface 01 of low-power processor 801 is disconnected, and the path between front camera 802-2 and interface 03 of low-power processor 801 is disconnected.

[0154] That is, the low-power processor outputs the first control signal or the second control signal through the interface 02 to control the switch 803 - 1 and the switch 803 - 2 , thereby realizing the switching control of multiple cameras.

[0155] It should be noted that the circuit connection relationship in Figures 8a to 8c is only an illustrative example. In other embodiments, the circuit may include more or fewer paths, and the purpose is to achieve switching between multiple cameras by controlling the switch. The specific circuit design can be set according to actual needs and is not limited in this application.

[0156] In an embodiment of the present application, the low-power processor can switch the corresponding camera based on the scene requirements of the electronic device to obtain the image captured by the specified camera, and implement the low-power function based on the captured image.

[0157] In conjunction with Figures 8a to 8c, the camera control method in the embodiment of the present application is described in detail below with reference to a specific application scenario. Figure 9 is an exemplary module interaction flow chart, which specifically includes:

[0158] In the embodiment of the present application, during the initialization phase of the electronic device (e.g., restart or restore to factory settings), the CameraHAL outputs camera configuration information to the Camera selection module. The camera configuration information is used to enable the corresponding camera and may include, but is not limited to, power supply configuration files, camera module configuration files, power-on sequence, I2C address, and other information. This information can be set according to actual needs and is not limited by this application.

[0159] In the embodiment of the present application, the camera configuration information stored by the CameraHAL includes, but is not limited to, the configuration information of the front camera 0, the configuration information of the front camera 1, and the configuration information of the rear camera. In this example, the low-power processor is used to control the switching between the front camera 0 and the front camera 1. Accordingly, the CameraHAL can output the configuration information of the front camera 0 and the configuration information of the front camera 1 to the camera selection module.

[0160] Optionally, the camera configuration information saved by CameraHAL is factory-set, or updated with system updates, which is not limited in this application.

[0161] Optionally, CameraHAL can send camera configuration information to the AO management service, which outputs the camera configuration information to the low-power module driver, and the low-power module driver forwards it to the Camera selection module in the AO component.

[0162] Optionally, the Camera selection module saves the acquired camera configuration information in a memory, such as a designated area in RAM.

[0163] Optionally, the electronic device's settings application may include a low-power function option, which the user can trigger to enable low-power mode. In response to a received user operation, the CameraHAL may indicate camera control permissions to the low-power side (i.e., the AO component). For example, the CameraHAL may send camera registration indication information to the AO management service, indicating that camera usage permissions are allocated to the low-power side, i.e., the AO component. The AO management service outputs the camera registration indication information to the AO component via the low-power module driver. In response to the camera registration indication information, the camera selection module determines that the camera can be used, which can also be understood as control of the camera. The camera selection module may call a registration function to register for use of the camera group. Registration can also be understood as binding; that is, only after the camera selection module is bound to the camera group can the AO component exchange data (including control signals and image data, etc.) with the camera group. It should be noted that during the binding process between the AO component and the camera group, other modules on the AP side cannot call the camera component. That is, the images captured by the camera component are only used by the low-power side to perform corresponding low-power processing.

[0164] Exemplarily, the Camera selection module may save the correspondence between the camera configuration information and the application scenario requirements based on pre-set control rules (which may be set according to actual needs and are not limited in this application). For example, the correspondence saved by the Camera selection module may include but is not limited to: the correspondence between the unfolded state and the configuration information of the front camera-1, the folded state and the configuration information of the front camera-0, etc. Among them, the correspondence between the unfolded state and the configuration information of the front camera-1 is used to indicate that when the electronic device is in the unfolded state, based on the configuration information of the front camera-1, the front camera-1 is enabled. The folded state and the configuration information of the front camera-0 are used to indicate that when the electronic device is in the folded state, based on the configuration information of the front camera-0, the front camera-0 is enabled.

[0165] It should be noted that the corresponding relationship in the embodiment of the present application is only an illustrative example. In other embodiments, the corresponding relationship can be set according to the actual scenario requirements. For example, in the usage scenario of a straight-screen mobile phone, the corresponding relationship can also include a corresponding relationship between the hand-raising action (i.e., the operation of lifting the mobile phone) and the configuration relationship of the rear camera, to indicate that after recognizing that the user has lifted the electronic device, the rear camera is enabled. That is, the electronic device can determine whether the electronic device is lifted by obtaining the motion state of the electronic device (for example, the electronic device switches from a motion state to a stationary state) and posture (such as parameters such as the tilt angle of the electronic device). If the electronic device is lifted, the rear camera is called. The electronic device can determine whether the image includes a graphic code based on the image captured by the rear camera. If a graphic code is included, the code scanning function can be called. In other words, the triggering conditions for the electronic device to switch the camera in the embodiment of the present application can be set according to different operations and different states, and this application does not limit it.

[0166] It should be further noted that enabling each camera in the embodiments of the present application refers to enabling the camera in low-power mode. In other words, the image captured by the camera is processed by the AO ISP and then input to the low-power side for subsequent processing.

[0167] Please refer to Figure 9. In this example, the electronic device is changed from the folded state of Figure 3 (1) to the unfolded state shown in Figure 3 (3) as an example. The sensor outputs the detection parameters to the folding screen unfolding algorithm module. The folding screen unfolding algorithm module can determine that the current state of the electronic device is the unfolded state based on the detection parameters input by the sensor. The folding screen unfolding algorithm module drives the output of unfolding state indication information to the sensor to indicate that the electronic device is in the unfolded state.

[0168] The sensor driver outputs expanded state indication information to the SensorHAL, indicating that the electronic device is in the expanded state. It should be noted that the information transmission between modules may require format conversion to meet the format requirements of the interface between the modules. The specific transmission method can be set according to actual needs and will not be repeated in this application or below.

[0169] SensorHAL sends expansion state indication information to the display subsystem to indicate that the electronic device is in the expansion state. The display subsystem can switch the display content of the electronic device to the inner screen (for example, the inner screen 103 in (3) of Figure 3) for display. In addition, the display subsystem sends expansion state indication information to CameraHAL. CameraHAL detects that it is currently in low power consumption mode, that is, CameraHAL can call the interface function between the AO management service based on the low power consumption process that has been started to send expansion state indication information to the AO management service to indicate that the electronic device is in the expansion state.

[0170] In response to the CameraHAL's instructions, the AO management service sends an expansion state indication to the low-power module driver. The low-power module driver sends the expansion state indication to the AO component. The Camera selection module in the AO component can enable the corresponding camera based on the expansion state indication. It should be noted that the interaction between the modules on the AP side in the embodiments of the present application can be understood as calling the interface function of the peer module to output information or data to another module, and this description will not be repeated below.

[0171] Exemplarily, the camera selection module determines that the electronic device is currently in the expanded state based on the expanded state indication information. The camera selection module may determine that the front camera 1 needs to be enabled based on the correspondence between the cached expanded state and the configuration information of the front camera 1. Exemplarily, the camera selection module may call a registration function to register (also understood as binding) with the front camera 1. In other words, the current usage permission of the front camera 1 corresponds to the AO component. Other upper-layer modules cannot call the front camera 1.

[0172] Accordingly, as shown in FIG8a , assuming that the front camera 802-2 in FIG8a is the front camera-1 in the inner screen, the camera selection module outputs a second control signal through the interface 02 of the low-power processor to control the switch 803-1 to be positioned at the H terminal and the switch 803-2 to be positioned at the H terminal. Accordingly, as shown in FIG8c , the path between the front camera 802-2 and the interface 01 of the low-power processor 801 is connected, and the path between the front camera 802-2 and the interface 03 of the low-power processor 801 is also connected.

[0173] Still referring to FIG. 9 , illustratively, after the camera selection module switches to front camera 1 via the control selection circuit, the camera selection module may interact with front camera 1 to transmit control data (or control signals) based on the configuration information of front camera 1. For example, the camera selection module may output electrical signals, clock signals, etc. to front camera 1 via interface 02 of the low-power processor, although this application does not limit this.

[0174] Exemplarily, after the front camera-1 is enabled, it captures images. The AO ISP can perform image processing on the image captured by the front camera-1 and output the processed image to the AO component. The Camera recognition module and the Camera acquisition module in the AO component can process and recognize the image. As mentioned above, the Camera recognition module can be pre-set with at least one algorithm, for example, including but not limited to: face recognition algorithm, face orientation algorithm, smart code scanning algorithm, air gesture algorithm, etc. The Camera recognition module can recognize the image based on at least one algorithm to recognize air gestures, face orientation, facial features and / or image codes, etc.

[0175] It can be understood that the low-power scenarios (also referred to as low-power services) in the embodiments of the present application may include but are not limited to: code recognition, air gestures, face orientation, face appearance, and eye gaze.

[0176] The code recognition scenario features include intelligent code recognition. The user experience includes: aiming the front camera at the image code, confirming it, and then pulling up the corresponding service page.

[0177] Features of air gesture scenes include: page flipping, screenshots, and customizable quick task cards. The corresponding user experiences are: continuous up and down scrolling of apps or sections within apps; screenshots can be taken by air; and quick task cards pop up automatically with a press gesture.

[0178] Features for face-facing scenes include face-facing smart landscape and portrait modes and face-facing assisted photo imaging. The corresponding user experiences are: when using the device sideways, the app's orientation matches the user's face, and the resulting image is aligned with the face's orientation when framing the scene.

[0179] Features for scenarios where faces appear include Face AOD (Always on Display). The corresponding user experience is: when the screen is off and someone approaches, the AOD is visible without touching the screen.

[0180] The features of human eye gaze scenarios include: gaze without turning off the screen and gaze to reduce the volume of incoming calls. The corresponding user experiences are: the screen does not turn off with continuous or intermittent gaze; looking at the screen during an incoming call reduces the ringtone volume.

[0181] The low-power side (specifically the Camera Face Detect module, etc.) can be based on the above-mentioned scenario requirements and combined with pre-stored algorithms for recognition to identify facial features (such as human eye gaze), air gestures, etc. The AO component can output the recognition machine results to the AO management service. The AO management service outputs the recognition results to the AO service. The AO service can perform corresponding low-power functions based on the recognition results. For example, if the recognition result indicates that it is human eye gaze, the screen can be kept on. The specific implementation method can refer to the existing technical embodiments, and this application will not go into details.

[0182] Exemplarily, when the electronic device is in the unfolded state, the low-power processor (ie, the low-power side) continuously calls the front camera-1 to perform corresponding low-power functions based on the image captured by the front camera-1 in the low-power mode.

[0183] In the embodiment of the present application, as shown in FIG10 , if the electronic device switches from the unfolded state to the folded state, the display subsystem can obtain the folded state indication information accordingly. The specific acquisition method is similar to that of the unfolded state and will not be repeated here.

[0184] The display subsystem can send folding state indication information to the CameraHAL, and the CameraHAL sends the folding state indication information to the Camera selection module. The specific transmission process can be referred to Figure 9 and will not be repeated here.

[0185] Exemplarily, the camera selection module responds to the received folding state indication information and determines based on the cached correspondence that it is necessary to switch to the front camera - 0. Optionally, the camera selection module deregisters with the front camera - 1 and calls the registration interface to register with the front camera - 0.

[0186] Accordingly, as shown in FIG8a , assuming that the front camera 802-1 in FIG8a is the front camera-0 in the inner screen, the camera selection module outputs a first control signal through the interface 02 of the low-power processor to control the switch 803-1 to be positioned at the L end and the switch 803-2 to be positioned at the L end. Accordingly, as shown in FIG8b , the path between the front camera 802-1 and the interface 01 of the low-power processor 801 is connected, and the path between the front camera 802-1 and the interface 03 of the low-power processor 801 is connected. At the same time, the path between the front camera 802-1 and the interface 01 of the low-power processor 801 is disconnected, and the path between the front camera 802-1 and the interface 03 of the low-power processor 801 is disconnected.

[0187] Still referring to FIG. 10 , illustratively, after the camera selection module switches to front camera 0 via the control selection circuit, the camera selection module may interact with front camera 0 based on the configuration information of front camera 0, transmitting control data (or control signals). For example, the camera selection module may output electrical signals, clock signals, etc. to front camera 0 via interface 02 of the low-power processor, although this application does not limit this.

[0188] For example, after enabling front camera 0, it captures images. The AO ISP processes the images captured by front camera 0 and outputs the processed images to the AO component. The camera recognition module and camera acquisition module in the AO component process and recognize the images. For other undescribed parts, please refer to Figure 9 and will not be repeated here.

[0189] In an embodiment of the present application, if the user triggers the camera application (or other applications), the low-power processor releases the control and use rights of the camera group, and the CameraHAL reclaims the control and use rights of the camera group. Figure 11 is an exemplary schematic diagram of the module interaction process. Please refer to Figure 11. The user clicks on the camera application. The camera application responds to the received user operation and sends a trigger indication message to the CameraHAL to instruct the call of the camera. In response to the instruction of the camera application, the CameraHAL sends a release indication message to the AO management service to instruct the release of the use rights of the camera component (which can also be understood as unregistration or unbinding). The AO management service sends a release indication message to the low-power module driver, and the low-power module driver sends a release indication message to the AO component.

[0190] The camera selection module calls the registration function (also known as the deregistration function) to deregister (i.e., unbind) the camera group. This can be understood as releasing control and usage rights over the camera components. It can also be understood as indicating that the path between the camera selection module and the selection circuit is currently unavailable. For example, the camera selection module still caches camera configuration information.

[0191] Exemplarily, after the Camera Selection Module releases the camera component, it sends a response message to the Low Power Module Driver to indicate that the camera component has been released. The Low Power Module Driver sends the response message to the AO Management Service. Based on the response message sent by the AO Management Service, the CameraHAL determines that the Camera Selection Module has released the camera component.

[0192] Exemplarily, CameraHAL may determine that the front camera-0 (i.e., the front camera of the external screen) needs to be called based on the acquired form of the electronic device (e.g., the folded state acquired in FIG11 ). CameraHAL sends a control signal to the camera driver to instruct the call of the front camera-0. Based on the instruction of CameraHAL, the camera driver controls the selection circuit to enable the front camera-0. It should be noted that the selection circuit between the camera driver and the camera group may be the same as the selection circuit between the Camera selection module and the camera group, that is, they share the same set of circuits. In other embodiments, the selection circuit between the camera driver and the camera group may also be independent, which is not limited in this application.

[0193] For example, after the front camera 0 is enabled, it captures an image. The ISP processes the image captured by the front camera and outputs it to the camera driver. The camera driver outputs the image to the CameraHAL, which then outputs it to the camera application. The specific interaction details between these modules can be found in existing technical embodiments and are not limited in this application.

[0194] Exemplarily, after the user closes the camera application, the CameraHAL again instructs the low-power side on the control and use rights of the camera group. FIG12 is an exemplary schematic diagram of module interaction. Referring to FIG12 , the camera application closes the camera application in response to the received user operation. The camera application sends a shutdown instruction to the CameraHAL to instruct the camera application to stop using the camera component. The CameraHAL sends AO registration indication information to the AO management service, wherein the AO registration indication information includes the current form of the electronic device (folded state or unfolded state). The AO registration indication information is used to instruct the low-power side to register the camera component (i.e., bind with the camera component). The AO management service sends an AO registration indication to the low-power module driver, and the low-power module driver sends an AO registration indication information to the AO component. It should be noted that the current form of the electronic device sent by the CameraHAL is the state of the electronic device that the CameraHAL last obtained from the display subsystem. For example, during the process of the camera application using the camera group, the electronic device may change from a folded state to an unfolded state. The CameraHAL can switch the corresponding camera based on the form conversion of the electronic device. After the camera application is closed, CameraHAL can determine that the most recently acquired state of the electronic device is the unfolded state, and CemaraHAL can

[0195] The camera selection module responds to the AO registration instruction by calling the registration function to register the camera component. This effectively binds the camera component, which can also be understood as restoring the path between the camera selection module and the selection circuit. Subsequent image processing steps can be found in Figures 9 and 10 and are not detailed here.

[0196] For example, as described above, after the Camera Selection Module deregisters from the camera component, it still caches the previously acquired camera configuration information. Accordingly, after the Camera Selection Module reregisters with the camera component, it can continue to use the cached camera configuration information to interact with the camera for data or control signals.

[0197] It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0198] In one example, FIG13 shows a schematic block diagram of an apparatus 1300 according to an embodiment of the present application. The apparatus 1300 may include: a processor 1301 and a transceiver / transceiver pin 1302 , and optionally, a memory 1303 .

[0199] The various components of the device 1300 are coupled together via a bus 1304, wherein the bus 1304 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all buses are referred to as bus 1304 in the figure.

[0200] Optionally, the memory 1303 may be used for instructions in the aforementioned method embodiment. The processor 1301 may be used to execute instructions in the memory 1303 and control the receiving pin to receive a signal and control the transmitting pin to send a signal.

[0201] The apparatus 1300 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0202] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0203] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0204] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0205] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0206] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0207] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A camera control method, characterized in that: Applied to an electronic device, the electronic device includes a first camera and a second camera, the first camera and the second camera are coupled to a first switch, the first switch is coupled to a low-power processor, the first switch is used to selectively conduct a first data path between the first camera and the low-power processor or a second data path between the second camera and the low-power processor, the method includes: In response to the received first user operation, the electronic device switches from the first state to the second state; It is detected that the electronic device switches from the first state to the second state, and the first switch turns on the first data path; Performing image recognition on the acquired first image and obtaining a first image recognition result; the first image is acquired by the first camera and is obtained after being processed by a low-power image signal processor ISP; In response to a received second user operation, the electronic device switches from the second state to the first state; It is detected that the electronic device switches from the second state to the first state, and the first switch turns on the second data path; Perform image recognition on the acquired second image and obtain a second image recognition result; the second image is acquired by the second camera and is processed by the low-power ISP.

2. The method according to claim 1, characterized in that Detecting that the electronic device switches from the first state to the second state, the first switch turns on the first data path, comprising: Based on the camera configuration information, determining that the camera corresponding to the second state is the first camera; Based on the configuration information corresponding to the first camera in the camera configuration information, opening the first data path; The first image is acquired through the first data path.

3. The method according to claim 2, characterized in that The first camera and the second camera are coupled to a second switch, the second switch is coupled to the low-power processor, and the second switch is used to selectively conduct a first control path between the first camera and the low-power processor or a second control path between the second camera and the low-power processor; It is detected that the electronic device switches from the first state to the second state, and the first switch turns on the first data path, further comprising: Based on the configuration information corresponding to the first camera, control the second switch to conduct the first control path; A control signal is transmitted to the first camera via the first control path.

4. The method according to claim 3, characterized in that Detecting that the electronic device switches from the second state to the first state, the first switch turns on the second data path, comprising: Based on the camera configuration information, determining that the camera corresponding to the first state is the second camera; Based on the configuration information corresponding to the second camera in the camera configuration information, the second data path is opened.

5. The method according to claim 4, characterized in that Detecting that the electronic device switches from the second state to the first state, the first switch turns on the second data path, comprising: Based on the configuration information corresponding to the second camera, control the second switch to conduct the second control path; A control signal is transmitted to the second camera via the second control path.

6. The method according to claim 3 or 5, characterized in that: The first switch is coupled to a first port of the low power consumption processor; The second switch is coupled to a second port of the low power processor.

7. The method according to any one of claims 1 to 6, characterized in that: The first state of the electronic device is a folded state, and the second state of the electronic device is an unfolded state.

8. The method according to claim 7, characterized in that The first camera is arranged on the outer screen of the electronic device, and the second camera is arranged on the inner screen of the electronic device.

9. The method according to any one of claims 1 to 7, characterized in that: The first camera is a front camera, and the second camera is a rear camera.

10. A control circuit, characterized in that: include: Low power processor; a first switch coupled to the low power consumption processor; A first camera and a second camera, wherein the first camera and the second camera are coupled to the first switch, and the first switch is used to selectively conduct a first data path between the first camera and the low-power processor or a second data path between the second camera and the low-power processor.

11. The control circuit according to claim 10, characterized in that: The first camera is coupled to a first terminal of the first switch, and the second camera is coupled to a second terminal of the first switch.

12. The control circuit according to claim 10, characterized in that: The first camera and the second camera are coupled to a second switch, the second switch is coupled to the low-power processor, and the second switch is used to selectively turn on a first control path between the first camera and the low-power processor or a second control path between the second camera and the low-power processor.

13. The control circuit according to claim 12, characterized in that: The first switch is coupled to a first port of the low power consumption processor; The second switch is coupled to a second port of the low power processor.

14. The control circuit according to any one of claims 10 to 13, characterized in that: The low-power processor is configured to send a first control signal to the first switch to control the first switch to conduct the first data path; The low-power processor is further used to send a second control signal to the first switch to control the first switch to conduct the second data path.

15. The control circuit according to any one of claims 10 to 14, characterized in that: The first camera is arranged on the outer screen of the electronic device, and the second camera is arranged on the inner screen of the electronic device.

16. The control circuit according to any one of claims 10 to 14, characterized in that: The first camera is a front camera, and the second camera is a rear camera.

17. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the method according to any one of claims 1 to 9.

18. A computer storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

19. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 9.

20. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 9.

Images