Display screen control method and electronic device

Abstract

The present application provides a method for controlling a display screen and an electronic device. The method includes: the electronic device starts detection based on the state of the display screen, the state of the proximity light sensor, and whether a user operation is received. During the detection process, if the electronic device detects the target biometric information, the electronic device can control the display screen from a semi-off state to a bright state. Thus, a method for controlling a display screen is provided, so that the user can light up the display screen of the electronic device without operating the electronic device. In addition, the electronic device starts detection when certain conditions are met, thereby effectively reducing power consumption while ensuring the real-time and high efficiency of the detection.

Description

Technical Field

[0001] The present application relates to the field of terminal devices, and in particular to a display screen control method and electronic device. Background Art

[0002] At present, when the screen of a mobile phone is on, if no user operation is received within a period of time, the mobile phone will enter the screen-off state. In other words, if the user expects the mobile phone to remain on the screen continuously, the user needs to touch or perform other operations on the mobile phone before the screen goes off. However, when the user is reading or in a scenario where it is inconvenient to operate the mobile phone, the manual triggering of the mobile phone to keep the screen on will affect the user experience. Summary of the invention

[0003] In order to solve the above problems, the present application provides a display screen control method and an electronic device. In the method, the electronic device can control the display screen to light up based on biometric information, thereby improving the user experience.

[0004] In a first aspect, the present application provides an electronic device. The electronic device includes: one or more processors, a memory and a fingerprint sensor. And one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: when the electronic device is in a screen-off state, a first user operation is received. In response to the received first user operation, the electronic device changes from a screen-off state to a screen-on state. When the electronic device is in a screen-on state, no user operation is received within a first time period, and the proximity light sensor of the electronic device meets a preset condition, and a first detection is started. In the first detection, the electronic device detects biometric information. At a first moment in the first detection, the electronic device changes from a screen-on state to a half-screen-off state. At a second moment in the first detection, the electronic device detects the first target biometric information, and the electronic device changes from a half-screen-off state to a screen-on state. In this way, the electronic device can monitor the status of each module and device of the electronic device based on the set conditions, so as to start detecting the biometric information only when the set conditions are met. Thereby avoiding the problem of excessive system power consumption caused by continuous detection, effectively reducing system power consumption, and saving battery power consumption. When the specified biometric information is detected, the display screen of the electronic device lights up. Thus, a method for controlling the screen lighting of an electronic device is provided, so that the display screen of the electronic device can be lit without user operation, thereby improving the user experience.

[0005] Exemplarily, the preset condition corresponding to the proximity light sensor is that the proximity light sensor is not blocked.

[0006] Exemplarily, if the proximity light sensor does not meet the preset condition, the electronic device does not initiate the first detection.

[0007] Exemplarily, the first detection may optionally be a detection window in the present application.

[0008] Exemplarily, the first user operation may be the user pressing a power button.

[0009] Exemplarily, the electronic device may collect biometric information through a camera.

[0010] Exemplarily, the frame rate at which the camera collects biometric information may be 5fps, thereby reducing system power consumption while ensuring detection efficiency and real-time performance.

[0011] According to the first aspect, when the computer program is executed by one or more processors, the electronic device performs the following steps: from the third moment when the electronic device changes from the off screen state to the on screen state to the first moment, no user operation is detected, and the electronic device changes from the on screen state to the half-off screen state at the first moment. In this way, after the electronic device turns on the screen, if no user operation is received within a specified time, the electronic device changes from the on screen state to the half-off screen state.

[0012] Exemplarily, when the screen is on, the brightness of the display screen is a first brightness. When the screen is half off, the brightness of the display screen is a second brightness, which is lower than the first brightness.

[0013] Exemplarily, when the display screen of the electronic device is in a bright screen state, the display screen may display the desktop or the application interface.

[0014] According to the first aspect, or any implementation of the first aspect above, the duration between the third moment and the first moment is the second duration.

[0015] Exemplarily, the user may set a second duration between the third moment and the first moment in the display and brightness interface.

[0016] Exemplarily, the second duration may be 30 seconds, 1 minute, 5 minutes, 10 minutes, etc.

[0017] According to the first aspect, or any implementation of the first aspect above, the duration between the start time of the first detection and the first moment is a third duration, and the third duration is less than the second duration. In this way, the electronic device can set the first duration based on different second durations, so that the duration between the start time of the detection and the moment when the screen is half-off is fixed.

[0018] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: detecting the first target biometric information and ending the first detection. In this way, after detecting the biometric information, the electronic device can end the detection to reduce power consumption.

[0019] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and the second detection is started. At the fourth moment in the second detection, the second target biometric feature is detected, and the electronic device is controlled to maintain the bright screen state. In this way, the electronic device can control the display screen of the electronic device to remain in the bright screen state by detecting biometric information, without the user having to repeatedly operate the mobile phone, thereby improving the user experience.

[0020] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device executes the following steps: detecting a second target biometric feature and ending the second detection.

[0021] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device maintains a screen-on state, does not receive a user operation within a first duration, and the proximity light sensor meets a preset condition, and starts a third detection; the third detection duration is the second duration. At the fifth moment within the third detection, the third target biometric feature is detected, and the electronic device is controlled to maintain a screen-on state. In this way, the electronic device can start the detection each time it detects that the set conditions are met, so as to save system power consumption while maintaining the efficiency of the detection.

[0022] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device executes the following steps: detecting a third target biometric feature and ending the second detection.

[0023] According to the first aspect, or any implementation of the first aspect above, the duration between the start time of the second detection and the fourth time is the fifth duration, and the duration between the start time of the third detection and the fifth time is the sixth duration; the sixth duration is different from the fifth duration.

[0024] Exemplarily, during the process of continuous gaze by the user, the fifth duration and the sixth duration may be equal, for example, both are 1 second.

[0025] Exemplarily, during the user's intermittent gaze, the fifth duration and the sixth duration may be different.

[0026] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and the fourth detection is started. At the sixth moment in the fourth detection, the user's operation of pressing the power button is detected, and the electronic device changes from a bright screen state to an off screen state. In this way, the user can continue to monitor the status of each module or device of the electronic device during the detection process, and when the power button of the electronic device is pressed, the display screen of the electronic device is turned off. And this detection ends.

[0027] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device executes the following steps: detecting that the user presses the power button, and ending the fourth detection.

[0028] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets the preset conditions, and the fifth detection is started. At the seventh moment in the fifth detection, a first voice call message is received, and the fifth detection ends. In this way, when the electronic device turns on a specific application, the electronic device may not need to perform detection, thereby further saving power consumption. In the example performance given, the specific application may be a voice application, an application that turns on the application lock, and the like. Exemplarily, after the electronic device receives a voice call, the screen turns off after a preset time period.

[0029] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device changes from a half-screen-off state to a bright-screen state, and no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and the sixth detection is started; within the sixth detection, biometric information is detected; at the eighth moment within the sixth detection, the electronic device changes from a bright-screen state to a half-screen-off state; at the ninth moment, the sixth detection ends, and the electronic device changes from a half-screen-off state to a screen-off state. In this way, if the electronic device does not detect biometric information, the electronic device ends the detection when the screen is off.

[0030] According to the first aspect, or any implementation of the first aspect above, the first target biometric characteristic information is human eye characteristic information. In this way, the present application can control the display screen to light up when it is determined that the user is looking at the screen by identifying the gaze of the human eye. This provides a smart way to control the display screen to light up, without the user having to operate the electronic device by hand, thereby improving the user experience.

[0031] According to the first aspect, or any implementation of the first aspect above, when the computer program is executed by one or more processors, the electronic device performs the following steps: displaying a smart perception interface, which includes a "watch the screen without turning off" option; receiving an operation of a user clicking on the "watch the screen without turning off" option, and determining that the "watch the screen without turning off" function is turned on; when the electronic device is in a bright screen state, no user operation is received within a first time period, and the proximity light sensor of the electronic device meets a preset condition, and the "watch the screen without turning off" option is turned on, a first detection is initiated. In this way, when the user turns on the "watch the screen without turning off" function, the electronic device can execute the display screen control method in this application.

[0032] In the second aspect, the present application provides a method for controlling a display screen. The method includes: when the electronic device is in the screen-off state, a first user operation is received; in response to the received first user operation, the screen-off state is changed to the screen-on state; when the electronic device is in the screen-on state, no user operation is received within a first time period, and the proximity light sensor of the electronic device meets a preset condition, a first detection is started; within the first detection, biometric information is detected; at a first moment within the first detection, the electronic device changes from the screen-on state to the semi-screen-off state; at a second moment within the first detection, the first target biometric information is detected, and the electronic device changes from the semi-screen-off state to the screen-on state.

[0033] According to the second aspect, the electronic device changes from a screen-on state to a half-screen-off state, including: from the third moment when the electronic device changes from the screen-off state to the screen-on state to the first moment, no user operation is detected, and the electronic device changes from the screen-on state to the half-screen-off state at the first moment.

[0034] According to the second aspect, or any implementation of the second aspect, the duration between the third moment and the first moment is the second duration.

[0035] According to the second aspect, or any implementation of the second aspect above, the duration between the start time of the first detection and the first time is a third duration, and the third duration is smaller than the second duration.

[0036] According to the second aspect, or any implementation of the second aspect above, the method further includes: detecting the first target biometric information and ending the first detection.

[0037] According to the second aspect, or any implementation of the second aspect above, the method also includes: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and a second detection is started; at a fourth moment within the second detection, a second target biometric feature is detected, and the electronic device is controlled to maintain the bright screen state.

[0038] According to the second aspect, or any implementation of the second aspect above, the method further includes: detecting a second target biometric feature and ending the second detection.

[0039] According to the second aspect, or any implementation of the second aspect above, the method also includes: the electronic device maintains a screen-on state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and a third detection is started; the third detection time period is the second time period; at a fifth moment within the third detection, a third target biometric feature is detected, and the electronic device is controlled to maintain the screen-on state.

[0040] According to the second aspect, or any implementation of the second aspect, the method further includes: detecting a third target biometric feature and ending the second detection.

[0041] According to the second aspect, or any implementation of the second aspect above, the method also includes: the time length between the start time of the second detection and the fourth time is the fifth time length, and the time length between the start time of the third detection and the fifth time is the sixth time length; the sixth time length is different from the fifth time length.

[0042] According to the second aspect, or any implementation of the second aspect above, the method also includes: the electronic device changes from a semi-screen-off state to a screen-on state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and a fourth detection is started; at a sixth moment within the fourth detection, an operation of a user pressing a power button is detected, and the electronic device changes from the screen-on state to the screen-off state.

[0043] According to the second aspect, or any implementation of the second aspect above, the method further includes: detecting that a user presses a power button, and ending the fourth detection.

[0044] According to the second aspect, or any implementation of the second aspect above, the method also includes: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and the fifth detection is started; at a seventh moment within the fifth detection, a first voice call message is received, and the fifth detection is ended.

[0045] According to the second aspect, or any implementation of the second aspect above, the method also includes: the electronic device changes from a semi-off screen state to a bright screen state, no user operation is received within a first time period, and the proximity light sensor meets a preset condition, and a sixth detection is started; within the sixth detection, biometric information is detected; at an eighth moment within the sixth detection, the electronic device changes from the bright screen state to a semi-off screen state; at a ninth moment, the sixth detection ends, and the electronic device changes from the semi-off screen state to the off screen state.

[0046] According to the second aspect, or any implementation of the second aspect above, the method further includes: the first target biometric feature information is human eye feature information.

[0047] According to the second aspect, or any implementation of the second aspect above, the method also includes: displaying a smart perception interface, which includes a "gaze without turning off the screen" option; receiving an operation of a user clicking on the "gaze without turning off the screen" option, and determining that the "gaze without turning off the screen" function is turned on; when the electronic device is in a screen-on state, no user operation is received within a first time period, and the proximity light sensor of the electronic device meets a preset condition, and the "gaze without turning off the screen" option is turned on, starting a first detection.

[0048] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0049] In a third aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0050] In a fourth aspect, the present application provides a computer program, comprising instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0051] In a fifth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the method in the second aspect or any possible implementation of the second aspect to control the receiving pin to receive a signal and control the sending pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0053] Figure 2A schematic diagram of the software structure of an electronic device is shown as an example;

[0054] Figure 3a A schematic diagram of an exemplary user interface;

[0055] Figure 3b A schematic diagram of an exemplary user interface;

[0056] Figure 3c A schematic diagram of an exemplary user interface;

[0057] Figure 4 A schematic diagram of a display screen state of an electronic device is shown as an example;

[0058] Figure 5 A schematic diagram of an exemplary user interface;

[0059] Figure 6 is a schematic diagram of module interaction shown as an example;

[0060] Figure 7 is a schematic diagram of a detection condition judgment process flow diagram shown as an example;

[0061] Figure 8 The following is a schematic diagram of an exemplary process of detecting the screen without turning off the screen while looking at it;

[0062] Fig. 9 is a schematic diagram of module interaction shown as an example;

[0063] Fig.10 is a schematic diagram of image frame acquisition shown as an example;

[0064] Fig.11 is a schematic diagram of an exemplary identification process;

[0065] Fig.12 is a schematic diagram of module interaction shown as an example;

[0066] Fig.13 The following is a schematic diagram of an exemplary non-off screen detection;

[0067] Fig.14 is a schematic diagram of exemplary detection of watching without turning off the screen;

[0068] Fig.15a is a schematic diagram of an application scenario shown as an example;

[0069] Fig.15b A schematic diagram of a display screen state of an electronic device is shown as an example;

[0070] Fig.16 is a schematic diagram of an exemplary identification process;

[0071] Fig.17The following is a schematic diagram of an exemplary non-off screen detection;

[0072] Fig.18 A schematic diagram of a display screen state of an electronic device is shown as an example;

[0073] Fig.19 The following is a schematic diagram of an exemplary non-off screen detection;

[0074] Fig. 20 The following is a schematic diagram of an exemplary non-off screen detection;

[0075] Fig.21 A schematic diagram of a display screen state of an electronic device is shown as an example;

[0076] Fig. 22 The following is a schematic diagram of an exemplary non-off screen detection;

[0077] Fig.23 The following is a schematic diagram of an exemplary non-off screen detection;

[0078] Fig.24 The following is a schematic diagram of an exemplary non-off screen detection;

[0079] Fig.25 Schematic diagram of the structure of the device shown as an example. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0082] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0083] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0084] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0085] Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. It should be understood that Figure 1 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits. It should be noted that in the embodiment of the present application, only the electronic device 100 is described as a mobile phone. The electronic device in the embodiment of the present application can also be a tablet, wearable device, TV, smart home device, car-mounted device, etc. with a display screen, and the present application does not limit it.

[0086] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0087] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0088] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0089] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0090] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0091] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0092] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0093] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0094] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0095] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0096] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0097] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0099] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0100] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0101] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0102] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0103] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0104] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0105] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. Exemplarily, the processor may control the state of the display screen. Optionally, in an embodiment of the present application, the state of the display screen includes a bright screen, a half-off screen, and an off screen.

[0106] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0107] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0108] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0109] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0110] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0111] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0112] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0113] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0114] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0115] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0116] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0117] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0118] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0119] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0120] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0121] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0122] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0123] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0124] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0125] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0126] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0127] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0128] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The electronic device 100 may receive a button input and generate a key signal input related to the user settings and function control of the electronic device 100. Exemplarily, in an embodiment of the present application, the processor may determine whether the non-screen-off condition is met based on the triggering of the proximity sensor, the touch sensor, and / or the power button. The specific details will be described in detail in the following embodiments.

[0129] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0130] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0131] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0132] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0133] The layered architecture of the electronic device 100 divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0134] The application layer can include a series of application packages.

[0135] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, screen-on application, etc. Exemplarily, the screen-on application (also referred to as the screen-on application) can be used to control the electronic device to perform screen-on detection, and can control the electronic device to keep the screen on, that is, in the unlocked state of the screen on.

[0136] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0137] like Figure 2As shown, the application framework layer may include a window manager, a view system, a resource manager, a perception service, an input management service (Input Manager Service), an AO (always on, permanently turned on, also known as a non-screen) service, a trusted execution environment (TEE), a power management service, etc.

[0138] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0139] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0140] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0141] The input management service is used to manage the input part of the entire system, including buttons (such as the HOME button, power button, etc.), keyboard, mouse, touch screen, etc.

[0142] The power management service is used to provide a system interface for the non-screen application, which allows the non-screen application to control the electronic device's display screen to remain on through the interface provided by the power management service.

[0143] AO service, also known as low-power AO service, is started after the non-off screen application is triggered by the perception service. It can identify whether there is human eye gaze through pre-stored detection algorithms, such as face recognition algorithm and human eye gaze recognition algorithm. After identifying human eye gaze, the recognition result is reported to the perception service.

[0144] The perception service is used for subscription management and interface provision, etc. Exemplarily, the perception service can be called by multiple applications or services. The perception service can establish a corresponding process for each application or service through subscription management, and transmit the data uploaded by the underlying module to different applications or services according to the process corresponding to the application or service. Exemplarily, the perception service can provide an interface for the upper-level application to call the underlying module according to the needs of the application. For example, the perception service can receive an indication signal from the non-screen application through the interface between the perception service and the non-screen application. In response to the indication of the non-screen application, the perception service can call the corresponding module or service through the interface between other modules or services, for example, calling the input management service, the power management service, etc. in response to the indication of the non-screen application.

[0145] TEE is used to isolate highly security-sensitive applications from general software environments, provide a dedicated trusted execution environment TEE, and protect the confidentiality, integrity and access rights of application resources and data. Applications executed on the TEE side are called trusted applications (TA), such as applications that perform key services such as signing, encryption and decryption calculations. Since TA runs in the trusted execution environment TEE, TA's deployment / upgrade operations need to strictly follow the security verification specifications of the TEE issuer (usually the terminal manufacturer), such as using digital signatures and other measures to ensure that each link of the TEE is truly trusted. Optionally, in an embodiment of the present application, TEE can be used to store images captured by the camera to protect user privacy. The AO service can detect graphics in the TEE based on a detection algorithm to determine whether the image includes a face and whether there is gaze. The specific details will be described in the following embodiments.

[0146] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0147] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0148] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0149] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0150] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0151] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0152] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0153] A 2D graphics engine is a drawing engine for 2D drawings.

[0154] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, Bluetooth driver, sensor driver, etc.

[0155] Understandably, Figure 2 The components included in the application framework layer, system library and runtime layer shown do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine some components, or split some components, or arrange the components differently.

[0156] Figure 3a 3, illustratively, the display and brightness interface 301 includes one or more options. The options include, but are not limited to, font size and thickness, display size, screen options 302, etc. It should be noted that due to the size display of the interface, the options displayed in the display and brightness interface 301 are only part of the options in the display and brightness settings.

[0157] The exemplary screen option 302 includes one or more options, including but not limited to: sleep option 303, color adjustment and color temperature, screen resolution and more display settings. Among them, the sleep option 303 is used to set the screen off time. For example, the sleep option 303 currently indicates that the sleep time is 10 minutes later, that is, the mobile phone will turn off the screen after 10 minutes when the screen is on and there is no user operation (including touch operation, key operation, etc.).

[0158] Still refer to Figure 3a , exemplarily, the user clicks the sleep 303 option. Please refer to Figure 3b In response to the received user click operation, the mobile phone displays a sleep option box 306 at the bottom of the display and brightness interface 301. It should be noted that the size and position of the sleep option box 306 are only illustrative examples and are not limited in this application.

[0159] Exemplarily, the sleep option box 306 includes, but is not limited to: a 15 second option, a 30 second option 306, a 1 minute option, a 5 minute option, a 10 minute option, and a cancel option 307. It should be noted that the values ​​and quantities of the options are only illustrative examples and are not limited in this application. If the user clicks the cancel option 307, the electronic device cancels the display of the sleep option box 306 in response to the received user operation.

[0160] Exemplarily, the user clicks on the 30 seconds option 306. Figure 3cIn response to the received user operation, the mobile phone displays the updated sleep parameters in the sleep option 303 in the display and brightness interface 301, that is, displays "after 30 seconds". For example, please refer to Figure 4 (1), the mobile phone displays the desktop. It should be noted that the mobile phone may enter the desktop from the lock screen interface, or from other applications. For example, the mobile phone enters the desktop in response to receiving the user unlocking operation. The mobile phone detects whether the user operation is received from the time of the last (or latest) user operation received. Optionally, as Figure 4 As shown in (2), if no user operation is received within 24 seconds, the mobile phone enters half-screen off mode. It can be understood that the display screen of the mobile phone becomes darker. Among them, the degree of dimming of the display screen can be set according to actual needs, and this application does not limit it. It should be noted that when the mobile phone is in half-screen off mode, the various applications and modules in the mobile phone operate normally. For example, in half-screen off mode, the mobile phone can still receive user operations, and, in response to the received user operations, the screen is turned on again.

[0161] Please refer to Figure 4 (3), illustratively, when the mobile phone is in the semi-off screen mode, if no user operation is detected within 6 seconds, the mobile phone enters the off screen mode. Optionally, the off screen mode can be understood as the mobile phone locks the screen, and the mobile phone display screen is off (ie, black screen).

[0162] However, in some scenarios, when the user is using the phone and cannot operate the phone, if the user expects the phone to light up, and because the phone is set to sleep mode, the phone will turn off the screen after 30 seconds of inactivity. For example, if the user is cooking and the user uses the phone to browse recipes. At this time, it is not convenient for the user to operate the phone at any time, and the phone will sleep after detecting that the user has not operated for 30 seconds. The solution to this problem is currently to change the sleep option, such as selecting a longer sleep time, or choosing not to sleep. But in fact, there are more scenarios where the user has started cooking and cannot operate the phone with his hands. At this time, it is not convenient for the user to change the sleep option. In this case, it will affect the user experience.

[0163] In an embodiment of the present application, a method for achieving non-turning off the screen is provided, which can control the mobile phone to keep the screen on when the non-turning off conditions are met, so as to improve the user experience. Figure 5 is a schematic diagram of an exemplary user interface. Figure 5 For example, the smart perception interface 501 includes but is not limited to a smart screen-on option box and an air gesture option box. Optionally, the smart screen-on option box includes but is not limited to: a gaze screen-on option 502, a sound screen-on option 503, and a shield screen-on option 504.

[0164] For example, if the user can click the "attention without turning off the screen" option 502 to trigger the "attention without turning off the screen" function, in the embodiment of the present application, the mobile phone responds to the received user operation and starts the "attention without turning off the screen" function (also referred to as "attention without turning off the screen service"), and the mobile phone controls the mobile phone to keep the screen on based on detecting the user's gaze. The specific embodiments will be described in the following embodiments and will not be described here in detail.

[0165] Exemplarily, if the user clicks on the sound without turning off the screen option 503, in an embodiment of the present application, the mobile phone responds to the received user operation and starts the sound without turning off the screen function. The mobile phone will control the mobile phone to keep the screen on based on the detected sound.

[0166] For example, if the user clicks on the block without turning off the screen option 504, in an embodiment of the present application, the mobile phone responds to the received user operation and starts the block without turning off the screen function. The mobile phone will control the mobile phone to keep the screen on based on the blockage status of the proximity light sensor.

[0167] It should be noted that in the embodiments of the present application, only the examples of watching the screen, sound and occlusion are used for explanation. That is to say, the mobile phone can determine whether to keep the screen on based on whether it is being watched, whether a specific sound is heard, and whether it is detected to be blocked. In other embodiments, the mobile phone can also be based on other conditions, such as whether a blink is detected, whether a specific gesture is detected, whether a tap is detected, etc., to control the screen on state of the display, which is not limited in this application. Accordingly, the smart screen on option box can display options corresponding to different conditions, such as the option of blinking and not turning off the screen.

[0168] It should be further explained that, as described above, the smart screen-on option box optionally includes multiple options, and the user can choose not to turn on all of them, or can choose to turn on one or more options, which is not limited in this application. For example, if the user turns on the gaze screen-on option 502 and the sound screen-on option 503, the mobile phone can control the mobile phone to keep the screen on during the detection process after detecting the user's gaze and / or receiving a specified sound.

[0169] Figure 6 This is an exemplary diagram of module interaction. Figure 6, illustratively, take the example of a user clicking on the "stare at the screen without turning off" option 502. The "screen-not-turning-off application" (also referred to as an "intelligent screen-not-turning-off application") receives the user operation. The "screen-not-turning-off application" sends an indication message to the perception service through the interface between the perception service, which is used to instruct the perception service to start the judgment process of the detection conditions. Optionally, the "judgment process of the detection conditions" in the embodiment of the present application optionally refers to a conditional judgment on whether it is necessary to start the "stare at the screen without turning off" detection. Only when all the detection conditions are met, the mobile phone will start the "stare at the screen without turning off" detection. If the detection conditions are not met, the "stare at the screen without turning off" detection will not be started. It can be understood that the detection condition is a prerequisite for executing the "stare at the screen without turning off" detection. Sound detection is similar to occlusion detection and will not be repeated here.

[0170] For example, after the perception service is started in response to the instruction of the non-off screen application, the state of the proximity light sensor, touch sensor and button can be obtained through the input management service. In addition, the perception service can also obtain the state of the display screen through the display driver. The perception service can combine the obtained states of each module (or device) to determine whether the detection condition is met.

[0171] Combination Figure 6 , Figure 7 This is an exemplary test condition judgment process. Please refer to Figure 7 ,The specific process of the perception service to determine whether the detection conditions are met includes:

[0172] S701, the perception service determines whether to turn on the screen.

[0173] For example, please refer to Figure 6 , the perception service responds to the instruction of the non-screen-off application and obtains the state of the display screen through the display driver. Optionally, the display screen state may include screen-off, half-off and screen-on, wherein the screen-on further includes screen-lock and screen-unlock. For example, when the mobile phone is in the screen-off state, it receives the operation of the user clicking the power button, and after the mobile phone is unlocked by face recognition, the desktop is displayed, and the display screen is in the screen-on state. Exemplarily, when the mobile phone is in the screen-off state, it receives the operation of the user clicking the power button, and when the face recognition result is unsuccessful, the lock screen interface is displayed, and the display screen is in the screen-on state. Another example is given, in the lock screen mode (display screen is on), if no user operation is received within a specified time (for example, 20s, which can be set according to actual needs and is not limited in this application), the lock screen interface changes to a half-off state, and after no user operation is received within 3s (which can be set according to actual needs and is not limited in this application), it enters the screen-off mode, that is, the display screen is off.

[0174] Please continue to refer to Figure 6For example, the perception service can obtain the state of the display screen through the display driver to determine whether the state of the display screen is on. In one example, if the display screen is currently in the on screen state, S702 is executed. In another example, if the display screen is currently in the off screen state or the semi-off screen state, it is determined that the detection condition is not met, and this judgment process ends.

[0175] Optionally, in an embodiment of the present application, the perception service may trigger the display driver to report the status of the display screen in real time after being started. For example, the perception service may send an instruction message to the display driver to instruct the display driver to report the status of the display screen to the perception service.

[0176] S702: The perception service determines whether the proximity light sensor is blocked.

[0177] For example, please refer to Figure 6 The perception service may send instruction information to the input manager to instruct the input manager to obtain the status of the proximity light sensor and the touch sensor from the sensor driver, and to obtain the status of a key (such as a power button) from the key driver. In response to the instruction of the perception service, the input management service obtains the status of the proximity light sensor and the touch sensor in real time through the sensor driver, and obtains the status of the key in real time through the key driver.

[0178] For example, if the user places the phone on the table or puts the phone in the pocket, the proximity light sensor will be blocked. After the proximity light sensor detects the blockage, it sends a detection signal to the sensor driver. The sensor driver can determine that the proximity light sensor is blocked based on the received detection signal, that is, there is a proximity light sensor blockage event. The sensor driver can output the proximity light sensor blockage event to the input management service. In response to the received proximity light sensor blockage event, the input management service determines that the proximity light sensor is blocked. The input management service indicates to the perception service that the proximity light sensor is blocked.

[0179] Exemplarily, if the proximity light sensor is not blocked, the proximity light sensor may not output a detection signal, that is, the sensor driver determines that there is no proximity light blocking event when it does not receive a detection signal sent by the proximity light sensor. Figure 7 The "end" step shown is optionally when the perception service determines that a certain condition is not satisfied, the perception service ends the current judgment process. After the current process ends, the perception service re-executes S701, which will not be repeated below.

[0180] S703, the perception service determines whether it is unlocked.

[0181] For example, please refer to Figure 6, the perception service can obtain the current state of the display screen through the display driver. When it is determined that the display screen is on and the proximity light sensor is not blocked, the perception service can further obtain whether the display screen is currently in an unlocked state through the display driver. It should be noted that the steps executed by the modules in the embodiments of the present application are only illustrative examples. For example, in other embodiments, the perception service can also obtain the display state or unlock state of the display screen through other modules, which is not limited in this application.

[0182] In one example, if the perception service obtains that the mobile phone is currently unlocked, for example, the mobile phone currently displays the desktop, or the mobile phone currently displays a game application (it can also be other applications, which is not limited in this application), S704 is executed. In another example, if the perception service determines that the mobile phone is not unlocked, for example, the mobile phone screen is on but in lock screen mode, this judgment process ends.

[0183] S704, the perception service determines whether to start the set application.

[0184] Exemplarily, the perception service can retrieve information about the foreground application. Exemplarily, the "foreground application" may optionally be an application that is currently running and displayed on the display. For example, the display interface of a mobile phone currently displays a video application and a document, and both the document and the video application are foreground applications. Of course, in some embodiments, the "foreground application" may optionally be an application that is currently running and displayed on the display, as well as the application that the user last operated. This application does not limit this.

[0185] Exemplarily, the perception service can determine whether the foreground application is a designated application based on the information of the foreground application obtained. In an embodiment of the present application, when the mobile phone starts some designated applications, the mobile phone can determine that the detection conditions are not met, that is, there is no need to perform subsequent non-screen detection. For example, if the foreground application is a voice call application, correspondingly, when the user is using a voice call, even if the mobile phone turns off the screen, it will not affect the voice call application. Therefore, for voice call applications, there is no need to start the non-screen detection process. Of course, in addition to voice calls, the designated application may also include one or more other applications (or functions, or services). For example, if the mobile phone is running a video application and the video application is displayed in full screen. At this time, the video application will turn on the application lock when it is displayed in full screen. When the application lock is turned on, the mobile phone will continue to keep the screen on and not locked. For another example, if the mobile phone is running a video conferencing application, optionally, the video conferencing application automatically turns on the application lock after startup. After detecting that the video conferencing application is started, the perception service ends this detection process. It should be noted that the designated applications in the embodiments of the present application are only illustrative examples. In other embodiments, the designated applications may also be applications that require the screen to remain in a continuously lit and unlocked state, and this application does not limit this.

[0186] In one example, if the perception service detects that the designated application is started, the current process ends. Optionally, after the designated application ends running, the perception service starts executing from S701.

[0187] In another example, if the perception service detects that the specified application is not started, S705 is executed.

[0188] It should be noted that the order of determining the conditions of S702 to S704 is not limited in the embodiment of the present application. For example, the perception service may first determine whether the device is unlocked, and then determine whether the proximity light sensor is blocked. In the embodiment of the present application, the perception service may also detect each condition at the same time, which is not limited in the present application.

[0189] S705, the perception service determines whether there is any user operation within a preset time period.

[0190] For example, please refer to Figure 6 As described above, the perception service can determine whether there is a user operation through the input management service. User operations include, but are not limited to, touching the display screen and pressing the power button. For example, if the user clicks the touch screen, the touch sensor will detect the user's click operation and output a detection signal to the sensor driver. The sensor driver responds to the received detection signal to determine that there is a touch event. The sensor driver outputs the touch event to the input management service. Based on the received touch event, the input management service determines that there is currently a user touch operation, and sends an indication message to the perception service to indicate that there is currently a user touch operation. Accordingly, the perception service determines that there is a user touch operation, and this detection process ends. For another example, if the user presses the power button, the key driver will detect that the key is pressed, and the key driver outputs the power button pressing event to the input management service. The input management service responds to the received event and sends an indication message to the perception service to indicate that there is currently a power button pressing operation. Accordingly, the perception service determines that there is an operation of the user pressing the power button, and this detection process ends.

[0191] For example, the sensing service can pre-set the user operation detection time. Figure 3bAs shown, in an embodiment of the present application, the mobile phone can provide several sleep options, including a 15-second option, a 30-second option 306, a 1-minute option, a 5-minute option, and a 10-minute option. Correspondingly, depending on the sleep duration, the corresponding user operation detection duration is also different. For example, taking the sleep duration of 30 seconds as an example, the mobile phone starts from the last operation in the screen-on state. If no user operation is received within 24 seconds, the mobile phone enters the half-screen-off mode. In the half-screen-off mode, if no user operation is received within 6 seconds, the mobile phone enters the screen-off mode. It should be noted that the various time settings described in the embodiments of the present application are only illustrative examples, which are not limited in this application and will not be repeated below. Exemplarily, in the scenario where the sleep duration is 30s, the mobile phone executes S701 to S704, that is, when the perception service detects that the mobile phone is in the bright screen state, unlocked state, no specified application is started, and the proximity light sensor is not blocked, the perception service starts to monitor whether the user operation is received, and continues to monitor for 19s, that is, when the sleep duration is 30s, the corresponding preset duration (that is, the duration of detecting whether the user operation is received) is 19s. Optionally, if the perception service does not receive any user operation indicated by the input management service within 19s, the perception service determines that no user operation is received within the preset duration, and starts the non-off screen detection process, that is, executes S706. Optionally, if the perception service detects user operation within 19s, this process ends and S701 is repeated. Optionally, if the perception service detects that the proximity light sensor is so within 19s, for example, the user buckles the front of the mobile phone on the table or puts the mobile phone in the pocket, the perception service ends this process and S701 is repeated. It can be understood that when the screen is in sleep mode for 30 seconds, the perception service starts to perform non-screen detection 5 seconds before entering the semi-screen shutdown mode, thereby ensuring the real-time detection while reducing system power consumption.

[0192] S706, the perception service starts detection.

[0193] For example, refer to Figure 6 , after the perception service detects that the detection conditions are met, that is, after all the conditions in S701 to S705 are met, the perception service sends an indication message to the non-screen application to indicate that the detection conditions are met. Exemplarily, the non-screen application responds to the indication message sent by the perception service and determines that all the detection conditions are met, and then starts the non-screen detection. The non-screen application can send an indication message to the perception service to instruct the start of the non-screen detection. Optionally, the non-screen application can indicate to the perception service which specific non-screen detection to enable based on the user's choice. It should be noted that for different detection types, the corresponding detection conditions may be the same or different. For example, if the user Figure 5If the selection in the screen-on-gaze option is included, the screen-on-gaze application sends an indication message to the perception service to instruct the perception service to start the screen-on-gaze detection. Figure 5 If the selection in includes the option of sound without screen off, the screen-on application sends an instruction message to the perception service to instruct the perception service to start the sound without screen off detection. Examples are not given one by one here.

[0194] S707, the perception service determines whether the conditions for not turning off the screen are met.

[0195] As mentioned above, different on-screen detections may correspond to different on-screen conditions. The following uses the on-screen gaze scenario as an example to illustrate. Figure 8 For example, after the perception service starts the screen-on detection in response to the instruction of the screen-on application, the detection is performed based on multiple conditions corresponding to the screen-on-focus function. The screen-on-focus condition includes but is not limited to at least one of the following:

[0196] 1) During the test, no voice calls were made;

[0197] 2) During the detection period, no touch operation is detected.

[0198] 3) During the detection period, no screen lock operation was detected.

[0199] 4) During the detection period, the foreground application does not activate the application lock.

[0200] 5) During the detection period, the proximity light sensor is not continuously blocked.

[0201] 6) During the detection period, gaze is detected.

[0202] It should be noted that the above conditions are only used as examples in the embodiments of the present application. In other embodiments, more or fewer conditions may be included, and the present application does not limit them.

[0203] It should be further explained that the perception service may be set with a detection duration. In an embodiment of the present application, different sleep durations may have the same or different corresponding detection durations. For example, if the sleep duration is 15s, the corresponding detection duration may optionally be 7s. If the sleep duration is 30s, the corresponding detection duration may optionally be 11s. If the sleep duration is 1 minute, 2 minutes, 5 minutes or 10 minutes, the corresponding detection duration may be 11s. The above values ​​are only illustrative examples and can be set according to actual needs. This application does not limit them.

[0204] Exemplarily, in the embodiment of the present application, the perception service performs detection based on the above conditions. When any of the conditions 1) to 5) is not met, the detection process ends and S701 is repeated. When condition 6) is met, that is, the perception service determines that there is gaze, the perception module notifies the screen-on application, and the gaze-on-screen condition is met. The screen-on application can control the phone to turn on the screen.

[0205] In a possible implementation, the conditions for the sound not to turn off the screen include but are not limited to at least one of the following:

[0206] 1) During the test, no voice calls were made;

[0207] 2) During the detection period, no touch operation is detected.

[0208] 3) During the detection period, no screen lock operation was detected.

[0209] 4) During the detection period, the foreground application does not activate the application lock.

[0210] 5) During the detection period, the proximity light sensor is not continuously blocked.

[0211] 6) During the detection period, a specified sound is detected.

[0212] For example, the description of conditions 1) to 5) can be referred to above and will not be repeated here. Optionally, the specified sound may have a voice command with specified content, for example, when a user points at the mobile phone and says "turn on the screen", the mobile phone may keep the screen on in response to the detected voice.

[0213] In another possible implementation, the screen-not-turning-off condition includes but is not limited to at least one of the following:

[0214] 1) During the test, no voice calls were made;

[0215] 2) During the detection period, no touch operation is detected.

[0216] 3) During the detection period, no screen lock operation was detected.

[0217] 4) During the detection period, the foreground application does not activate the application lock.

[0218] 5) During the detection period, the proximity light sensor is not continuously blocked.

[0219] 6) During the detection period, the proximity light sensor is blocked a specified number of times.

[0220] Exemplarily, the description of conditions 1) to 5) can refer to the above and will not be repeated here. Exemplarily, the proximity light sensor is blocked a specified number of times, which can optionally be the proximity light sensor is blocked once or multiple times, and the duration of each blockage is less than the set threshold (for example, 1s). For example, the user can wave his palm at the display screen. During the waving process, the proximity light sensor is blocked multiple times, and the duration of each blockage is less than the set threshold, then the requirement of condition 6) is met. It should be noted that in the embodiment of the present application, the detection of watching without turning off the screen is used as an example for explanation. The implementation method of the sound without turning off the screen and the blocking without turning off the screen can refer to the specific content of the detection of watching without turning off the screen, and this application will no longer give examples one by one.

[0221] The following takes the detection of the condition of looking at the screen without turning it off as an example for explanation. Figure 8 , Fig. 9 This is an exemplary diagram of module interaction. Fig. 9 For example, the perception service responds to the instruction of the screen-on application and starts to detect the condition of the screen-on-gaze. The perception service can obtain the status of the proximity light sensor, the touch sensor, and the power button through the input management service to determine whether there is a user operation. The specific description can be referred to above and will not be repeated here.

[0222] In one example, if the perception service detects the presence of touch operations, lock screen operations, etc. based on the information input by the input management service, or detects that the proximity light sensor is continuously blocked, the perception service determines that the condition of watching the screen without turning it off is not met, and re-executes S701. For example, if during the detection period, the user buckles the mobile phone on the table so that the proximity light sensor is blocked, the perception module stops detecting the condition of watching the screen without turning it off and re-executes S701 after detecting that the proximity light sensor is continuously blocked (for example, for more than 3 seconds, which can be set according to actual needs and is not limited in this application). Exemplarily, if during the detection period, the perception service detects that the user presses the power button or touches the screen, the perception service stops detecting the condition of watching the screen without turning it off and re-executes S701.

[0223] In another example, the perception service can monitor whether the voice application is started, and the perception service can monitor whether the foreground application has opened the application lock. The specific description can be referred to above and will not be repeated here.

[0224] Please continue to refer to Fig. 9, exemplarily, the following is explained by taking the case where the perception service determines that all of the above conditions 1) to 5) are satisfied during detection. The perception service responds to the information received from the non-screen application instructing the start of detection, and the perception service starts detection. During the detection period, in addition to continuously detecting whether conditions 1) to 5) are satisfied, the perception service instructs the AO service to perform gaze detection. In response to the instruction of the perception service, the AO service instructs the camera driver to start the camera and output the image captured by the camera to the TEE. In response to the instruction of the AO service, the camera driver obtains the image captured by the camera from the camera and outputs the captured image to the TEE.

[0225] Please refer to Fig.10 , for example, the detection duration is still 11s as an example. In the embodiment of the present application, the camera driver can output image frames to the TEE at a frame rate of 5fps (i.e., the number of image frames collected per second), thereby reducing the occupancy of the TEE while ensuring timeliness. Among them, 5fps can optionally mean that the camera outputs 5 image frames to the TEE per second. This value is only an illustrative example and is not limited in this application.

[0226] Still refer to Fig.10 , illustratively, in an embodiment of the present application, the frame rate of the camera is optionally 60fps, that is, the number of image frames collected per second is optionally 60. The camera driver can obtain an image frame from the camera every 200ms, and output it to the TEE, and last for 11s (i.e., the detection duration). Optionally, in other embodiments, the camera driver can also control the frame rate of camera acquisition. For example, the camera can capture images based on a frame rate of 5fps, that is, capture an image frame every 200ms and output it to the camera driver. The frame rate is only an illustrative example and is not limited in this application.

[0227] Still refer to Fig. 9 For example, TEE obtains image frames captured by the camera from the camera driver. For example, TEE can obtain 5 image frames from the camera driver every 1 second. The AO service can recognize images in the TEE based on a preset detection algorithm (also called a recognition algorithm).

[0228] Fig.11 This is a schematic diagram of an exemplary recognition process. Fig.11, exemplarily, the camera driver outputs the image frame to the TEE. The AO service identifies the image frames in the TEE based on the face detection algorithm to detect whether the image frame includes a single face. Optionally, if the image frame does not include a face or includes multiple faces, the AO service determines that the recognition of the image frame has failed and continues to process the next image frame. Optionally, in an embodiment of the present application, after the AO service recognizes an image frame, the recognized image frame is deleted from the TEE. Optionally, the AO service may also delete the 5 recognized image frames after every 5 (or other numbers, not limited in this application) image frames are recognized, thereby reducing TEE occupancy and further improving the security of user privacy.

[0229] Exemplarily, if the AO service detects that an image frame includes a single face, the AO service identifies whether the eyes in the single face are looking based on a gaze detection algorithm. For example, suppose the user is facing the phone, but the user's eyes are looking at other places outside the phone. Accordingly, the image captured by the camera includes the user's face, and the AO service identifies a single face based on the face detection algorithm. During the gaze detection process, the AO service recognizes that the user is not looking at the phone, then it is determined that this recognition has failed, and the next image frame will continue to be processed. Exemplarily, if the user is facing the phone and the user is looking at the display screen. Accordingly, the image captured by the camera includes the user's face, and the AO service identifies a single face based on the face detection algorithm. During the gaze detection process, the AO service recognizes that the user is looking at the phone, then it is determined that the gaze detection is successful. It should be noted that the specific details of the algorithm can be referred to the existing technology, and this application will not repeat them.

[0230] Still refer to Fig. 9 Exemplarily, the AO service determines that the gaze detection is successful. At the same time, if the conditions 1) to 5) described above are all met, the AO service determines that the gaze-on-screen condition is met. The AO service sends an indication message to the screen-on-screen application to indicate that the gaze-on-screen condition is met.

[0231] Please refer to Fig.12 , exemplarily, the non-screen-off application determines that the non-screen-off condition is met in response to the received indication information of the perception service. The non-screen-off application sends indication information to the perception service to instruct the screen to be turned on. In response to the received indication information, the perception service sends indication information to the power management service through the interface provided by the power management service to instruct the display screen to be controlled to be turned on. The power management service responds to the indication of the perception service and controls the display screen to be turned on through the display driver. In an embodiment of the present application, "the display screen is turned on" can optionally mean that the display screen is currently in a turned-on state, and the display screen is controlled to continue to be in a turned-on state. Alternatively, the display screen may be currently in a semi-turned-off state, and the display screen is controlled to enter a turned-on state. Specific examples will be described in detail in the following embodiments.

[0232] Exemplarily, after controlling the display screen to light up, the perception service repeatedly executes S705, that is, continues to monitor whether there is any user operation within the preset time, and after detecting that there is no operation within the preset time, continues to execute the watching without turning off the screen monitoring process. Exemplarily, if the perception service determines that the watching without turning off the screen condition is not met, this process ends, and the perception service repeats S701.

[0233] The following is a detailed description of the non-off-screen solution in the embodiment of the present application in combination with a specific example. For example, the user chooses to turn on the non-off-screen function. Fig.13 This is an exemplary diagram of the on-screen detection. Fig.13 For example, at time T0, the mobile phone enters the desktop from the lock screen mode (or off screen mode) in response to the unlock operation received from the user. This scenario is only an example, and the scenario where the mobile phone enters the bright screen state can also be that the mobile phone enters the desktop after exiting other applications; or the user has been using the mobile phone, and from time T0, the mobile phone is in the bright screen state, and there is no user operation.

[0234] For example, at time T0, the perception module executes S701 to S704. After all of S701 to S704 are satisfied, the perception module continues to execute S705, that is, to detect whether a user operation is received within a preset time (e.g., 19 seconds). For the specific detection process, please refer to Figure 6-7 The relevant contents are not repeated here. In this example, the case where the perception service does not detect a user operation within 19 seconds is used as an example for explanation.

[0235] At T1, combined Fig. 9 , the perception service responds to the instruction of the non-screen-off application and starts the staring non-screen detection. It can also be understood that the perception service starts the detection window, and detects whether the staring non-screen-off condition is met within the detection window. Optionally, taking the sleep duration of 30s as an example, the time difference between the starting edge of the detection window (or rising edge, i.e., moment T1) and the start moment of the half-screen off (i.e., moment T2) is 5s, and the duration of the detection window is 11s, that is, the end edge of the detection window (i.e., the falling edge, i.e., moment T3) is the same as the end moment of the half-screen off mode. For the specific detection process, please refer to Figures 9 to 11 The relevant content will not be repeated here. That is to say, the perception service can start detection 5 seconds before the phone enters the half-screen off state, and continue detection when the phone remains in the half-screen off state. It should be noted that the various moments and durations in the embodiments of this application are only illustrative examples, and this application does not limit them, and will not be repeated below.

[0236] For example, in Fig.13In the scenario, for example, the user has not been looking at the phone. Accordingly, the perception service does not detect the user's gaze during the detection period (i.e. within 11 seconds), which means that the condition of not turning off the screen while looking is not met. Fig. 9 For example, the perception service may send an instruction message to the AO service to instruct the AO service to stop gaze detection. In response to the instruction of the perception service, the AO service instructs the camera to stop collecting images through the camera driver. Of course, if the camera driver and the camera are being called by other applications, the camera driver and the camera may continue to work, and the camera driver stops transmitting image frames to the TEE.

[0237] Still refer to Fig.13 For example, the mobile phone maintains the semi-off screen state for 6 seconds and enters the off screen mode, that is, displays the off screen interface. The current detection process of the perception service ends, and S701 is re-executed, that is, after the mobile phone turns on the screen again, the above steps are repeated.

[0238] Fig.14 This is another schematic diagram of an exemplary non-off screen detection. Fig.14 , for example, the description of time T0 to T2 can be referred to above, and will not be repeated here. At time T3, the user is facing the mobile phone and looking at the mobile phone display. At this time, the mobile phone is in a semi-off state, such as Fig.15a shown.

[0239] Combination Fig. 9 , exemplarily, the camera driver outputs the image frames captured by the camera to the TEE. Please refer to Fig.16 For example, at time T3, which can also be understood as the 8th second after the start of this detection window, the image frame 3 captured by the camera includes the user's face image. Fig. 9 For example, during the detection of image frame 3, the AO service determines that image frame 3 includes a single face, and based on the gaze detection algorithm, determines that the user is looking at the image frame 3. The AO service can indicate to the perception service that the user is looking at the image frame 3. In response to the instruction of the AO service, the perception service sends an instruction message to the screen-on service to indicate that the screen-on condition is met. The screen-on application responds to the instruction of the perception service and determines that the screen-on condition is met. Accordingly, the screen-on application can instruct the perception service to control the display screen to light up. The specific interaction process of each module can be referred to. Fig.12 The relevant description in is not repeated here. Fig.15bAs shown, exemplarily, at time T4, the display screen of the mobile phone changes from a semi-off state to a bright screen state. It should be noted that the delay (including camera acquisition delay and algorithm delay) in the embodiment of the present application is only an illustrative example, and optionally, the algorithm delay can be in milliseconds. For example, the user looks at the screen at time T3, and the camera starts to collect image frame 3 200ms (or less than 200ms) after T3, and image frame 3 includes the user image. The AO service recognizes image frame 3, and after recognizing the human eye, the perception service controls the display screen to light up at time T4 based on the recognition result of the AO service. Optionally, the interval between time T3 and time T4 can be optionally in milliseconds. The delay shown in the drawings of the embodiments of the present application is only a schematic distance, and this application does not limit it.

[0240] For example, the user looks at the display screen at time T3, but due to delays in camera acquisition and algorithms, the display screen turns on at time T4 (1 second apart from time T3).

[0241] Fig.17 FIG. 1 is another exemplary schematic diagram of a non-off screen detection. For example, in this scenario, the user intermittently stares at the mobile phone screen as an example. It should be noted that: Fig.17 The duration and time of user gaze shown in are only illustrative examples and are not limited in this application. Fig.17 For example, the screen of the mobile phone is in the off-screen mode from time T0. Of course, the mobile phone may be in the off-screen mode before time T0, or the user may press the power button at time T0 to put the mobile phone into the off-screen mode. This application does not limit this. At time T1, if Fig.18 As shown in (1), the user presses the power button. Fig.18 As shown in (2), the mobile phone responds to the received user operation, displays the desktop after unlocking through face recognition, and at this time, the mobile phone is in a bright screen state.

[0242] Still refer to Fig.17 , for example, at time T0-T1, the sensing service is Figure 6-7The process shown is executed, where, since the mobile phone is always in the screen-off state from T0 to T1, the perception service determines that the detection condition is not met and repeats S701. Exemplarily, when the mobile phone is in the screen-off mode, that is, when the display screen is in the screen-off state, the user clicks the power button, and the mobile phone performs face recognition unlocking on the user. At T1, the mobile phone is successfully unlocked and the desktop is displayed. It should be noted that the mobile phone in the embodiment of the present application changes from the screen-off mode to the desktop mode, or it can be understood that the way from the screen-off state to the screen-on state is only a schematic example. For example, in other embodiments, the user can also change the mobile phone from the screen-off state to the screen-on state (referring to the screen-on state after unlocking) by sliding unlocking and other methods. The perception service obtains from the display driver that the state of the display screen is the screen-on state, and the mobile phone is in the unlocked state (that is, the desktop mode), the proximity light sensor of the mobile phone is not blocked, and, no application with an application lock is running, that is, S701 to S704 all meet the conditions, and the perception service continues to monitor for 19s from T1 to monitor whether there is a user operation within 19s. The embodiment of the present application is described as an example in which no user operation is detected. That is, the perception service determines that the detection condition is met, and at time T2, the perception service sends an indication message to the non-off-screen application to indicate that the detection condition is met. In response to the indication of the perception service, the non-off-screen application sends an indication message to the perception service to instruct the perception service to perform subsequent gaze non-off-screen detection. In response to the indication of the non-off-screen application, the perception service opens a detection window, such as Fig.17 As shown, at time T2, the perception service starts detection and opens the detection window. The description of the detection window can be found above and will not be repeated here. It should be noted that at time T1-T2, although the user intermittently looks at the display screen, the perception service does not start the detection window at time T1-T2, so the perception service cannot obtain the user's gaze situation.

[0243] For example, from time T2 to time T3, the user does not look at the display screen. Accordingly, the sensing module fails to detect the user's gaze. The specific detection process can be referred to Figure 8-Figure 9The description is not repeated here. Exemplarily, at time T3 (wherein the interval between time T3 and time T2 is 3s), the user looks at the display screen. The perception service detects the user's gaze, determines that the condition of not turning off the screen while looking is met, and the perception service notifies the application that does not turn off the screen. At time T4 (wherein the interval between time T4 and time T3 is 1s), the perception service responds to the instruction of the application that does not turn off the screen, and controls the display screen to light up through the power management module. It can be understood that if the perception service does not detect the user's gaze at time T4, the display screen will enter a semi-off state after the screen-on duration reaches 24s. At time T4, due to the detection of user gaze, the display screen maintains the screen-on state, and the duration of the screen-on state is re-timed (duration is 24s). That is to say, if no user gaze or user operation is detected after the next 24s (starting from time T4), the display screen will enter a semi-off state.

[0244] Please continue to refer to Fig.17 , exemplarily, at time T4, the display screen restarts the screen-on timing, and the timing duration is 24s. The perception service starts execution from S705, that is, starting from time T4, it detects again whether there is any user operation within 19s. Exemplarily, in the next detection window, the perception service detects the user's gaze at the 4th second after the detection window is opened, and at the 5th second, it can also be understood that before the display screen is about to be half-turned off, the display screen is controlled to continue to remain in the screen-on state. For specific descriptions, please refer to the above and will not be repeated here.

[0245] Fig.19 FIG. 1 is another exemplary schematic diagram of a non-off screen detection. For example, in this scenario, the user intermittently stares at the mobile phone screen as an example. It should be noted that: Fig.19 The duration and time of user gaze shown in are only illustrative examples and are not limited in this application. Fig.19 For example, the screen of the mobile phone is in the off-screen mode from time T0. Of course, the mobile phone may be in the off-screen mode before time T0, or the user may press the power button at time T0 to put the mobile phone into the off-screen mode. This application does not limit this. At time T1, if Fig.18 As shown in (1), the user presses the power button. Fig.18 As shown in (2), the mobile phone responds to the received user operation, displays the desktop after unlocking through face recognition, and at this time, the mobile phone is in a bright screen state.

[0246] Still refer to Fig.19 , for example, at time T0-T1, the sensing service is Figure 6-7 The process shown is executed, and the specific description can be referred to Fig.17 The relevant contents are not repeated here. For example, Fig.19As shown, in this embodiment, the user continues to stare at the mobile phone display screen from time T2.

[0247] At time T3, the perception service starts to perform the gaze-on-screen detection and opens the detection window. The description of the detection window can be found above and will not be repeated here. It should be noted that from time T2 to T3, although the user continues to gaze at the display screen, the perception service does not start the detection window from time T2 to T3, so the perception service cannot obtain the user's gaze situation.

[0248] like Fig.19 As shown, in this embodiment, the user is continuously looking at the display screen, so the perception service can detect the user's gaze after opening the detection window. At time T4, the perception service can control the display screen to continue to keep the screen on. Optionally, as described above, the time difference between time T3 and time T4 can be an algorithm delay, for example, it can be 1s, of course, it can also be a smaller or larger value, which is not limited in this application.

[0249] Exemplarily, the display screen re-times 24s from time T4. Also, the perception service repeats S705 from time T4, that is, continues to monitor whether user operations are received within 19s. The specific process can be referred to the above description and will not be repeated here. Exemplarily, at time T5, the perception service restarts the staring screen detection. At time T6, after the perception service detects the user's gaze, it controls the display screen to remain on.

[0250] Fig. 20 This is another schematic diagram of an exemplary non-off screen detection. Fig. 20 For example, the sensing service starts to detect the screen being watched at time T1, that is, the detection window is opened. For example, the preset detection time is 11 seconds. In this embodiment, at time T2, that is, when the detection time is 3 seconds, Fig.21 As shown in (1), the user presses the power button. Fig.21 As shown in (2), the mobile phone enters the screen-off mode in response to the received user operation, that is, the display screen is turned off. Correspondingly, at time T2, the perception module can determine through the input management service that the power button is pressed, and the perception module can determine that the mobile phone is locked. Figure 8 , when the perception module performs the gaze-on-screen detection, if Figure 8 If any of the conditions is triggered, the detection process ends. Fig. 20 The sensing module ends the detection at time T2 and repeats S701. The undescribed parts can be referred to above and will not be described here.

[0251] Fig. 22 This is another schematic diagram of an exemplary non-off screen detection. Fig. 22 For example, the perception service starts to detect the condition of not turning off the screen while looking at it at time T1, that is, opens the detection window. At time T2, the user touches the screen, and the screen remains on. In response to the user touching the screen operation obtained, the perception service determines that the condition of not turning off the screen while looking at it is not met, and the detection process ends. The perception service repeats S701. It should be noted that Fig. 22 The user touch operation scenario is only schematically shown. In other embodiments, Fig. 22 The process shown is also applicable to the scenario where the perception service detects that the foreground application activates the application lock during the detection process. That is, if the foreground application activates the application lock at time T2, the mobile phone keeps the screen on, the perception service ends this detection process, and repeats S701.

[0252] Fig.23 This is another schematic diagram of an exemplary non-off screen detection. Fig.23 Exemplarily, at time T1, the perception service starts detecting whether the user is staring at the screen without turning it off, that is, opens the detection window. At time T2, the user answers a voice call. The perception service detects that the user is answering the voice call, and determines that the condition of staring at the screen without turning it off is not met, and this detection process ends, and the perception service repeats S701. Exemplarily, at time T3, the display screen stays on for 24 seconds and then enters a semi-off state. In other words, during the process of a user making or receiving a voice call, whether the phone's screen is on does not affect the user's use of the phone. Accordingly, the phone can enter a semi-off state to save power consumption. It should be noted that Fig.23 The scenario of voice call is only schematically shown. In other embodiments, Fig.23 The process shown is also applicable to the scenario where the perception service detects that the proximity light sensor is continuously blocked during the detection process. That is, if at time T2, the user places the phone on the table, causing the proximity light sensor to be continuously blocked, the perception service terminates the detection process accordingly and repeats S701. The phone can enter the semi-off state after the screen remains on for 24 seconds.

[0253] Fig.24 This is another schematic diagram of an exemplary non-off screen detection. Fig.24 , exemplarily, when the perception module is executing S705, that is, detecting whether there is a user operation within 19 seconds, at time T1 (for example, 15 seconds after the start of detection), the perception service detects that the user touches the screen. The perception module can respond to the received user touch screen operation and determine that the detection condition is not met, and this detection process ends. The perception service repeats S701, and the subsequent process can refer to the relevant description in the above embodiment, which will not be repeated here.

[0254] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0255] In one example, Fig.25 A schematic block diagram of a device 2500 according to an embodiment of the present application is shown. The device 2500 may include: a processor 2501 and a transceiver / transceiver pin 2502 , and optionally, a memory 2503 .

[0256] The various components of the device 2500 are coupled together via a bus 2504, wherein the bus 2504 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus 2504 in the figure.

[0257] Optionally, the memory 2503 may be used for the instructions in the aforementioned method embodiment. The processor 2501 may be used to execute the instructions in the memory 2503, and control the receiving pin to receive a signal, and control the sending pin to send a signal.

[0258] The device 2500 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0259] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0260] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0261] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0262] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0263] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0264] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0265] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0266] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0267] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0268] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0269] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0270] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0271] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0272] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0273] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for controlling a display screen, applied to an electronic device, characterized in that: When the electronic device is in the screen-off state, in response to receiving a first operation from a user, the electronic device changes from the screen-off state to the screen-on state; In the screen-on state, when the electronic device does not receive a second operation from the user for a first period of time, the screen enters a half-screen-off state; in the half-screen-off state, when the electronic device does not receive a third operation from the user for a second period of time, the screen enters a screen-off state; In the half-screen-off state, the display screen of the electronic device becomes dark while the applications and modules of the electronic device operate normally; The method includes: determining whether the electronic device is in the screen-on state, whether a proximity light sensor of the electronic device is blocked, whether the electronic device is unlocked, and whether the electronic device starts a set application, wherein the set application is an application that automatically turns on an application lock so that the electronic device continues to maintain the screen-on state; When it is determined that the electronic device is in a screen-on state, the proximity light sensor is not blocked, the electronic device is unlocked, and the set application is not started, the electronic device performs a first detection, the first detection is used to detect whether a fourth operation of the user is received, and the first detection lasts for a third time period; When it is determined that no user operation is detected within the third time period, a second detection is performed, the second detection lasts for a fourth time period, and the sum of the third time period and the fourth time period is equal to the sum of the first time period and the second time period; Executing the second detection includes: acquiring an image frame, and detecting whether the image frame includes a human face and is watched by a user; When it is determined that the first image frame includes a human face and there is gaze; The electronic device is made to enter the screen-on state.

2. The method according to claim 1, characterized in that Also includes: The electronic device captures the image frames at a first frame rate, which is 60fps or 5fps.

3. The method according to claim 2, characterized in that The electronic device is deployed with a trusted execution environment TEE, which is used to isolate highly security-sensitive applications from a general software environment; The capturing of image frames and detecting whether the image frames include a human face and are being watched by a user include: outputting the image frames captured by the camera to the TEE at the first frame rate, and detecting whether the image frames in the TEE include a human face and are being watched by a user.

4. The method according to claim 3, characterized in that Also includes: After detecting whether the image frame in the TEE includes a human face and whether the user is looking at the image frame, the image frame is deleted from the TEE.

5. The method according to claim 1, characterized in that The detecting whether the image frame includes a human face and whether there is user gaze comprises: identifying a human face in the image frame based on a face detection algorithm; In the case of recognizing that the image frame includes a single human face, recognizing whether eyes of a person of the single human face are looking at the single human face based on a gaze detection algorithm; When the image frame includes a plurality of human faces or includes no human face, a human face in an image frame next to the image frame is identified based on the face detection algorithm.

6. The method according to claim 1, characterized in that The method further comprises: Before determining whether the electronic device is in the screen-on state, whether the proximity light sensor of the electronic device is blocked, whether the electronic device is unlocked, and whether the electronic device starts a set application, the set application is an application that automatically turns on the application lock so that the electronic device continues to keep the screen on, Displaying a smart perception interface, wherein the smart perception interface includes an option of "watching without turning off the screen"; Receive a user's operation of turning on the option of "watching without turning off the screen".

7. The method according to claim 1, characterized in that The method further includes: determining whether the electronic device is in the screen-on state, whether the proximity light sensor of the electronic device is blocked, whether the electronic device is unlocked, and whether the electronic device starts a set application, wherein the set application is an application that automatically turns on the application lock so that the electronic device continues to maintain the screen-on state, including: Determining whether the electronic device is in the screen-on state; When the electronic device is in the screen-on state, determining whether the proximity light sensor is blocked; When the electronic device is not in the screen-on state, the determination process ends; When the proximity light sensor is not blocked, determining whether the electronic device is unlocked; When the proximity light sensor is blocked, the judgment process ends; When the electronic device is unlocked, determining whether the electronic device starts a set application; When the electronic device is not unlocked, the determination process ends; 8. The method according to claim 1, characterized in that The execution of starting the second detection also includes: detecting whether a voice call is running, detecting whether the electronic device receives a touch operation, detecting whether the electronic device receives a lock screen operation, detecting that the foreground application has not started the application lock, and detecting that the proximity light sensor is not continuously blocked.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The invention comprises computer program codes, which, when executed by an electronic device, implement the steps of the information identification method according to any one of claims 1 to 8.

Images