Display method and electronic device

Abstract

A display method and an electronic device (100) applied to the field of data processing. The electronic device (100) comprises a first camera, wherein the first camera is a full-depth-resolution TOF camera. The method comprises: acquiring first brightness of the environment, in which the electronic device is located, at a first moment (S101); if the first brightness is less than a preset threshold, acquiring a first image (S102); then determining a first display direction according to the first image (S103); and finally, displaying a first interface on the basis of the first display direction (S104), wherein the first image is an image captured by the first camera at the first moment. Because the first camera is a TOF camera, the definition of the image captured by the first camera is not affected by the intensity of ambient light, and the probability that the first display direction determined according to the first image having higher definition is the same as a display direction expected by a user is improved, so that the display direction of the display interface can better meet the expectation of the user.

Description

Display method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on April 27, 2023, with application number 202310481419.2 and application name “Display Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of data processing, and more particularly, to a display method and an electronic device. Background Art

[0003] When using an electronic device, as the user's holding posture of the electronic device changes, the display direction of the display interface on the electronic device will change accordingly. For example, the display direction of the display interface can include horizontal screen display and vertical screen display.

[0004] At present, electronic devices are usually equipped with an accelerometer (usually a three-axis accelerometer) and a camera. The electronic device can determine the posture of the electronic device based on the accelerometer. At the same time, the face image can be acquired through the camera. Then, the electronic device can determine the display direction of the display interface based on the posture of the electronic device and the direction indicated by the face image. However, in some scenarios, for example, when the user is lying on his side and using a mobile phone at night, due to the poor ambient light, the resolution of the collected face image is low and cannot be used to determine the screen direction. At this time, the display direction determined by the posture of the mobile phone determined by the electronic device through the accelerometer is different from the display direction expected by the user. This will cause the display direction finally displayed on the display interface in the electronic device to be different from the display direction expected by the user.

[0005] Based on this, how to make the display direction of the display interface on the electronic device more in line with the user's expectations has become an urgent problem to be solved.

[0006] Summary of the Invention

[0007] The present application provides a display method that can make the display direction of a display interface on an electronic device more in line with user expectations.

[0008] In a first aspect, a display method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera, which is a full-resolution depth TOF camera. The method includes: obtaining a first brightness of an environment in which the electronic device is located at a first moment; if the first brightness is less than a preset threshold, obtaining a first image, which is an image captured by the first camera at the first moment; determining a first display direction based on the first image; and displaying a first interface based on the first display direction.

[0009] The display method provided in an embodiment of the present application is applied to an electronic device, which includes a first camera, wherein the first camera is a full-resolution depth TOF camera, which first obtains a first brightness of the environment in which the electronic device is located at a first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines a first display direction based on the first image, and finally displays a first interface based on the first display direction, wherein the first image is an image captured by the first camera at the first moment. Since the first camera is a TOF camera, the clarity of the image captured by the first camera will not be affected by the intensity of the ambient light, so that the probability that the first display direction determined according to the first image with higher clarity is consistent with the display direction expected by the user can be increased, thereby making the display direction of the display interface more in line with the user's expectations.

[0010] In combination with the first aspect, in certain embodiments of the first aspect, the electronic device also includes a first sensor, the first sensor is used to collect posture data of the electronic device, and determine a first display direction based on the first image, including: if the second display direction is obtained based on the first image, the first display direction is the same as the second display direction; if the second display direction is not obtained based on the first image, obtaining first data, the first data is the posture data of the electronic device collected by the first sensor at the first moment; determining a third display direction based on the first data, the first display direction is the same as the third display direction.

[0011] The display direction provided in an embodiment of the present application is applied to an electronic device, which includes a first sensor and a first camera, wherein the first sensor is used to collect posture data of the electronic device, and the first camera is a comprehensive depth resolution time-of-flight camera, which obtains first data and first brightness, and obtains the first brightness of the environment in which the electronic device is located at a first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines whether a second display direction can be obtained based on the first image. If the second display direction can be obtained based on the first image, the display direction of the electronic device is set to the second display direction. If the second display direction cannot be obtained based on the first image, a third display direction is determined based on the first data, and then the display direction of the electronic device is set to the third display direction. This is equivalent to, in the process of determining the display direction of the electronic device, the display direction obtained based on the first image has a higher priority than the display direction obtained based on the first data. Since the first image can indicate the direction of a face, the display direction obtained based on the direction of the user's face is more in line with the user's expectations than the display direction obtained based on the posture data (first data) of the electronic device, thereby making the display direction finally displayed by the electronic device more in line with the user's expectations.

[0012] In combination with the first aspect, in certain embodiments of the first aspect, the first image includes an infrared (IR) image, and the method further includes: obtaining key points in the IR image, the key points being used to indicate the locations of facial features in the IR image; determining a first angle corresponding to the IR image based on the key points, the first angle referring to the angle between the face in the IR image and a preset direction; and determining a second display direction based on the first angle.

[0013] The display method provided in the embodiment of the present application is that the electronic device can obtain an IR image captured by a TOF camera at a first moment, then obtain key points in the IR image indicating the positions of facial features, and then determine a first angle between the face in the IR image and a preset direction based on the key points, and then determine a second display direction according to the first angle. Since the key points can indicate the positions of the facial features in the IR image, the first angle determined according to the key points can more accurately indicate the angle between the face and the preset direction, so that the second display direction determined according to the first angle is more consistent with the display direction expected by the user, thereby improving the user experience.

[0014] In combination with the first aspect, in certain embodiments of the first aspect, the first image also includes a depth image, and the method further includes: determining whether the IR image is a face image based on the depth image; if the IR image is a face image, determining the second display direction based on the IR image.

[0015] The display direction provided in the embodiment of the present application can, before determining the second display direction based on the IR image, first determine whether the IR image is a face image based on the depth image collected at the same time as the IR image, that is, whether the IR image is obtained by photographing the object rather than photographing the picture or photo. Then, when the IR image is determined to be obtained by photographing the object, the second display direction is determined based on the IR image. This is equivalent to excluding the situation of determining the second display direction based on the picture or photo before determining the second display direction based on the IR image. Since the picture or photo is not related to the user's current posture, the situation of determining the display direction of the mobile phone's display interface by the picture or photo is avoided, that is, the situation of avoiding the situation where the determined display direction is irrelevant to the user's current posture, and at the same time, making the determined display direction more in line with the user's expectations.

[0016] In combination with the first aspect, in certain embodiments of the first aspect, if after obtaining the second display direction based on the first image, the method further includes: acquiring a second image and second data, the second image including an image captured by the first camera at a second moment, the second moment being a moment after the first moment, and the second data being posture data captured by the first sensor at the second moment; if the fourth display direction is not obtained based on the second image, determining the fifth display direction according to the second data; and displaying the second interface based on the fifth display direction.

[0017] The display method provided in the embodiment of the present application is applied to an electronic device, which includes a first sensor and a first camera, wherein the first sensor is used to collect posture data of the electronic device, and the first camera is a comprehensive depth resolution TOF camera, which obtains first data and first brightness, and obtains the first brightness of the environment in which the electronic device is located at the first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines whether a second display direction can be obtained based on the first image. If the second display direction can be obtained based on the first image, the display direction of the electronic device is set to the display direction indicated by the second display direction. After the second display direction can be obtained based on the first image, if the corresponding display direction cannot be obtained again based on the image collected by the TOF camera, the display direction determined by the posture data is compared with the first image. Whether the display direction is the same is used to determine the fifth display direction of the second interface displayed at the next moment. Usually, the display direction of the electronic device will not change frequently. When the display direction determined at the first moment is the first display direction, at the second moment, the probability that the display direction of the electronic device is the same as the first display direction is greater. Therefore, after the face is enabled (that is, the second display direction can be obtained through the first image), the face is lost (that is, the fourth display direction cannot be obtained through the second image), first determine that the display direction determined based on the second data is the same as the first display direction at the first moment, and then set the display direction of the second interface to be the same as the display direction of the first interface displayed at the first moment. This can reduce the probability that the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, thereby improving the user experience.

[0018] In combination with the first aspect, in certain embodiments of the first aspect, determining the fifth display direction based on the second data includes: if the sixth display direction is not obtained based on the correspondence between the second data and the display direction, then the fifth display direction is the same as the first display direction.

[0019] The display method provided in the embodiment of the present application first determines whether the correspondence between the second data and the display direction can obtain the sixth display direction when the fourth display direction cannot be obtained based on the second image, and adopts the first display direction as the fifth display direction when the sixth display direction cannot be obtained through the correspondence between the second data and the display direction. In this way, when the display direction cannot be obtained by the image collected by the first camera and the posture data of the electronic device, the first display direction when the first interface was last displayed is adopted as the fifth display direction of the currently displayed second interface, avoiding the problem of being unable to determine the display direction of the second interface.

[0020] In combination with the first aspect, in certain embodiments of the first aspect, determining the fifth display direction based on the second data includes: if a sixth display direction is obtained based on the correspondence between the second data and the display direction, determining whether the sixth display direction is the same as the first display direction; if the sixth display direction is the same as the first display direction, then the fifth display direction is the same as the first display direction.

[0021] The display method provided in an embodiment of the present application, when the fourth display direction cannot be obtained based on the second image, first determines whether the correspondence between the second data and the display direction can obtain a sixth display direction. If the sixth display direction can be obtained based on the correspondence between the second data and the display direction, further determines whether the sixth display direction is the same as the first display direction. If the sixth display direction is the same as the first display direction, the display direction of the second interface is set to the first display direction, that is, the sixth display direction obtained based on the correspondence between the second data and the display direction. Generally, the display direction of the electronic device does not change frequently. If the display direction determined at a first moment is the first display direction, the probability that the display direction of the electronic device is the same as the first display direction at a second moment is greater. Therefore, when the display direction can only be determined based on the second data, first determine that the display direction determined based on the second data is the same as the first display direction at the first moment, and then set the display direction of the second interface to be the same as the display direction of the first interface at the first moment. This can reduce the probability that the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, thereby improving the user experience.

[0022] In combination with the first aspect, in certain embodiments of the first aspect, the method further includes: if the sixth display direction is different from the first display direction, turning on the first camera; obtaining a third image through the first camera, the third image including an image captured by the first camera at a third moment, the third moment being a moment after the second moment; and determining the seventh display direction based on the third image.

[0023] The display method provided in the embodiment of the present application, when the fourth display direction cannot be obtained based on the second image, first determines whether the correspondence between the second data and the display direction can obtain the sixth display direction, and when the sixth display direction can be obtained based on the correspondence between the second data and the display direction, further determines whether the sixth display direction is the same as the first display direction, and when the sixth display direction is different from the first display direction, turns on the TOF camera, obtains a third image through the TOF camera, and then determines the seventh display direction based on the third image, and finally displays the third interface based on the seventh display direction, so that even if the display direction of the second interface cannot be accurately determined by the second image and / or the second data at the second moment, the third image obtained by the TOF camera can be used at the next moment (that is, the third moment) to determine the display direction of the third interface, that is, the display direction of the third interface displayed at the third moment is based on the third image captured by the TOF camera, which is more in line with the user's expectation, thereby improving the user experience.

[0024] In combination with the first aspect, in certain embodiments of the first aspect, the electronic device also includes a second camera, which is used to capture ambient light AO images. The method also includes: if the first brightness is greater than or equal to a preset threshold, obtaining a fourth image, the fourth image being the image captured by the second camera at the first moment; determining the first display direction based on the fourth image; and displaying the first interface based on the first display direction.

[0025] The display method provided by the embodiment of the present application is applied to an electronic device, which includes a first sensor, a first camera, and a second camera, wherein the first sensor is used to collect posture data of the electronic device, the first camera is a full-resolution depth TOF camera, and the second camera is an ambient light camera for collecting AO images. The method includes: in response to a first operation, obtaining first data and a first brightness, and then when the first brightness is less than a preset threshold, obtaining a first image, and then determining a first display direction based on the first data and / or the first image; when the first brightness is greater than or equal to the preset threshold, obtaining a fourth image, and determining the first display direction based on the first data and / or the fourth image; and finally The first interface is displayed based on the first display direction. That is to say, by adopting the display method provided by the present application, when the ambient light is weak, the first display direction can be determined by using the first image captured by the TOF camera that is not affected by the ambient light intensity. When the ambient light is strong, the first display direction can be determined by using the AO image captured by the ambient light camera with lower power consumption. Therefore, when the ambient light is weak, the probability that the first display direction determined according to the first image and / or first data with higher clarity is consistent with the display direction expected by the user can be increased. When the ambient light is strong, the first display direction can be determined by using the AO image with lower power consumption to reduce the power consumption of the electronic device.

[0026] In combination with the first aspect, in some embodiments of the first aspect, after acquiring the first image, the method further includes: turning off the first camera.

[0027] The display method provided in the embodiment of the present application, since the power consumption of the TOF camera is usually large, therefore, the TOF camera is turned off after the first image is captured, or the TOF camera is turned off after the second display direction is obtained based on the first image, so as to reduce the power consumption caused by the TOF camera.

[0028] In a second aspect, a display device is provided, comprising a unit for executing any of the methods in the first aspect. The device may be a server, a terminal device, or a chip within a terminal device. The device may include an input unit and a processing unit.

[0029] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal device executes any one of the methods in the first aspect.

[0030] When the device is a chip in a terminal device, the processing unit may be a processing unit inside the chip, and the input unit may be an output interface, a pin or a circuit, etc.; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods in the first aspect.

[0031] In one possible implementation, a memory is used to store computer program code; a processor executes the computer program code stored in the memory, and when the computer program code stored in the memory is executed, the processor is used to perform the following: obtaining a first brightness of an environment in which the electronic device is located at a first moment; if the first brightness is less than a preset threshold, obtaining a first image, the first image being an image captured by a first camera at the first moment; determining a first display direction based on the first image; and displaying a first interface based on the first display direction.

[0032] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by a display device, the display device executes any one of the display methods in the first aspect.

[0033] In a fourth aspect, a computer program product is provided, comprising: a computer program code, wherein when the computer program code is executed by a display device, the display device executes any one of the device methods in the first aspect.

[0034] The display method provided in an embodiment of the present application is applied to an electronic device, which includes a first camera, wherein the first camera is a full-resolution depth TOF camera, which first obtains a first brightness of the environment in which the electronic device is located at a first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines a first display direction based on the first image, and finally displays a first interface based on the first display direction, wherein the first image is an image captured by the first camera at the first moment. Since the first camera is a TOF camera, the clarity of the image captured by the first camera will not be affected by the intensity of the ambient light, so that the probability that the first display direction determined according to the first image with higher clarity is consistent with the display direction expected by the user can be increased, thereby making the display direction of the display interface more in line with the user's expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0036] FIG2 is a schematic diagram of a software system of an electronic device applicable to the present application;

[0037] FIG3 is a schematic diagram of different display directions of an electronic device;

[0038] FIG4 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0039] FIG5 is a flow chart of a display method provided in an embodiment of the present application;

[0040] FIG6 is a flow chart of another display method provided in an embodiment of the present application;

[0041] FIG7 is a schematic flow chart of another display method provided in an embodiment of the present application;

[0042] FIG8 is a schematic diagram of a first image provided in an embodiment of the present application;

[0043] FIG9 is a schematic diagram of key points in an IR image provided by an embodiment of the present application;

[0044] FIG10 is a schematic diagram of a first angle provided by an embodiment of the present application;

[0045] FIG11 is a schematic diagram of different display directions provided by an embodiment of the present application;

[0046] FIG12 is a flow chart of another display method provided in an embodiment of the present application;

[0047] FIG13 is a flow chart of another display method provided in an embodiment of the present application;

[0048] FIG14 is a schematic flow chart of another display method provided in an embodiment of the present application;

[0049] FIG15 is a schematic diagram of a display device provided by the present application;

[0050] FIG16 is a schematic diagram of an electronic device with a display provided by the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0052] In the following, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include one or more of the features.

[0053] The display method provided in the embodiment of the present application can be applied to electronic devices. Optionally, the electronic device includes a terminal device, which can also be called a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device can be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0054] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0055] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0056] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0057] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0058] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0059] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0060] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0061] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0062] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0063] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0064] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0065] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. In one possible case, the electronic device 100 may include a camera for collecting ambient light images and a full-resolution depth (Time of Flight, TOF) camera. Among them, the TOF camera can be used to collect depth images and infrared (Infrared Radiation, IR) images.

[0066] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0067] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0068] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0069] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0070] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0071] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0072] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0073] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0074] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0075] It should be noted that any electronic device mentioned in the embodiments of the present application may include more or fewer modules in the electronic device 100.

[0076] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0077] FIG2 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0078] The layered architecture of electronic device 100 divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into four layers: from top to bottom, the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0079] The application layer can include a series of application packages.

[0080] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0081] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0082] As shown in Figure 2, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a display direction decision module, a face direction recognition module, a sensor direction recognition module, and a face loss fusion module.

[0083] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0084] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0085] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0086] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0087] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0088] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0089] The display direction decision module is used to determine the final display direction of the application based on the display direction determined by the sensor and / or camera. It should be understood that the display direction determined by the display direction decision module is the display direction used when the electronic device actually displays the interface.

[0090] The face direction recognition module is used to determine the face direction based on the image captured by the TOF camera, and then determine the candidate display direction based on the face direction. It should be understood that the candidate display direction determined by the face direction recognition module may not be the display direction used when the electronic device actually displays the interface. The display direction used when the electronic device actually displays the interface is ultimately determined by the display direction decision module based on the priority order of the candidate display directions determined by the face direction recognition module.

[0091] The sensor orientation recognition module is used to determine candidate display orientations based on the electronic device's posture data collected by the acceleration sensor. It should be understood that the candidate display orientations determined by the sensor orientation recognition module may not be the display orientation used by the electronic device when it actually displays the interface. The display orientation used by the electronic device when it actually displays the interface is ultimately determined by the display orientation decision module based on the priority order of the candidate display orientations determined by the sensor orientation recognition module.

[0092] The face loss fusion module is used to determine the candidate display direction based on the candidate display direction determined by the sensor direction recognition module and the display direction used by the electronic device when it actually displayed the interface last time, when the face direction recognition module cannot determine the candidate display direction after the face direction recognition module determines the candidate display direction.

[0093] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0094] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0095] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0096] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0097] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0098] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0099] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0100] A 2D graphics engine is a drawing engine for 2D drawings.

[0101] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, Wi-Fi drivers, etc.

[0102] It should be noted that the electronic device mentioned in the embodiments of the present application may include more or fewer modules in the above-mentioned electronic devices. For example, the electronic device may also include a memory, a timer, etc.

[0103] To facilitate understanding, some of the examples given are provided for reference to the description of concepts related to the embodiments of the present application.

[0104] 1. Depth Image

[0105] A depth image refers to an image in which the distance from the image collector (usually a TOF camera) to each point in the image scene is used as a pixel value.

[0106] 2. Ambient Occlusion (AO) Image

[0107] An AO image typically refers to an image obtained by simulating global illumination using a specific calculation method. AO images can often be used to improve display issues caused by insufficient light in an image. AO images are typically obtained by processing images captured by a camera, and the power consumption required to obtain AO images is relatively low. Due to the high image quality and low power consumption required to obtain AO images, electronic devices can continuously and uninterruptedly obtain AO images, which in turn allows AO images to be used to determine the display orientation of electronic devices.

[0108] 3. Infrared Radiation (IR) images

[0109] IR images typically refer to infrared images captured by a TOF camera. Keypoints of the subject can often be found in IR images. Keypoints are used to identify the location of the subject's features within the image. For example, keypoints can indicate the location of the subject's facial features within the IR image.

[0110] Currently, when using electronic devices, the display orientation of the electronic device's display interface changes as the user's grip changes. For example, when the user holds the phone vertically, the phone displays the interface vertically along the Z axis, as shown in Figure 3 (a). When the user holds the phone horizontally, the phone displays the interface horizontally along the Y axis, as shown in Figure 3 (b).

[0111] Typically, an accelerometer is provided in an electronic device. The accelerometer can detect the magnitude and direction of gravity when the electronic device is stationary, and is used to identify the posture of the electronic device. The electronic device is also typically provided with a camera, such as a camera for collecting ambient light images and a TOF camera. The camera for collecting ambient light images can be used to collect AO images including human faces. The electronic device can determine the posture of the electronic device based on the accelerometer, determine the direction of the user's face based on the AO image, and then determine the display direction of the display interface that meets the user's expectations based on the direction of the face and the posture of the electronic device.

[0112] However, in one possible scenario, for example, at night when the ambient light is low, the user is lying down to look at the phone. In this case, due to the low ambient light, the pixels of the AO image captured by the camera are low, and the phone cannot determine the user's face direction based on the AO image. At the same time, due to the user's posture changing from lying down, the posture of the phone also changes. The display direction determined by the electronic device's posture obtained by the acceleration sensor may be different from the display direction determined by the face direction. Therefore, in this case, it is easy for the display direction of the phone to be different from the display direction expected by the user.

[0113] In view of this, the display method provided in an embodiment of the present application is applied to an electronic device, which includes a first camera, wherein the first camera is a full-resolution depth TOF camera, which first obtains the first brightness of the environment in which the electronic device is located at the first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines the first display direction based on the first image, and finally displays the first interface based on the first display direction, wherein the first image is the image captured by the first camera at the first moment. Since the first camera is a TOF camera, the clarity of the image captured by the first camera will not be affected by the intensity of the ambient light, so that the probability that the first display direction determined according to the first image with higher clarity is consistent with the display direction expected by the user can be increased, thereby making the display direction of the display interface more in line with the user's expectations.

[0114] The application scenarios provided by the embodiments of the present application are described below with reference to the accompanying drawings.

[0115] FIG4 is a schematic diagram of an application scenario used by the display method provided in an embodiment of the present application. As shown in FIG4 , a user is lying down and using a mobile phone 100 in a dimly lit environment. Due to the low brightness of the user's environment, the clarity of the AO image collected by the mobile phone 100 is low and cannot be used to determine the direction of the user's face. In addition, since the user is in a lying state, there is a small probability that the display direction determined by the posture of the mobile phone 100 determined by the acceleration sensor in the mobile phone 100 deviates from the display direction expected by the user. In a dimly lit environment, the display method provided in an embodiment of the present application can be used to determine the display direction of the mobile phone 100, thereby reducing the probability that the display direction determined by the mobile phone 100 is different from the display direction expected by the user.

[0116] It should be understood that the above is an example of an application scenario and does not limit the application scenario of this application.

[0117] The display method provided in the embodiment of the present application is described in detail below with reference to FIG. 5 to FIG. 14 .

[0118] FIG5 is a flow chart of a display method provided in an embodiment of the present application. The method is applied to an electronic device, and the electronic device includes a first camera, wherein the first camera is a TOF camera. As shown in FIG5 , the method includes:

[0119] S101: Obtain a first brightness of an environment where the electronic device is located at a first moment.

[0120] Exemplarily, the electronic device may obtain a first brightness of the environment at a first moment through the ambient light sensor 180L as shown in FIG. 1 .

[0121] S102: If the first brightness is less than a preset threshold, acquire a first image.

[0122] The first image is an image captured by the first camera at the first moment.

[0123] S103: Determine a first display direction according to the first image.

[0124] The first display orientation may refer to a display orientation that matches the user's reading direction when the user is holding the phone. For example, when the user is holding the phone vertically, as shown in FIG3(a), the first display orientation generally refers to the portrait display orientation. When the user is holding the phone horizontally, as shown in FIG3(b), the first display orientation generally refers to the landscape display orientation, and the orientation of the displayed content on the display interface matches the orientation in which the user is holding the phone.

[0125] For example, the electronic device may determine the first display direction according to the direction of the face in the first image.

[0126] For another example, the electronic device may determine display direction A based on the direction of the face in the first image, determine display direction B based on the posture data of the electronic device, and then select the first display direction from display direction A and display direction B according to a preset priority.

[0127] S104: Display a first interface based on the first display direction.

[0128] The first interface may refer to an interface that the electronic device needs to display at the current moment. For example, the first interface may refer to the desktop interface after the user wakes up the phone screen; it may also refer to the interface displayed before the user turns off the phone screen; or it may refer to the interface displayed at any time after the phone screen is woken up; and this embodiment of the application is not limited to this.

[0129] It should be understood that after the electronic device determines the first display direction, it can display the first interface according to the first display direction, or it can select a display direction with a higher priority to display the first interface based on the display direction corresponding to the application currently running on the mobile phone and the first display direction.

[0130] For example, the first display direction determined according to steps S101 to S103 refers to the vertical screen display direction. However, the application currently running on the mobile phone is an application for playing videos. Since horizontal screen playback usually has a better effect when playing videos, the mobile phone sets the display direction corresponding to the application for playing videos to the horizontal screen display direction. In this case, since usually when an application for playing videos is running, the display direction corresponding to the application for playing videos has a higher priority than the first display direction, even in the case where the first display direction determined based on steps S101 to S104 refers to the vertical screen display direction, the first interface, that is, the interface for playing videos, can still be displayed in the horizontal screen display direction.

[0131] For example, the first display direction determined according to steps S101 to S103 is a horizontal display direction. The application currently running on the mobile phone is an instant messaging application. Since the vertical display effect is usually better during instant messaging, the mobile phone usually sets the display direction corresponding to the instant messaging application to the vertical display direction. In this case, since the priority of the display direction corresponding to the instant messaging application is lower than the priority of the first display direction when the instant messaging application is running, when the first display direction determined based on steps S101 to S103 is a horizontal display direction, the first interface is displayed in the horizontal display direction.

[0132] As shown in FIG2 , the display direction decision module in the application framework layer can be used to determine the display direction of the first interface from the first display direction and the display direction corresponding to the application, that is, to display the first interface based on the first display direction.

[0133] The display method provided in an embodiment of the present application is applied to an electronic device, which includes a first camera, wherein the first camera is a full-resolution depth TOF camera, which first obtains a first brightness of the environment in which the electronic device is located at a first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines a first display direction based on the first image, and finally displays a first interface based on the first display direction, wherein the first image is an image captured by the first camera at the first moment. Since the first camera is a TOF camera, the clarity of the image captured by the first camera will not be affected by the intensity of the ambient light, so that the probability that the first display direction determined according to the first image with higher clarity is consistent with the display direction expected by the user can be increased, thereby making the display direction of the display interface more in line with the user's expectations.

[0134] In one possible case, the display method provided in the embodiment of the present application can be applied to a mobile phone, wherein the mobile phone includes an accelerometer for obtaining the posture information of the mobile phone, and a TOF camera for collecting IR images and / or depth images. The mobile phone can determine the display direction of the display interface of the mobile phone based on the posture information collected by the accelerometer and / or the IR image collected by the TOF camera. In the case where the IR image collected by the TOF camera can determine the display direction, the display direction determined by the IR image is used as the display direction of the display interface of the electronic device; in the case where the IR image cannot determine the display direction, the display direction of the display interface of the electronic device is determined based on the posture information collected by the accelerometer. The following is a detailed description through the embodiment shown in Figure 6.

[0135] FIG6 is a flow chart of a display method provided by another embodiment of the present application. As shown in FIG6 , the method is applied to a mobile phone, which includes an acceleration sensor (equivalent to a first sensor) and a TOF camera (equivalent to a first camera). The acceleration sensor is used to collect mobile phone posture data, and the TOF camera is used to collect IR images and / or depth images. The method includes:

[0136] S201 : In response to a first operation, obtain a first brightness.

[0137] Among them, the first brightness may refer to the brightness of the environment of the mobile phone at the first moment, and the first operation may be used to wake up the mobile phone from the screen-off state to the screen-on state.

[0138] Exemplarily, the first operation may be an operation of sliding a finger upward, the first operation may also refer to an operation of unlocking the screen through a fingerprint, or the first operation may also refer to an operation of unlocking the screen through a face. This embodiment of the present application does not limit this.

[0139] S202: If the first brightness is less than a preset threshold, acquire a first image.

[0140] The first image may refer to an image captured by the TOF camera at the first moment. The TOF camera may refer to the first camera. Typically, the images captured by the TOF camera include an IR image and a depth image. It should be understood that the IR image and the depth image captured by the TOF camera are typically captured at the same moment.

[0141] S203: Determine whether a second display direction can be obtained based on the first image.

[0142] It should be understood that if the clarity of the first image is less than the clarity threshold, the electronic device cannot determine the direction of the face in the first image based on the first image, and further cannot determine the display direction based on the direction of the face in the first image. In other words, the second display direction cannot be obtained based on the first image.

[0143] Exemplarily, the first image is an IR image, and the electronic device can extract key points indicating features of the subject from the IR image, and then determine the face direction of the subject based on the key points. In the case where the clarity of the IR image is too low, the electronic device cannot extract key points indicating features of the subject from the IR image, or the number of key points extracted by the electronic device from the IR image is small. Both of these situations may result in the electronic device being unable to determine the face direction of the subject based on the extracted key points, and thus being unable to obtain the second display direction. Generally, the situation where the electronic device cannot obtain the display direction based on the IR image is called face loss, and the situation where the electronic device can obtain the display direction based on the IR image is called face enablement.

[0144] If the second display direction can be obtained based on the first image, S204 is executed.

[0145] If the second display direction cannot be obtained based on the first image, execute S205.

[0146] Optionally, the first image is an IR image. The mobile phone may determine the second display direction through the method steps shown in FIG7 , as shown in FIG7 , including:

[0147] S2031. Obtain key points in the IR image.

[0148] Among them, the key points are used to indicate the locations of facial features in the IR image.

[0149] It should be understood that IR images are typically captured by a time-of-flight (TOF) camera, which typically captures a depth image simultaneously with the IR image. As shown in Figure 8, the images captured by the TOF camera include an IR image 210 and a depth image 220. IR images use different colors to represent the temperature of the subject. Because temperatures vary slightly at different locations on the human body, IR images use different colors to identify different locations on the body. For example, if an IR image is captured of a user's face, the electronic device can identify the location of the facial features from the IR image.

[0150] Key points are used to indicate the locations of facial features in the IR image. Obtaining key points in the IR image is equivalent to identifying the locations of facial features from the IR image.

[0151] For example, the IR image can be input into a preset face detection model to obtain key points in the IR image. The key points in the IR image can be shown in FIG9 , including key point 1 of the left eye, key point 2 of the right eye, key point 3 of the nose tip, key point 4 of the left corner of the mouth, and key point 5 of the right corner of the mouth. The preset face detection model can be a neural network model.

[0152] Before obtaining the key points in the IR image, it may also be determined whether the current IR image is a face image based on the depth image.

[0153] Before S2031 "obtaining key points in the IR image", optionally, the following steps may be further included:

[0154] S2030 , determining whether the IR image is a face image based on the depth image, and if the IR image is a face image, executing S2031 to S2033 in sequence.

[0155] It should be understood that a depth image generally refers to an image in which the distance from the TOF camera to each point in the image scene is used as a pixel value. When the shooting scene is a real human, the distance between each point in the depth image and the TOF camera is different. However, if the depth image is obtained by shooting a picture or photo, the distance between each point in the depth image and the TOF camera is usually the same. Therefore, based on the distance between different points in the depth image and the TOF camera, it can be determined whether the IR image is obtained by shooting a human, that is, to determine whether the IR image is a face image, rather than a photo or picture. Among them, whether the IR image is a face image means that when the TOF camera captures the image, the corresponding shooting object is a face, rather than a photo or picture.

[0156] The display direction provided in the embodiment of the present application can, before determining the second display direction based on the IR image, first determine whether the IR image is a face image based on the depth image collected at the same time as the IR image, that is, whether the IR image is obtained by photographing the object rather than photographing the picture or photo. Then, when the IR image is determined to be obtained by photographing the object, the second display direction is determined based on the IR image. This is equivalent to excluding the situation of determining the second display direction based on the picture or photo before determining the second display direction based on the IR image. Since the picture or photo is not related to the user's current posture, the situation of determining the display direction of the mobile phone's display interface by the picture or photo is avoided, that is, the situation of avoiding the situation where the determined display direction is irrelevant to the user's current posture, and at the same time, making the determined display direction more in line with the user's expectations.

[0157] S2032: Determine a first angle corresponding to the IR image according to the key point.

[0158] The first angle refers to the angle between the face in the IR image and the preset direction.

[0159] For example, as shown in FIG10 , the central axis β of the face can be determined based on the positions of the eyes, nose, and mouth in the face, and then the angle α between the central axis β and the X-axis (that is, the direction perpendicular to the direction of gravity), that is, the first angle, is obtained.

[0160] For example, as shown in FIG9 , the coordinate position of key point 1 (left eye key point) is (X1, Y1), the coordinate position of key point 2 (right eye key point) is (X2, Y2), the coordinate position of key point 3 (nose tip key point) is (X3, Y3), the coordinate position of key point 4 (left corner of mouth key point) is (X4, Y4), and the coordinate position of key point 5 (right corner of mouth key point) is (X5, Y5). The first angle α can be determined based on the coordinates of each key point and formula (1). Formula (1) includes:

[0161] When Y1=Y2 and Y4=Y5, if Y2>Y5, then the first angle α=90°; if Y2<Y5, then the first angle α=270°.

[0162] When X1=X2 and X4=X5, if X2>X5, the first angle α=0°; if X2<X5, the first angle α=180°.

[0163] S2033: Determine a second display direction based on the first angle.

[0164] After the first angle is obtained, the second display direction may be determined according to the correspondence between the first angle and the display direction.

[0165] For example, after obtaining the first angle, a parameter indicating the display direction of the mobile phone (first parameter) may be assigned a value according to the first angle.

[0166] For example, as shown in Table 1, when the first angle α satisfies "0° ≤ α ≤ 35° and 325° ≤ α ≤ 360°," the first parameter is assigned a value of "0." "0" corresponds to a left-landscape display orientation. In this case, adjusting the display orientation to the left-landscape orientation can align the display orientation with the user's face orientation, thereby ensuring that the phone's display orientation meets the user's expectations.

[0167] When the first angle α satisfies "55°≤α≤125°," the first parameter is assigned a value of "1." "1" corresponds to a portrait display orientation. In this case, adjusting the display orientation to portrait aligns the display orientation with the user's face orientation, thereby ensuring the phone's display orientation meets the user's expectations.

[0168] When the first angle α satisfies "145° ≤ α ≤ 215°," the first parameter is assigned a value of "2." "2" corresponds to a right-landscape display orientation. In this case, adjusting the display orientation to the right-landscape orientation can align the display orientation with the user's face orientation, thereby ensuring that the phone's display orientation meets the user's expectations.

[0169] When the first angle α satisfies "235°≤α≤305°," the first parameter is assigned a value of "3." "3" corresponds to an inverted portrait display orientation. In this case, adjusting the display orientation to an inverted portrait orientation can align the display orientation with the user's face orientation, thereby ensuring that the phone's display orientation meets the user's expectations.

[0170] If the first angle α does not meet any of the above conditions, the first parameter is assigned a value of "-2". When the assigned value of the first parameter is "-2", it indicates an abnormality, and the mobile phone does not adjust the display direction based on the assigned value of the first parameter.

[0171] Table 1

[0172] Here, α represents the first angle.

[0173] Among them, different display directions can be shown in Figure 11, including left horizontal screen display direction 0, vertical screen display direction 1, right horizontal screen display direction 2 and inverted vertical screen display direction 3.

[0174] For example, as shown in Figure 11, when the first parameter is assigned a value of "0", the current interface is displayed in the left horizontal screen display direction; when the first parameter is assigned a value of "1", the current interface is displayed in the vertical screen display direction; when the first parameter is assigned a value of "2", the current interface is displayed in the right horizontal screen inverted vertical screen display direction; when the first parameter is assigned a value of "3", the current interface is displayed in the inverted vertical screen display direction.

[0175] As shown in FIG2 , the face direction recognition module in the application framework layer can be used to execute the method steps of “determining whether the IR image is a face image based on the depth image, and when the IR image is a face image, obtaining key points in the IR image, determining the first angle corresponding to the IR image based on the key points, and determining the second display direction based on the first angle”.

[0176] The display method provided in the embodiment of the present application is that the electronic device can obtain an IR image captured by a TOF camera at a first moment, then obtain key points in the IR image indicating the positions of facial features, and then determine a first angle between the face in the IR image and a preset direction based on the key points, and then determine a second display direction according to the first angle. Since the key points can indicate the positions of the facial features in the IR image, the first angle determined according to the key points can more accurately indicate the angle between the face and the preset direction, so that the second display direction determined according to the first angle is more consistent with the display direction expected by the user, thereby improving the user experience.

[0177] S204: Set the second display direction as the first display direction.

[0178] After obtaining the second display direction based on the first image, the second display direction may be set as the first display direction, that is, the direction indicated by the first display direction may be set to be the same as the direction indicated by the second display direction.

[0179] S205: Acquire first data, and determine a third display direction based on the first data.

[0180] The first data may refer to the posture data collected by the acceleration sensor in the mobile phone at the first moment.

[0181] If the second display orientation cannot be obtained from the first image, the third display orientation is determined based on the first data. Since the first data may refer to the posture data collected by the accelerometer in the mobile phone at the first moment, the posture data may indicate the angle between the mobile phone and the direction of gravity. After obtaining the posture data, the third display orientation can be determined based on the angle between the mobile phone and the direction of gravity indicated by the posture data.

[0182] Similar to the correspondence between the first angle and the second display direction in Table 1, there is also a corresponding correspondence between the posture data and the display direction. The mobile phone can determine the third display direction based on the correspondence between the posture data and the display direction.

[0183] The steps of acquiring the first data and determining the third display direction based on the first data can be determined by a sensor direction recognition module in the application framework layer.

[0184] S206: Set the third display direction as the first display direction.

[0185] After obtaining the third display direction based on the first data, the third display direction may be set as the first display direction, that is, the direction indicated by the first display direction may be set to be the same as the direction indicated by the third display direction.

[0186] S207: Display the first interface based on the first display direction.

[0187] It should be understood that when face recognition is enabled, the first display direction is the same as the second display direction, and displaying the first interface based on the first display direction may refer to displaying the first interface using the second display direction. When face recognition is lost, the first display direction is the same as the third display direction, and displaying the first interface based on the first display direction may refer to displaying the first interface using the third display direction.

[0188] The display direction provided in an embodiment of the present application is applied to an electronic device, which includes a first sensor and a first camera, wherein the first sensor is used to collect posture data of the electronic device, and the first camera is a comprehensive depth resolution time-of-flight camera, which obtains first data and first brightness, and obtains the first brightness of the environment in which the electronic device is located at a first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines whether a second display direction can be obtained based on the first image. If the second display direction can be obtained based on the first image, the display direction of the electronic device is set to the second display direction. If the second display direction cannot be obtained based on the first image, a third display direction is determined based on the first data, and then the display direction of the electronic device is set to the third display direction. This is equivalent to, in the process of determining the display direction of the electronic device, the display direction obtained based on the first image has a higher priority than the display direction obtained based on the first data. Since the first image can indicate the direction of a face, the display direction obtained based on the direction of the user's face is more in line with the user's expectations than the display direction obtained based on the posture data (first data) of the electronic device, thereby making the display direction finally displayed by the electronic device more in line with the user's expectations.

[0189] In one possible case, when the second display direction can be obtained based on the first image, it is explained that the image acquisition device of the electronic device (such as a TOF camera) can normally capture images. In this case, if after the current moment, face loss occurs (i.e., the second display direction cannot be determined based on the first image), the electronic device can determine the display direction of the electronic device based on the posture data. For example, the electronic device can compare the sixth display direction (the display direction determined by the electronic device again through the posture data) with the first display direction. If the sixth display direction is the same as the first display direction, the first display direction is used as the display direction of the display interface of the mobile phone; if the sixth display direction is different from the first display direction, the first display direction is used as the display direction of the display interface of the mobile phone, and the TOF camera is started again to capture images, and the captured image is used as an image for determining the display direction of the display interface of the mobile phone at the next moment. The following is a detailed description of the embodiment shown in Figure 12.

[0190] FIG12 is a flow chart of a display method provided in another embodiment of the present application. As shown in FIG12 , the method is applied to a mobile phone, which includes an acceleration sensor and a TOF camera. The acceleration sensor is used to collect mobile phone posture data, and the TOF camera is used to collect IR images and / or depth images. The method includes:

[0191] S301 : In response to a first operation, obtain first data and first brightness.

[0192] Among them, the first data may refer to the posture data collected by the acceleration sensor in the mobile phone at the first moment, the first brightness may refer to the brightness of the environment described by the mobile phone at the first moment, and the first operation can be used to wake up the mobile phone from the screen-off state to the screen-on state.

[0193] Exemplarily, the first operation may be an operation of sliding a finger upward, the first operation may also refer to an operation of unlocking the screen through a fingerprint, or the first operation may also refer to an operation of unlocking the screen through a face. This embodiment of the present application does not limit this.

[0194] After acquiring the first data, the electronic device may also determine a candidate display direction based on the first data and store the candidate display direction so that the candidate display direction can be called in a subsequent process of determining the display direction of the electronic device.

[0195] S302: If the first brightness is less than a preset threshold, capture a first image through a TOF camera.

[0196] Exemplarily, the preset threshold may be 15 lux. When the first brightness is less than 15 lux, the first image is captured by the TOF camera.

[0197] S303: Determine whether a second display direction is obtained based on the first image.

[0198] The second display direction may refer to a display direction determined based on a correspondence between the first image and the display direction.

[0199] If the second display direction can be obtained based on the first image, S304 is executed.

[0200] The process of obtaining the second display direction based on the first image can refer to the method steps shown in the above S203, and will not be repeated here.

[0201] If the second display direction cannot be obtained based on the first image, execute S305.

[0202] S304: Set the second display direction as the first display direction, and display the first interface based on the first display direction.

[0203] The display direction of the display interface of the mobile phone may refer to a first display direction.

[0204] S305: Acquire a second image and second data.

[0205] The second image is an image captured by the TOF camera at a second moment, where the second moment is a moment after the first moment. The second data is the posture data captured by the acceleration sensor at the second moment.

[0206] It should be understood that the electronic device can periodically determine the display direction of the display interface according to a preset frequency. The first moment may refer to the moment of the Nth frame indicated by the preset frequency, and the second moment may refer to the moment of the N+mth frame. In one possible case, the second moment may refer to the moment of the N+1th frame.

[0207] S306: If the fourth display direction is not obtained based on the second image, determine the fifth display direction according to the second data.

[0208] Since the second image is an image captured at the second moment, and the second moment is a moment after the first moment, and since the first image captured by the electronic device at the first moment can determine the display direction, this is equivalent to enabling the face at the first moment. Therefore, when the fourth display direction cannot be obtained based on the second image, it is equivalent to the face being lost after the face is enabled. In this case, the electronic device can further obtain the posture data (i.e., the second data) at the second moment through the first sensor (equivalent to the acceleration sensor), and then determine the fifth display direction based on the second data. The fifth display direction is then used to display the second interface, which is the interface displayed by the electronic device at the second moment. It should be understood that the first interface can be the same display interface as the second interface, or it can be a different display interface, and the embodiments of the present application do not limit this.

[0209] If the fourth display orientation cannot be obtained from the second image, the fifth display orientation corresponding to the second data can be determined based on the correspondence between the electronic device's posture data and the display orientation. The steps of obtaining the second data and determining the fifth display orientation based on the second data can be performed by a sensor orientation recognition module in the application framework layer.

[0210] Optionally, determining the fifth display direction according to the second data may be implemented by the steps shown in FIG13 , as shown in FIG13 , including:

[0211] S3061: If the sixth display direction is not obtained based on the correspondence between the second data and the display direction, the fifth display direction is the same as the first display direction.

[0212] That is, if the display orientation cannot be determined based on the posture data of the electronic device at the second moment, the first display orientation used when displaying the first interface at the previous moment (i.e., the first moment) will be used as the display orientation used when displaying the second interface at the current moment (i.e., the second moment). That is, the fifth display orientation is the same as the first display orientation.

[0213] The display method provided in the embodiment of the present application first determines whether the correspondence between the second data and the display direction can obtain the sixth display direction when the fourth display direction cannot be obtained based on the second image, and adopts the first display direction as the fifth display direction when the sixth display direction cannot be obtained through the correspondence between the second data and the display direction. In this way, when the display direction cannot be obtained by the image collected by the first camera and the posture data of the electronic device, the first display direction when the first interface was last displayed is adopted as the fifth display direction of the currently displayed second interface, avoiding the problem of being unable to determine the display direction of the second interface.

[0214] S3062: If a sixth display direction is obtained based on the correspondence between the second data and the display direction, determine whether the sixth display direction is the same as the first display direction.

[0215] That is, if the display direction can be determined based on the posture data of the electronic device at the second moment, it is further confirmed whether the sixth display direction (equivalent to the display direction determined based on the posture data at the second moment) is the same as the first display direction.

[0216] S3063: If the sixth display direction is the same as the first display direction, then the fifth display direction is the same as the first display direction.

[0217] That is, when the display direction (sixth display direction) determined based on the posture data at the second moment is the same as the first display direction, the display direction (sixth display direction) determined by the posture data collected by the first sensor (acceleration sensor) is used as the display direction (fifth display direction) of the second interface.

[0218] The display method provided in an embodiment of the present application, when the fourth display direction cannot be obtained based on the second image, first determines whether the correspondence between the second data and the display direction can obtain a sixth display direction. If the sixth display direction can be obtained based on the correspondence between the second data and the display direction, further determines whether the sixth display direction is the same as the first display direction. If the sixth display direction is the same as the first display direction, the display direction of the second interface is set to the first display direction, that is, the sixth display direction obtained based on the correspondence between the second data and the display direction. Generally, the display direction of the electronic device does not change frequently. If the display direction determined at a first moment is the first display direction, the probability that the display direction of the electronic device is the same as the first display direction at a second moment is greater. Therefore, when the display direction can only be determined based on the second data, first determine that the display direction determined based on the second data is the same as the first display direction at the first moment, and then set the display direction of the second interface to be the same as the display direction of the first interface at the first moment. This can reduce the probability that the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, thereby improving the user experience.

[0219] In one possible case, if the sixth display direction is different from the first display direction, the method further includes:

[0220] S3064: If the sixth display direction is different from the first display direction, turn on the TOF camera.

[0221] When the display direction (sixth display direction) determined based on the posture data (second data) at the second moment is different from the first display direction, in order to avoid the situation where the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, the electronic device can re-wake up the TOF camera and then use the TOF camera to collect the image at the next moment (i.e., the third moment) for determining the display direction.

[0222] S3065. Acquire a third image through the TOF camera.

[0223] The third image includes an image captured by the TOF camera at a third moment, and the third moment is a moment after the second moment.

[0224] S3066: Determine a seventh display direction based on the third image.

[0225] It should be understood that the specific process of determining the seventh display direction based on the third image is similar to the method steps shown in S2031 to S2033, and will not be repeated here.

[0226] S3067. Display the third interface based on the seventh display direction.

[0227] It should be understood that the third interface can be the same display interface as the first interface and the second interface, or it can be a different display interface, and the embodiment of the present application does not limit this.

[0228] It should be understood that the face loss fusion module in the application framework layer can be used to execute the method steps shown in S3061 to S3065 above.

[0229] The display method provided in the embodiment of the present application, when the fourth display direction cannot be obtained based on the second image, first determines whether the correspondence between the second data and the display direction can obtain the sixth display direction, and when the sixth display direction can be obtained based on the correspondence between the second data and the display direction, further determines whether the sixth display direction is the same as the first display direction, and when the sixth display direction is different from the first display direction, turns on the TOF camera, obtains a third image through the TOF camera, and then determines the seventh display direction based on the third image, and finally displays the third interface based on the seventh display direction, so that even if the display direction of the second interface cannot be accurately determined by the second image and / or the second data at the second moment, the third image obtained by the TOF camera can be used at the next moment (that is, the third moment) to determine the display direction of the third interface, that is, the display direction of the third interface displayed at the third moment is based on the third image captured by the TOF camera, which is more in line with the user's expectation, thereby improving the user experience.

[0230] S307: Display the second interface based on the fifth display direction.

[0231] The specific process of displaying the second interface based on the fifth display direction is similar to the specific process of S104 "displaying the first interface based on the first display direction", and will not be repeated here.

[0232] The display method provided in the embodiment of the present application is applied to an electronic device, which includes a first sensor and a first camera, wherein the first sensor is used to collect posture data of the electronic device, and the first camera is a comprehensive depth resolution TOF camera, which obtains first data and first brightness, and obtains the first brightness of the environment in which the electronic device is located at the first moment, and then obtains a first image when the first brightness is less than a preset threshold, and then determines whether a second display direction can be obtained based on the first image. If the second display direction can be obtained based on the first image, the display direction of the electronic device is set to the display direction indicated by the second display direction. After the second display direction can be obtained based on the first image, if the corresponding display direction cannot be obtained again based on the image collected by the TOF camera, the display direction determined by the posture data is compared with the first image. Whether the display direction is the same is used to determine the fifth display direction of the second interface displayed at the next moment. Usually, the display direction of the electronic device will not change frequently. When the display direction determined at the first moment is the first display direction, at the second moment, the probability that the display direction of the electronic device is the same as the first display direction is greater. Therefore, after the face is enabled (that is, the second display direction can be obtained through the first image), the face is lost (that is, the fourth display direction cannot be obtained through the second image), first determine that the display direction determined based on the second data is the same as the first display direction at the first moment, and then set the display direction of the second interface to be the same as the display direction of the first interface displayed at the first moment. This can reduce the probability that the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, thereby improving the user experience.

[0233] In one possible scenario, when the phone screen is awakened, it is necessary to first determine whether the ambient light intensity in the current environment is weak. If the ambient light intensity is weak, the TOF camera is awakened and the image captured by the TOF camera is used to determine the display orientation of the phone display interface. If the ambient light intensity is not weak, the second camera is directly used to capture the AO image to determine the display orientation of the phone display interface. This is explained in detail below using the embodiment shown in Figure 14.

[0234] FIG14 is a flow chart of a display method provided by another embodiment of the present application. As shown in FIG14 , the method is applied to a mobile phone, which includes an acceleration sensor, a TOF camera, and an ambient light camera. The acceleration sensor is used to collect mobile phone posture data, the TOF camera is used to collect IR images and / or depth images, and the ambient light camera is used to collect AO images. The method includes:

[0235] S401 : In response to a first operation, obtain first data and first brightness.

[0236] Among them, the first data may refer to the posture data collected by the acceleration sensor in the mobile phone at the first moment, the first brightness may refer to the brightness of the environment described by the mobile phone at the first moment, and the first operation can be used to wake up the mobile phone from the screen-off state to the screen-on state.

[0237] Exemplarily, the first operation may be an operation of sliding a finger upward, the first operation may also refer to an operation of unlocking the screen through a fingerprint, or the first operation may also refer to an operation of unlocking the screen through a face. This embodiment of the present application does not limit this.

[0238] S402: Determine whether the first brightness is less than a preset threshold.

[0239] If the first brightness is less than the preset threshold, execute S403.

[0240] If the first brightness is greater than or equal to the preset threshold, execute S409.

[0241] S403: Capture a first image through a TOF camera.

[0242] The first image refers to the IR image captured by the TOF camera at the first moment.

[0243] S404: Determine whether a second display direction is obtained based on the first image.

[0244] Optionally, a specific process of obtaining the second display direction based on the first image may include:

[0245] S4041. Obtain key points in the IR image.

[0246] S4042. Determine a first angle corresponding to the IR image according to the key point.

[0247] S4043. Determine a second display direction based on the first angle.

[0248] If the second display direction can be obtained based on the first image, S405 is executed.

[0249] If the second display direction cannot be obtained based on the first image, execute S406.

[0250] S405: Set the second display direction as the first display direction.

[0251] It should be understood that since the power consumption of the TOF camera is usually large, the TOF camera can be turned off after the first image is captured, or the TOF camera can be turned off after the second display direction is obtained based on the first image to reduce the power consumption caused by the TOF camera.

[0252] S406: Acquire a second image and second data.

[0253] S407: If the fourth display direction is not obtained based on the second image, determine the fifth display direction according to the second data.

[0254] Optionally, the specific process of determining the fifth display direction according to the second data may include:

[0255] S4071: If the sixth display direction is not obtained based on the correspondence between the second data and the display direction, the fifth display direction is the same as the first display direction.

[0256] S4072: If a sixth display direction is obtained based on the correspondence between the second data and the display direction, determine whether the sixth display direction is the same as the first display direction.

[0257] That is, if the display direction can be determined based on the posture data of the electronic device at the second moment, it is further confirmed whether the sixth display direction (equivalent to the display direction determined based on the posture data at the second moment) is the same as the first display direction.

[0258] S4073: If the sixth display direction is the same as the first display direction, then the fifth display direction is the same as the first display direction.

[0259] That is, when the display direction (sixth display direction) determined based on the posture data at the second moment is the same as the first display direction, the display direction (sixth display direction) determined by the posture data collected by the first sensor (acceleration sensor) is used as the display direction (fifth display direction) of the second interface.

[0260] In one possible case, if the sixth display direction is different from the first display direction, the method further includes:

[0261] S4074: If the sixth display direction is different from the first display direction, turn on the TOF camera.

[0262] When the display direction (sixth display direction) determined based on the posture data at the second moment is different from the first display direction, in order to avoid the situation where the display direction determined based on the posture data of the electronic device is inconsistent with the display direction expected by the user, the electronic device can re-wake up the TOF camera and then use the TOF camera to collect the image for determining the display direction at the next moment.

[0263] S4075. Acquire a third image through the TOF camera.

[0264] The third image includes an image captured by the TOF camera at a third moment, and the third moment is a moment after the second moment.

[0265] S4076. Determine a seventh display direction based on the third image.

[0266] It should be understood that the specific process of determining the seventh display direction based on the third image is similar to the method steps shown in S2031 to S2033, and will not be repeated here.

[0267] S4077. Display the third interface based on the seventh display direction.

[0268] It should be understood that the third interface can be the same display interface as the first interface and the second interface, or it can be a different display interface, and the embodiment of the present application does not limit this.

[0269] It should be understood that the face loss fusion module in the application framework layer can be used to execute the method steps shown in S4071 to S4075 above.

[0270] S408: Display the second interface based on the fifth display direction.

[0271] S409: Acquire a fourth image.

[0272] The fourth image is the AO image captured by the ambient light camera at the first moment. Because the TOF camera consumes more power than the ambient light camera, if the clarity of the AO image captured by the ambient light camera is sufficient to determine the direction of the face in the image when the first brightness is greater than or equal to the preset threshold, the TOF camera can be turned off, and the AO image captured by the ambient light camera can be used to determine the first display direction.

[0273] S410: Determine a first display direction according to a fourth image.

[0274] S411: Display a first interface based on a first display direction.

[0275] The display method provided by the embodiment of the present application is applied to an electronic device, which includes a first sensor, a first camera, and a second camera, wherein the first sensor is used to collect posture data of the electronic device, the first camera is a full-resolution depth TOF camera, and the second camera is an ambient light camera for collecting AO images. The method includes: in response to a first operation, obtaining first data and a first brightness, and then when the first brightness is less than a preset threshold, obtaining a first image, and then determining a first display direction based on the first data and / or the first image; when the first brightness is greater than or equal to the preset threshold, obtaining a fourth image, and determining the first display direction based on the first data and / or the fourth image; and finally The first interface is displayed based on the first display direction. That is to say, by adopting the display method provided by the present application, when the ambient light is weak, the first display direction can be determined by using the first image captured by the TOF camera that is not affected by the ambient light intensity. When the ambient light is strong, the first display direction can be determined by using the AO image captured by the ambient light camera with lower power consumption. Therefore, when the ambient light is weak, the probability that the first display direction determined according to the first image and / or first data with higher clarity is consistent with the display direction expected by the user can be increased. When the ambient light is strong, the first display direction can be determined by using the AO image with lower power consumption to reduce the power consumption of the electronic device.

[0276] It should be understood that, although the various steps in the flow chart in the above-described embodiment are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated herein, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0277] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0278] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. It should be noted that the names of the modules in the embodiment of the present application are schematic and are not limited to the names of the modules in actual implementation.

[0279] FIG15 is a schematic structural diagram of a display device provided in an embodiment of the present application.

[0280] It should be understood that the display device 600 can execute the display method shown in Figures 5 to 14; the display device 600 includes: an acquisition unit 610 and a processing unit 620.

[0281] The acquisition unit 610 is configured to acquire a first brightness of an environment where the electronic device is located at a first moment;

[0282] The processing unit 620 is used to obtain a first image if the first brightness is less than a preset threshold, where the first image is an image captured by the first camera at a first moment; determine a first display direction based on the first image; and display a first interface based on the first display direction.

[0283] In one embodiment, the processing unit 620 is also used to: if the second display direction is obtained based on the first image, the first display direction is the same as the second display direction; if the second display direction is not obtained based on the first image, obtain first data, the first data being the posture data of the electronic device collected by the first sensor at the first moment; determine the third display direction based on the first data, the first display direction is the same as the third display direction.

[0284] In one embodiment, the acquisition unit 610 is also used to acquire key points in the IR image, which are used to indicate the locations of facial features in the IR image; the processing unit 620 is also used to determine the first angle corresponding to the IR image based on the key points, which refers to the angle between the face in the IR image and a preset direction; and determine the second display direction based on the first angle.

[0285] In one embodiment, the processing unit 620 is further configured to determine whether the IR image is a face image according to the depth image; if the IR image is a face image, determine the second display direction based on the IR image.

[0286] In one embodiment, the acquisition unit 610 is also used to acquire a second image and second data, the second image includes an image captured by the first camera at a second moment, the second moment is a moment after the first moment, and the second data is posture data captured by the first sensor at the second moment; the processing unit 620 is also used to determine the fifth display direction based on the second data if the fourth display direction is not obtained based on the second image; and display the second interface based on the fifth display direction.

[0287] In one embodiment, the processing unit 620 is further configured to: if the sixth display direction is not obtained based on the correspondence between the second data and the display direction, then the fifth display direction is the same as the first display direction.

[0288] In one embodiment, the processing unit 620 is also used to determine whether the sixth display direction is the same as the first display direction if a sixth display direction is obtained based on the correspondence between the second data and the display direction; if the sixth display direction is the same as the first display direction, the fifth display direction is the same as the first display direction.

[0289] In one embodiment, the processing unit 620 is also used to turn on the first camera if the sixth display direction is different from the first display direction; obtain a third image through the first camera, the third image includes an image captured by the first camera at a third moment, and the third moment is a moment after the second moment; determine the seventh display direction based on the third image.

[0290] In one embodiment, the processing unit 620 is also used to obtain a fourth image if the first brightness is greater than or equal to a preset threshold, where the fourth image is the image captured by the second camera at the first moment; determine the first display direction based on the fourth image; and display the first interface based on the first display direction.

[0291] In one embodiment, the processing unit 620 is further configured to turn off the first camera.

[0292] The display device provided in this embodiment is used to execute the display method of the above embodiment. The technical principles and technical effects are similar and will not be described in detail here.

[0293] It should be noted that the display device 600 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0294] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0295] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0296] Figure 16 shows a schematic diagram of the structure of an electronic device provided by the present application. The dotted lines in Figure 16 indicate that the unit or module is optional. The electronic device 700 can be used to implement the display method described in the above method embodiment.

[0297] The electronic device 700 includes one or more processors 701, which can support the electronic device 700 to implement the display method in the method embodiment. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0298] The processor 701 can be used to control the electronic device 700, execute software programs, and process data of the software programs. The electronic device 700 can also include a communication unit 705 to implement signal input (reception) and output (transmission).

[0299] For example, the electronic device 700 may be a chip, the communication unit 705 may be an input and / or output circuit of the chip, or the communication unit 705 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic device.

[0300] For another example, the electronic device 700 may be a terminal device, and the communication unit 705 may be a transceiver of the terminal device, or the communication unit 705 may be a transceiver circuit of the terminal device.

[0301] The electronic device 700 may include one or more memories 702 on which a program 704 is stored. The program 704 can be executed by the processor 701 to generate instructions 703, so that the processor 701 executes the impedance matching method described in the above method embodiment according to the instructions 703.

[0302] Optionally, data may be stored in the memory 702. Optionally, the processor 701 may read data stored in the memory 702. The data may be stored at the same storage address as the program 704, or may be stored at a different storage address than the program 704.

[0303] The processor 701 and the memory 702 may be provided separately or integrated together; for example, they may be integrated on a system on chip (SOC) of a terminal device.

[0304] Exemplarily, the memory 702 may be used to store a program 704 related to the display method provided in an embodiment of the present application. The processor 701 may be used to call the program 704 related to the display method stored in the memory 702 when displaying an image, and execute the display method of the embodiment of the present application; including: obtaining a first brightness of the environment in which the electronic device is located at a first moment;

[0305] If the first brightness is less than a preset threshold, a first image is acquired, where the first image is an image captured by the first camera at the first moment; a first display direction is determined based on the first image; and a first interface is displayed based on the first display direction.

[0306] The present application also provides a computer program product, which, when executed by the processor 701, implements the display method described in any method embodiment of the present application.

[0307] The computer program product may be stored in the memory 702 , for example, a program 704 , which is converted into an executable target file that can be executed by the processor 701 after undergoing preprocessing, compilation, assembly, and linking.

[0308] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements the display method described in any method embodiment of the present application. The computer program can be a high-level language program or an executable target program.

[0309] The computer-readable storage medium is, for example, memory 702. Memory 702 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0310] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0311] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0312] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0313] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0314] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of the units is merely a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0315] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0316] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0317] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A display method, characterized in that: The method is applied to an electronic device, the electronic device includes a first camera, the first camera is a full resolution depth TOF camera, and the method includes: Acquire a first brightness of an environment where the electronic device is located at a first moment; If the first brightness is less than a preset threshold, acquiring a first image, where the first image is an image captured by the first camera at the first moment; determining a first display direction according to the first image; A first interface is displayed based on the first display direction.

2. The method according to claim 1, characterized in that The electronic device further includes a first sensor, and the first sensor is used to collect posture data of the electronic device. The determining the first display direction according to the first image includes: If a second display direction is obtained based on the first image, the first display direction is the same as the second display direction; If the second display direction is not obtained based on the first image, obtaining first data, where the first data is the position and posture data of the electronic device collected by the first sensor at a first moment; A third display direction is determined based on the first data, and the first display direction is the same as the third display direction.

3. The method according to claim 2, characterized in that The first image comprises an infrared (IR) image, and the method further comprises: Acquire key points in the IR image, where the key points are used to indicate locations of facial features in the IR image; Determine a first angle corresponding to the IR image according to the key point, where the first angle refers to an angle between a face in the IR image and a preset direction; The second display direction is determined based on the first angle.

4. The method according to claim 3, characterized in that The first image also includes a depth image, and the method further includes: determining whether the IR image is a face image according to the depth image; If the IR image is a face image, the second display direction is determined based on the IR image.

5. The method according to any one of claims 2 to 4, characterized in that: After obtaining the second display direction based on the first image, the method further includes: Acquire a second image and second data, where the second image includes an image captured by the first camera at a second moment, where the second moment is a moment after the first moment, and the second data is position and posture data captured by the first sensor at the second moment; If the fourth display direction is not obtained based on the second image, determining a fifth display direction according to the second data; The second interface is displayed based on the fifth display direction.

6. The method according to claim 5, characterized in that The determining the fifth display direction according to the second data includes: If the sixth display direction is not obtained based on the correspondence between the second data and the display direction, the fifth display direction is the same as the first display direction.

7. The method according to claim 5, characterized in that The determining the fifth display direction according to the second data includes: If a sixth display direction is obtained based on the correspondence between the second data and the display direction, determining whether the sixth display direction is the same as the first display direction; If the sixth display direction is the same as the first display direction, the fifth display direction is the same as the first display direction.

8. The method according to claim 7, characterized in that The method further comprises: If the sixth display direction is different from the first display direction, turning on the first camera; Acquire a third image through the first camera, where the third image includes an image acquired by the first camera at a third moment, where the third moment is a moment after the second moment; A seventh display direction is determined based on the third image.

9. The method according to any one of claims 1 to 8, characterized in that: The electronic device further includes a second camera, and the second camera is used to collect an ambient light AO image. The method further includes: If the first brightness is greater than or equal to a preset threshold, acquiring a fourth image, where the fourth image is an image captured by the second camera at the first moment; determining the first display direction according to the fourth image; The first interface is displayed based on the first display direction.

10. The method according to any one of claims 1 to 9, characterized in that: After acquiring the first image, the method further includes: Turn off the first camera.

11. An electronic device, characterized in that: The electronic device comprises a module for executing the method according to any one of claims 1 to 10.

12. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the method according to any one of claims 1 to 10.

13. A chip system, characterized in that: The chip system includes a processor for calling and running a computer program from a memory, so that an electronic device equipped with the chip system executes the method according to any one of claims 1 to 10.

14. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

Images