Display method, electronic device, computer readable storage medium and program product

Abstract

The invention relates to the technical field of electronic equipment display, and provides a display method, electronic equipment, a computer readable storage medium and a program product, and is applied to the electronic equipment, the electronic equipment comprises a front camera module, and the front camera module comprises a first front camera. The method comprises the following steps: acquiring a first image through a first front-facing camera, and then determining a first mapping face distance according to a mapping relationship between a human eye pupil distance and a face distance in the first image; when the first mapping face distance is smaller than or equal to the preset distance threshold value, it is represented that the distance of using the electronic equipment by the user is too close, so that a more eye-protecting out-of-focus display mode is started, the eye using burden is relieved, eye fatigue is relieved, and the probability of myopia of the user is reduced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of electronic device display, and in particular to a display method, an electronic device, a computer readable storage medium, and a program product. BACKGROUND

[0002] With the development and popularization of electronic device technology, users use electronic devices more and more frequently, and spend more and more time using electronic devices. However, in the process of using electronic devices, if the user has bad usage habits, the eye burden of the user is likely to be aggravated, thereby causing eye fatigue, and in severe cases, myopia may be formed.

[0003] Therefore, when the user uses the electronic device, how to reduce the eye burden of the user, relieve the eye fatigue of the user, and reduce the possibility of myopia of the user has become a problem that needs to be solved in the prior art. SUMMARY

[0004] Embodiments of the present application provide a display method, an electronic device, a computer readable storage medium, and a program product, for reducing the eye burden of the user, relieving the eye fatigue of the user, and reducing the probability of myopia of the user.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a display method is provided, applied to an electronic device; the electronic device includes a front camera module; the front camera module includes a first front camera; the method includes:

[0007] acquiring a first image, the first image being acquired by the first front camera; determining a first mapped face distance according to a mapping relationship between a human eye pupil distance and a face distance in the first image; the mapping relationship is obtained by fitting the measured face distance and the human eye pupil distance; when the first mapped face distance is less than or equal to a preset distance threshold, starting a defocus display mode and displaying a prompt information; wherein the prompt information is used to indicate that the display mode of the electronic device is the defocus display mode.

[0008] In the first aspect, when the user uses the electronic device, the electronic device detects the distance between the user and the electronic device through the image acquired by the front camera. When the distance between the user and the electronic device is too close, the electronic device starts the more eye-protecting defocus display mode, thereby reducing the eye burden of the user, relieving the eye fatigue, and reducing the probability of myopia of the user.

[0009] In a possible implementation manner of the first aspect, the first front camera comprises an always-on (AO) mode, and the first image comprises an AO image captured by the first front camera in the AO mode. In this implementation manner, the image used to determine whether the face distance is too close is captured by the AO mode with lower power consumption, so that the device power consumption can be reduced.

[0010] In a possible implementation manner of the first aspect, when the first mapped face distance is less than or equal to a preset distance threshold, the defocus display mode is started and the prompt information is displayed, comprising: if the calibration condition is not met, it is determined that the distance mapping parameter does not need to be calibrated, and then when the first mapped face distance is less than or equal to the preset distance threshold, the defocus display mode is started and the prompt information is displayed; wherein the distance mapping parameter is a constant in an expression corresponding to the mapping relationship.

[0011] In this implementation manner, when the preset calibration condition is not met, the first mapped face distance is directly used to determine whether the defocus display mode is started, so that the parameter calibration is avoided frequently, thereby reducing the device power consumption and saving resources.

[0012] In a possible implementation manner of the first aspect, the front camera module further comprises a second front camera, and the second front camera is used to measure the distance; the display method further comprises: determining that the calibration condition is not met when the preset calibration period is not reached at the current time or the distance difference value is less than the preset error threshold; determining that the calibration condition is met when the preset calibration period is reached at the current time and the distance difference value is greater than or equal to the preset error threshold; wherein the distance difference value is a difference between the first measured face distance and the first mapped face distance; and the first measured face distance is measured by the second front camera.

[0013] In this implementation manner, the preset calibration period is used, and the calibration condition is determined to be met only when the error between the face distance measured by the second front camera and the face distance obtained by the expression mapping is too large, so that the frequency of parameter calibration can be reduced, thereby reducing the device power consumption and saving resources.

[0014] Further, because the parameter calibration needs to start the second front camera to capture images, the number of times of starting the second front camera can be reduced while the parameter calibration is reduced. Because the camera used to measure the distance is usually a high-power device, such as a TOF camera, the number of times of starting the second front camera is reduced, so that the device power consumption can be further avoided to increase.

[0015] In a possible implementation manner of the first aspect, the display method further includes: when the calibration condition is met, acquiring n infrared (IR) images; the n IR images are acquired by the second front camera, and the n IR images include corresponding second measured face distances; and a face deflection angle of the face in the n IR images is less than or equal to a preset angle threshold.

[0016] The distance mapping parameter in the mapping relationship corresponding expression is calibrated by using an interpupillary distance of the eyes in the n IR images and the second measured face distance, where n is a positive integer greater than or equal to i, and i is the highest order of the mapping relationship corresponding expression; the second mapping face distance is determined according to the interpupillary distance of the eyes in the first image and the expression after the parameter calibration; and when the second mapping face distance is less than or equal to a preset distance threshold, the defocus display mode is started and the prompt information is displayed.

[0017] In this implementation manner, when the preset calibration condition is met, the distance mapping parameter in the mapping relationship corresponding expression is calibrated, which can ensure the accuracy of the distance mapping parameter, thereby ensuring the accuracy of the mapping relationship corresponding expression, and improving the accuracy of the mapping face distance.

[0018] Further, the electronic device calibrates the parameter by using the IR image acquired by the second front camera in real time and the measured face distance, that is, the parameter is calibrated based on the actual situation when the user uses the electronic device, so that the accuracy of the parameter is ensured, and the adaptability of the distance mapping parameter to the user is improved, thereby improving the accuracy of the mapping face distance.

[0019] In a possible implementation manner of the first aspect, the display method further includes: updating the error threshold, and the updated error threshold is determined based on a difference between the second mapping face distance and the second measured face distance.

[0020] After the parameter calibration is completed, because the distance mapping parameter is updated, if the error threshold is determined based on the distance mapping parameter before the update, continuing to use this error threshold may affect the accuracy of subsequent parameter calibration judgment. Therefore, updating the error threshold after the parameter calibration can ensure the accuracy of subsequent judgment on whether to perform parameter calibration.

[0021] In a possible implementation manner of the first aspect, determining the first mapping face distance according to the mapping relationship between the interpupillary distance of the eyes and the face distance in the first image includes: performing eye gaze detection on the first image to determine whether the face in the first image gazes at the display screen of the electronic device; and when the face in the first image gazes at the display screen of the electronic device, determining the first mapping face distance according to the mapping relationship between the interpupillary distance of the eyes and the face distance in the first image.

[0022] In the case of determining that the user gazes at the display screen, the face mapping distance is detected, so that the detection of the face mapping distance is performed during the use of the electronic device by the user, and the accuracy of the opening of the defocus display mode is improved.

[0023] In a possible implementation of the first aspect, the first mapping face distance is determined according to the mapping relationship between the interpupillary distance and the face distance in the first image, including: detecting a face yaw angle of the face in the first image; when the face yaw angle is less than or equal to a preset angle threshold, determining the first mapping face distance according to the mapping relationship between the interpupillary distance and the face distance in the first image; when the face yaw angle is greater than the preset angle threshold, calculating the first mapping face distance according to the face size in the first image and the device resolution, and without determining whether the calibration condition is met.

[0024] In this implementation, when the face is a front face, the electronic device selects to calculate the mapping face distance by the interpupillary distance, and when the face is not a front face, the electronic device selects to calculate the mapping face distance by the face size, so as to avoid the influence of different postures of the user on the calculation of the mapping face distance, and ensure the accuracy of the mapping face distance.

[0025] In a possible implementation of the first aspect, the front camera module further includes a second front camera, and the second front camera is configured to measure a distance; and the first image is captured, including:

[0026] The light intensity is captured, and it is determined according to the light intensity whether the use scenario of the electronic device is a dark light scenario; in the non-dark light scenario, and when the duration of the display screen of the electronic device being in a bright screen state is greater than or equal to a preset duration, the first image is captured at a preset time interval; and the time difference between the capture times corresponding to two adjacent first images is equal to the time interval.

[0027] In this way, the first image is captured after the display screen is in the bright screen state for a long time, so that the scenario of the user using the electronic device for a short time is excluded. In addition, the first image is captured at a certain time interval, so that the defocus display mode is avoided from being frequently opened and closed, and the display effect is avoided from frequently changing.

[0028] In the dark light scenario, the first mapping face distance is measured by the second front camera, and without determining whether the calibration condition is met. In this implementation, because it is possible that the image including a clear face cannot be captured in the dark light scenario, the electronic device directly opens the second front camera to measure the face distance as the mapping face distance in the dark light scenario, so as to ensure the accuracy of the face distance.

[0029] In a possible implementation manner of the first aspect, the display method further includes: when the current application scenario is a preset target effective scenario, starting the defocus display mode and displaying prompt information. In this way, the defocus display mode can be avoided from being started in a scenario that does not support the defocus display mode, thereby saving resources.

[0030] In a possible implementation manner of the first aspect, the expression corresponding to the mapping relationship is as follows:

[0031] y=a*1 / x3+b*1 / x2+c*1 / x+d

[0032] wherein a, b, c, and d are distance mapping parameters, x is the interpupillary distance of the human eye, and y is the mapped face distance, and the mapped face distance includes the first mapped face distance or the second mapped face distance.

[0033] In a possible implementation manner of the first aspect, the display method further includes: collecting the bad eye use behavior and the personal basic information of the user; the personal basic information is collected after authorization of the user; predicting the risk degree of myopia of the user according to the bad eye use behavior and the personal basic information, and pushing the personalized eye protection prompt information based on the risk degree; wherein the bad eye use behavior is determined according to the use scenario, the application scenario, and the target face mapping distance of the electronic device; the use scenario is determined according to the inertial measurement unit data and the light intensity; and the target face mapping distance includes the first mapped face distance or the second mapped face distance.

[0034] In this way, the corresponding eye protection prompt information is pushed for different users, which can remind the user to pay attention to eye protection, thereby reducing the probability of myopia of the user.

[0035] In a second aspect, the present application provides an electronic device, comprising: a display screen, a front camera module, one or more processors and a memory; the front camera module, the display screen, and the memory are respectively coupled with the processor; the front camera module includes a first front camera; the memory stores one or more computer program codes, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device performs the following steps:

[0036] collecting a first image, the first image being collected by the first front camera; determining a first mapped face distance according to a mapping relationship between an interpupillary distance and a face distance in the first image; the mapping relationship is obtained by fitting the measured face distance and the interpupillary distance; when the first mapped face distance is less than or equal to a preset distance threshold, starting a defocus display mode and displaying prompt information; wherein the prompt information is used to indicate that the display mode of the electronic device is the defocus display mode.

[0037] In a possible implementation manner of the second aspect, the first front camera comprises an AO mode for collecting an AO image; and when the computer instructions are executed by the processor, the electronic device further performs the following steps: starting the AO mode of the first front camera, and collecting the AO image as the first image in the AO mode.

[0038] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: if the calibration condition is not met, determining that the parameter calibration of the distance mapping parameter is not needed, and starting the defocus display mode and displaying the prompt information when the first mapped face distance is less than or equal to the preset distance threshold; wherein the distance mapping parameter is a constant in an expression corresponding to the mapping relationship.

[0039] In a possible implementation manner of the second aspect, the front camera module further comprises a second front camera for measuring a distance; and when the computer instructions are executed by the processor, the electronic device further performs the following steps:

[0040] In a possible implementation manner of the second aspect, the calibration condition is determined based on whether the current time reaches a preset calibration period or whether a distance difference is less than a preset error threshold; the calibration condition is determined to be met when the current time reaches the preset calibration period and the distance difference is greater than or equal to the preset error threshold; and the distance difference is a difference between a first measured face distance and a first mapped face distance; and the first measured face distance is measured by the second front camera.

[0041] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: when the calibration condition is met, obtaining n infrared (IR) images; the n IR images are collected by the second front camera, and the n IR images comprise corresponding second measured face distances; and a face deflection angle of a face in the n IR images is less than or equal to a preset angle threshold; performing parameter calibration on a distance mapping parameter in an expression corresponding to the mapping relationship by using an interpupillary distance of an eye in the n IR images and the second measured face distance; n is a positive integer greater than or equal to i, i is a highest order of the expression corresponding to the mapping relationship; determining a second mapped face distance according to the interpupillary distance of the eye in the first image and the expression after the parameter calibration; and starting the defocus display mode and displaying the prompt information when the second mapped face distance is less than or equal to the preset distance threshold.

[0042] In a possible implementation manner of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: updating the error threshold, and the updated error threshold is determined based on a difference between the second mapped face distance and the second measured face distance.

[0043] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: performing human eye gaze detection on the first image to determine whether the face in the first image gazes at the display screen of the electronic device; and determining the first mapped face distance according to the mapping relationship between the interpupillary distance and the face distance of the human eye in the first image when the face in the first image gazes at the display screen of the electronic device.

[0044] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: detecting a face yaw angle of the face in the first image; determining the first mapped face distance according to the mapping relationship between the interpupillary distance and the face distance of the human eye in the first image when the face yaw angle is less than or equal to a preset angle threshold; and determining the first mapped face distance according to the face size and the device resolution in the first image when the face yaw angle is greater than the preset angle threshold, without determining whether the calibration condition is met.

[0045] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: collecting the light intensity, determining whether the use scenario of the electronic device is a dark light scenario according to the light intensity; collecting the first image at a preset time interval after the display screen of the electronic device is turned on for a duration greater than or equal to a preset duration in a non-dark light scenario; the time difference between the collection times of two adjacent first images is equal to the time interval; and measuring the first mapped face distance by using the second front camera in a dark light scenario, without determining whether the calibration condition is met.

[0046] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: when the current application scenario is a preset target effective scenario, enabling the defocus display mode and displaying the prompt information.

[0047] In a possible implementation of the second aspect, when the computer instructions are executed by the processor, the electronic device further performs the following steps: collecting the user's bad eye use behavior and personal basic information; the personal basic information is obtained after the user's authorization; predicting the risk degree of the user's myopia according to the bad eye use behavior and the personal basic information, and pushing the personalized eye protection prompt information based on the risk degree; the bad eye use behavior is determined according to the use scenario, the application scenario, and the target face mapping distance of the electronic device; the use scenario is determined according to the inertial measurement unit data and the light intensity; and the target face mapping distance includes the first mapped face distance or the second mapped face distance.

[0048] In a third aspect, the present application provides a computer readable storage medium having stored thereon a computer program, which, when executed by a processor in an electronic device, causes the electronic device to perform the display method according to the first aspect and any possible implementation thereof.

[0049] In a fourth aspect, the present application provides a computer program product, which, when run on a computer, causes the computer to perform the method according to the first aspect and any possible implementation thereof. The computer can be the electronic device described above.

[0050] It can be understood that the electronic device according to any possible implementation of the second aspect, the computer readable storage medium according to the third aspect, and the computer program product according to the fourth aspect can have the beneficial effects as described above with respect to the first aspect and any possible implementation thereof, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A scene schematic diagram of face distance provided for an embodiment of the present application;

[0052] Figure 2 A structural schematic diagram of an electronic device provided for an embodiment of the present application;

[0053] Figure 3 A software structural block diagram of an electronic device provided for an embodiment of the present application;

[0054] Figure 4 A flowchart of a display method provided for an embodiment of the present application Figure 1 ;

[0055] Figure 5 A principle schematic diagram of geometric imaging provided for an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a human eye pupil distance provided for an embodiment of the present application;

[0057] Figure 7 An interface schematic diagram of defocus display prompt provided for an embodiment of the present application;

[0058] Figure 8 A flowchart of a display method provided for an embodiment of the present application Figure 2 ;

[0059] Figure 9 A flowchart of a display method provided for an embodiment of the present application Figure 3 ;

[0060] Figure 10 A flowchart of a display method provided for an embodiment of the present application Figure 4 ;

[0061] Figure 11 An interface diagram of a defocus display switch provided by an embodiment of the present application;

[0062] Figure 12 A principle block diagram of a display method provided by an embodiment of the present application;

[0063] Figure 13 A structure block diagram of a chip system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to limit the present application. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, if the same items or similar items with basically the same functions and effects are distinguished by using “first”, “second”, etc. The person skilled in the art can understand that “first”, “second”, etc. do not limit the quantity and execution order, and “first”, “second”, etc. also do not necessarily mean different. In addition, in the description of the embodiments of the present application, unless otherwise specified, “multiple” means two or more.

[0065] Hereinafter, the technical terms related to the embodiments of the present application will be introduced.

[0066] Always on (AO) mode: The AO mode is a low-power consumption camera mode in which the camera is always on. That is, the camera is always in an on state in the AO mode. The image collected in the AO mode is called an AO image. Because the AO mode is a low-power consumption camera mode, in order to save power consumption, the collected AO image is usually a low-resolution single-channel image. Compared with an RGB image, the AO image can be regarded as a low-resolution grayscale image. For example, if the RGB image collected by the camera in the RGB mode is 640*240, the AO image collected by the camera in the AO mode can be 320*120.

[0067] In some embodiments, the camera can only include the AO mode or the RGB mode. Alternatively, the camera can also include both the AO mode and the RGB mode. Taking the front camera of a mobile phone as an example, the front camera can include both the AO mode and the RGB mode. During the user's selfie, the front camera of the mobile phone can switch to use the RGB mode to collect a selfie image in the RGB format for the user, to ensure the quality of the selfie image. During the user's non-selfie, the front camera can be switched to the AO mode.

[0068] Time of flight (TOF) camera: A TOF camera is a camera that uses the time of flight principle to measure distance. A TOF camera calculates the distance between a target object and the camera by emitting a light pulse to the target object and measuring the time it takes for the light pulse to travel from the emission to the reflection back from the target object. In some embodiments, a TOF camera can measure distance by emitting infrared light. In this case, the TOF camera can obtain an infrared (IR) image containing depth information (i.e., measured distance).

[0069] With the development and popularization of electronic device technology, users use electronic devices more and more frequently, and spend more and more time using electronic devices. For example, some users use mobile phones, tablets and other electronic devices for work at a high frequency and for a long time, and some users use mobile phones, tablets and other electronic devices for learning at a high frequency and for a long time.

[0070] However, during the use of electronic devices, if the user has bad habits, such as frequently using electronic devices at close range, or using electronic devices for a long time in environments such as walking, eating, and shaking in a car, dim light, and direct sunlight, the user's eye burden is likely to be increased, leading to eye fatigue, and in severe cases, myopia. Therefore, during the use of electronic devices by users, how to reduce the eye burden of users, relieve the eye fatigue of users, and reduce the possibility of myopia of users has become a problem that needs to be solved.

[0071] Based on this, the embodiment of the present application provides a display method.

[0072] The display method provided by the embodiment of the present application is applied to an electronic device, and the electronic device includes a front camera module. The front camera module of the electronic device includes a first front camera and a second front camera. The first front camera includes an AO mode, that is, the first front camera is a camera integrated with an AO mode. Therefore, the first front camera of the electronic device can collect an AO image in the AO mode. The second front camera is a camera for measuring distance in the electronic device, and the second front camera is used to measure face distance in the embodiment of the present application. In a specific embodiment, the second front camera can be a TOF camera. The TOF camera can collect an IR image containing depth information (i.e., measured distance).

[0073] In brief, in the embodiments of the present application, during the use of the electronic device by the user, the first front camera can be used to capture a first image, such as an AO image. Then, the distance between the user and the electronic device, i.e., the face distance, can be determined through the mapping relationship between the interpupillary distance and the face distance in the AO image. In some embodiments, the mapping relationship can be a mapping table in which corresponding face distances are configured for different interpupillary distances. Alternatively, the mapping relationship can also be a corresponding expression, and by inputting the interpupillary distance into the expression, the corresponding face distance can be obtained. In a specific embodiment, the electronic device can calculate the face distance based on the interpupillary distance and the distance mapping parameter in the corresponding expression of the mapping relationship.

[0074] Exemplary, Figure 1 A scene schematic diagram of a face distance is shown.

[0075] As Figure 1 shown, during the use of the electronic device by the user, the distance between the face of the user and the electronic device is the mapping face distance referred to in the embodiments of the present application. In some embodiments, the face distance can also be referred to as the use distance. For the convenience of scheme description and differentiation, the face distance determined based on the mapping relationship will be referred to as the mapping face distance hereinafter in the embodiments of the present application.

[0076] Then, in the case where the mapping face distance determined by the electronic device is relatively close, indicating that the distance between the user and the electronic device is too close, the electronic device can switch the display mode of the electronic device to the defocus display mode.

[0077] The defocus display mode is a display mode designed based on the defocus mirror. After the electronic device is started in the defocus display mode, the electronic device can adjust the display content of the electronic device through the artificial intelligence (AI) technology, so as to simulate the display effect of the defocus mirror on the display content of the electronic device.

[0078] The defocus mirror is a myopia control lens with a defocus design. The principle is that light forms a clear image on the retina, and another part of light is focused in front of the retina to form a myopia defocus signal, thereby playing a role in inhibiting the deepening of myopia, correcting vision, and controlling the growth of myopia. In brief, the defocus mirror adjusts the peripheral visual light to focus in front of the retina, thereby sending a biological signal to the eye to grow towards the light focusing position in front of the retina, and thus slowing down the growth of the eye axis and controlling myopia.

[0079] In this way, the electronic device, when determining that the face distance is too close, adjusts the display content by starting the defocus display mode, simulates the display effect of the defocus mirror, that is, is equivalent to wearing defocus glasses for the user when the user uses the electronic device, so as to achieve a certain eye protection effect without disturbing the user, thereby reducing the user's eye burden, relieving eye fatigue, and reducing the probability of myopia.

[0080] At the same time, experimental data shows that after the user reads for a long time (25 minutes in the experimental data) on the electronic device starting the defocus display mode (i.e., the AI eye protection technology), the index of myopia for near vision is reduced by an average of 13 degrees, and the highest is reduced by 75 degrees.

[0081] In addition, in some embodiments, after the electronic device starts the defocus display mode, the electronic device can also adjust the display parameters such as display brightness and contrast based on the use scene, environment, or light of the electronic device, to further achieve the effect of eye protection, thereby reducing the eye burden, relieving eye fatigue, and reducing the probability of myopia.

[0082] In addition, if the electronic device uses the AO mode of the first front camera to collect the AO image as the first image for face distance detection, because the AO mode is a kind of always-on low-power camera mode, the device power consumption can also be avoided.

[0083] Of course, if the actual business requirement does not consider the problem of high device power consumption, the first image collected by the electronic device can also be a non-AO image. For example, the electronic device can also collect an RGB image in RGB mode, and use the collected RGB image as the first image.

[0084] Further, the electronic device can also periodically start the second front camera (i.e., the TOF camera) to measure the actual face distance, and then update the mapping relationship based on the actual measured face distance. For example, the distance mapping parameter in the expression corresponding to the mapping relationship can be recalibrated. In this way, the accuracy of the mapping relationship, such as the accuracy of the distance mapping parameter, can be ensured, thereby ensuring the accuracy of the mapped face distance and improving the accuracy of the start of the defocus display mode. Moreover, because the TOF camera is periodically started, compared with the always-on TOF camera distance measurement, the device power consumption can also be saved.

[0085] In some embodiments, the aforementioned electronic device may include at least one of the following: mobile phone, foldable electronic device, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device, or smart city device. This application embodiment does not impose any special limitation on the specific type of the electronic device 100.

[0086] For example, Figure 2 A schematic diagram of the structure of an electronic device is shown.

[0087] like Figure 2 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) connector 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera module 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0088] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0089] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0090] The processor 110 can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching instructions and executing instructions. Specifically, in the embodiments of the present application, the processor 110 can control the first front camera to capture an AO image, and then calculate a mapped face distance based on the mapping relationship between the interpupillary distance and the face distance of the human eye in the AO image. Next, the processor 110 determines whether to turn on the defocus display mode based on the calculated mapped face distance.

[0091] In addition, the processor 110 can also periodically control the second front camera to be turned on, that is, periodically control the TOF camera to measure the actual face distance, and update the mapping relationship based on the measured face distance, such as updating the distance mapping parameter, to ensure the accuracy of the mapping relationship, such as the distance mapping parameter, so as to ensure the accuracy of the mapped face distance.

[0092] The memory in the processor 110 can also be configured to store instructions and data. In some embodiments, the memory in the processor 110 can be a cache memory. The memory can save instructions or data that have been used or used frequently by the processor 110. If the processor 110 needs to use the instructions or data, it can directly call from the memory. Avoid repeated access and reduce the waiting time of the processor 110, thereby improving the efficiency of the system.

[0093] In some embodiments, the processor 110 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 110 can connect to modules such as touch sensors, audio modules, wireless communication modules, displays, and camera modules through at least one of these interfaces.

[0094] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0095] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, it can save music, video, and other files to the external memory card, or transfer music, video, and other files from the electronic device to the external memory card.

[0096] Internal memory 121 can be used to store computer executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phone book, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional methods or data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0097] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0098] Electronic device 100 can implement display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0099] The display screen 194 is used to display images, videos, and prompts. For example, in this embodiment, after the off-focus display mode is activated, the display screen can display prompts to inform the user that the current display mode of the electronic device is off-focus display mode.

[0100] The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized LED, a microLED, a quantum dot LED, or a QLED. In some embodiments, the electronic device 100 may include one or more display screens 194.

[0101] Electronic device 100 can realize camera function through camera module 193, ISP, video codec, GPU, display screen 194, application processor AP, neural network processor NPU, etc.

[0102] The camera module 193 can be used to acquire color image data and depth data of the subject. The Information Service Provider (ISP) can be used to process the color image data acquired by the camera module 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the photosensitive element transmits this electrical signal to the ISP for processing, converting it into a visible image. The ISP can also perform algorithmic optimization of image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera module 193.

[0103] In some embodiments, the electronic device 100 may include one or more camera modules 193. Specifically, such as Figure 2 As shown, the electronic device 100 may include a front-facing camera module and a rear-facing camera module.

[0104] The front-facing camera module is typically used to capture image data and depth data (i.e., depth information) of the photographer facing the display screen 194. The rear-facing camera module is used to capture color image data and depth data of the subject (such as a person, landscape, etc.) being photographed. In some embodiments, the camera module 193 may also consist of two or more cameras.

[0105] Specifically, in the embodiments of this application, such as Figure 2As shown, the front-facing camera module 193 may include a first front-facing camera and a second front-facing camera (such as a front-facing TOF camera). The first front-facing camera integrates an AO mode, thus enabling it to capture an AO image of the user facing the display screen 194. The second front-facing camera, the TOF camera, can capture an IR image of the user facing the display screen 194 and measure the distance between the electronic device 100 and the user, i.e., depth information.

[0106] In some embodiments, the camera module 193 may consist of a color camera module and a 3D sensing module.

[0107] In some embodiments, the photosensitive element of the camera in the color camera module can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats.

[0108] In some embodiments, the 3D sensing module can be a time-of-flight (TOF) 3D sensing module or a structured light 3D sensing module. Structured light 3D sensing is an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, etc. The working principle of a structured light 3D sensing module is to first emit a specific pattern of light onto the object being photographed, then receive the light coding on the object's surface, compare it with the original projected light pattern, and calculate the object's three-dimensional coordinates using triangulation principles. These three-dimensional coordinates include the distance between the electronic device 100 and the object being photographed. Similarly, TOF 3D sensing can be an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, etc. The working principle of a TOF 3D sensing module is to calculate the distance (i.e., depth) between the TOF 3D sensing module and the object being photographed by measuring the infrared reflection time to obtain a 3D depth map.

[0109] Structured light 3D sensing modules can also be applied to facial recognition, motion-sensing game consoles, and industrial machine vision inspection. Time-of-flight (TOF) 3D sensing modules can also be applied to game consoles, augmented reality (AR) / virtual reality (VR) and other fields.

[0110] In some embodiments, the CPU, GPU, or NPU in processor 110 can process the color image data and depth data acquired by camera module 193. In some embodiments, the NPU can identify the skeletal points of the subject by using neural network algorithms based on skeletal point recognition technology, such as convolutional neural network algorithms (CNN). The CPU or GPU can also run neural network algorithms to determine the skeletal points of the subject based on the color image data.

[0111] Digital signal processors (DSPs) are used to process digital signals, and can also process other digital signals. For example, when electronic device 100 selects a frequency point, the DSP is used to perform Fourier transforms on the frequency energy, etc.

[0112] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0113] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0114] Electronic device 100 can implement audio functions such as music playback and recording through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0115] The sensor module 180 may include an accelerometer 180A, a gyroscope 180B, and an ambient light sensor 180C. In other embodiments, the sensor module 180 may also include a pressure sensor, a barometric pressure sensor, a magnetic sensor, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, a bone conduction sensor, etc.

[0116] The accelerometer 180A can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic device, and can be applied to applications such as screen orientation switching and pedometers.

[0117] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and controls the lens to move in the opposite direction to counteract the shake of the electronic device 100, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0118] The ambient light sensor 180C can be used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of its display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180C can also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180C can also work in conjunction with a proximity sensor to detect whether the electronic device 100 is obstructed, such as when the electronic device is in a pocket. When obstruction or being in a pocket is detected, certain functions (such as touch functionality) can be disabled to prevent accidental operation.

[0119] In this embodiment, the electronic device can determine the user's usage scenario based on data from the accelerometer 180A and gyroscope 180B. For example, whether the user is walking, eating, or using the electronic device in a swaying vehicle. Additionally, the electronic device can collect light intensity L from the ambient light sensor 180C, and then determine whether the user is using the electronic device in a low-light environment based on the light intensity L.

[0120] In some embodiments, the software system of the aforementioned electronic device 100 may adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture of Android... TM Taking the system as an example, the software structure of electronic device 100 is illustrated.

[0121] For example, Figure 3 A software architecture block diagram of an electronic device is shown.

[0122] Layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 3As shown, from top to bottom, they are the application layer, application framework layer, Android runtime (ART) and native C / C++ library (i.e., system library layer), hardware abstraction layer (HAL), and kernel layer.

[0123] The application layer can include a series of application packages.

[0124] like Figure 3 As shown, the application package may include applications such as a display engine, gallery, calendar, map, WLAN, SMS, video, call, navigation, and Bluetooth. In this embodiment, the display engine decides whether to enable the off-focus display mode based on the mapped face distance reported from the lower layer.

[0125] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0126] like Figure 3 As shown, the application framework layer may include a window manager, activity manager, input manager, resource manager, notification manager, view system, content provider, etc.

[0127] The window manager provides a window management service (WMS), which can be used for window management, window animation management, surface management, and as a relay station for the input system.

[0128] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0129] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0130] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0131] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0132] The Activity Manager Service (AMS) can be used to start, switch, and schedule system components (such as activities, services, content providers, and broadcast receivers), as well as manage and schedule application processes.

[0133] The input manager can provide an input management service (IMS), which can be used to manage system inputs, such as touchscreen input, keypad input, and sensor input. IMS retrieves events from input device nodes and, through interaction with the WMS, distributes these events to the appropriate windows.

[0134] The Android runtime consists of the core libraries and the Android runtime itself. The Android runtime is responsible for converting source code into machine code. The Android runtime primarily employs ahead-of-time (AOT) compilation and just-in-time (JIT) compilation technologies. The core libraries mainly provide basic Java class library functionalities, such as libraries for basic data structures, mathematics, I / O, tools, databases, and networking. The core libraries provide APIs for users to develop Android applications.

[0135] A native C / C++ library (system library layer) can include multiple functional modules. For example... Figure 3 As shown, native C / C++ libraries can include distance management, surface manager, media framework, libc, OpenGLES, SQLite, Webkit, etc.

[0136] The distance management platform responds to the needs of the upper-layer display engine by calling relevant modules in the HAL layer to detect and determine the distance to the mapped face and report it to the display engine. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media framework supports playback and recording of various common audio and video formats, as well as still image files. The media library supports various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. OpenGL ES provides drawing and manipulation of 2D and 3D graphics in applications. SQLite provides a lightweight relational database for applications on the electronic device.

[0137] The Hardware Abstraction Layer (HAL) runs in user space, encapsulates kernel-level drivers, and provides calling interfaces to higher layers. For example... Figure 3 As shown, the hardware abstraction layer may include a human eye gaze detection module, a parameter calibration module, a distance detection module, a display HAL, an audio HAL, a camera HAL, etc.

[0138] The human eye gaze detection module is used to perform face detection and gaze detection on the AO images captured by the front-facing camera.

[0139] The distance detection module is used to calculate the distance to the mapped face. Specifically, the distance detection module can calculate the distance to the mapped face based on the interpupillary distance in the AO image and the distance mapping parameters.

[0140] The parameter calibration module is used to update the distance mapping parameters. Specifically, the parameter calibration module can acquire the IR image captured by the TOF camera and the corresponding TOF face distance to update the distance mapping parameters.

[0141] The kernel layer is the layer between hardware and software. The kernel layer includes at least the display driver, camera driver, audio driver, and Bluetooth driver.

[0142] The following is combined with Figure 2 The hardware of the electronic device shown Figure 3 The software of the electronic device shown exemplifies the workflow when the electronic device executes the display method provided in the embodiments of this application.

[0143] After the electronic device is powered on, the upper-layer display engine can register its distance detection capabilities with the distance management platform. Upon receiving the registration from the display engine, the distance management platform invokes the distance detection capabilities of the HAL layer. The HAL layer then drives the electronic device's first front-facing camera to capture autofocus (AO) images via the kernel layer. Next, the human eye gaze detection module and the distance detection module within the HAL layer calculate the mapped face distance based on the AO images uploaded by the first front-facing camera and the distance mapping parameters in the mapping relationship expression.

[0144] When parameter calibration is not required, the HAL layer directly reports the calculated mapped face distance to the distance control platform, which then relays it back to the upper-layer display engine. However, when distance mapping parameter calibration is needed, the HAL layer further drives the second front-facing camera (i.e., the TOF camera) to acquire an IR image including the TOF distance. The parameter calibration module in the HAL layer then updates the distance mapping parameters based on the interpupillary distance in the IR image and the measured TOF distance. The distance detection module recalculates the new mapped face distance based on the updated distance mapping parameters, and the HAL layer reports the recalculated mapped face distance to the distance control platform, which then relays it back to the upper-layer display engine.

[0145] After receiving the mapped face distance reported by the distance control platform (either the mapped face distance before or after the update), the display engine decides whether to enable the off-focus display mode based on the reported mapped face distance. When the off-focus display mode is enabled, it triggers display effect adjustments to reduce the user's eye strain, alleviate eye fatigue, and lower the probability of myopia.

[0146] The display method proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that the display methods in the following embodiments can all be implemented in the electronic device 100 having the above-described hardware structure.

[0147] For example, Figure 4 A flowchart illustrating a display method is shown. The following, in conjunction with... Figure 4 The display method provided in the embodiments of this application will be described in detail.

[0148] like Figure 4 As shown, the display method may include steps S401-S405.

[0149] S401, Electronic equipment acquires AO images.

[0150] When the screen of an electronic device is on, the first front-facing camera captures AO images (i.e., the first image) at certain intervals. Understandably, if the first front-facing camera integrates both AO and RGB modes, then during selfies, the first front-facing camera needs to switch to RGB mode, so it cannot capture AO images during selfies.

[0151] In some embodiments, in order to improve the accuracy of subsequent detection, after acquiring the AO image, preprocessing can be used to further exclude abnormal images that do not support subsequent detection, such as those without faces, excessively large face occlusion areas, eyes that are occluded, faces with low confidence, overly dark or overexposed images, and blurred faces.

[0152] In some embodiments, the electronic device may begin acquiring AO images after the display screen has been continuously lit for P minutes, thereby excluding scenarios where the user briefly uses the electronic device. For example, it can exclude scenarios where the user briefly views notification information. Here, P is a preset duration, which can be set according to actual needs, and this application embodiment does not impose any limitation on it.

[0153] For example, an electronic device can acquire an AO image after the display screen has been continuously lit for P = 5 minutes.

[0154] In some embodiments, to avoid the problem of frequent switching of display effects caused by frequently turning the off-focus display mode on and off (i.e., the ping-pong switching problem), the electronic device can acquire AO images at a certain period.

[0155] Specifically, the electronic device can acquire AO images at certain time intervals. For example, after acquiring one AO ​​image, the electronic device can acquire the next AO image after a 10-second interval. That is, the time difference between the acquisition times of two adjacent AO images is equal to the time interval. This allows sufficient processing time for adjusting the display effect, thereby avoiding frequent switching of the defocus display mode on and off, which adds lag to the adjustment of the display effect and avoids ping-pong jumps.

[0156] It is understood that the 10-second time interval mentioned in the embodiments of this application is merely an example of an embodiment of this application and does not constitute any limitation on the acquisition cycle of AO images. For example, the time interval for acquiring AO images can also be 5 seconds, 20 seconds, etc., which can be set according to actual business needs.

[0157] S402, Electronic devices perform human eye gaze detection.

[0158] After the electronic device obtains the AO image captured by the first front-facing camera, it performs eye gaze detection on the AO image. In this embodiment, eye gaze detection includes two parts: face detection and gaze detection.

[0159] In this process, face detection determines whether the AO image contains a face, and gaze detection, if the AO image contains a face, further determines whether the user is gazing at the display screen of the electronic device. It is understood that face detection and gaze detection can be performed using any existing method, and this application embodiment does not impose any limitations on them. For example, the electronic device can call a trained neural network model to perform face detection and gaze detection on the AO image.

[0160] If the electronic device detects that the AO image does not contain a face, or if the AO image contains a face but the user is not looking at the electronic device's display, it indicates that although the electronic device is currently on, the user is not using the device. In this case, there is no need to further determine whether to activate the defocus display mode to reduce eye strain. Therefore, if the electronic device determines that no one is looking at it, it directly ends the current process and waits for the next AO image to be acquired.

[0161] If the electronic device determines that the AO image includes a face, and if gaze detection confirms that the face in the AO image is gazing at the electronic device's display screen, then it can determine that the user is currently using the electronic device. Therefore, the electronic device can further calculate the mapped face distance and determine whether to activate the off-focus display mode based on this distance. That is, the electronic device executes step S403.

[0162] S403, the electronic device calculates the first mapped face distance based on the human eye interpupillary distance and distance mapping parameters in the AO image.

[0163] The distance mapping parameters are pre-fitted based on the measured face distance and the measured interpupillary distance in the image. In other words, using the fitted distance mapping parameters, the corresponding face distance y can be obtained based on the interpupillary distance x in the image.

[0164] In this embodiment of the application, the expression (1) for the interpupillary distance x and the face distance y in the image is known as follows:

[0165] y = a*1 / x³ + b*1 / x² + c*1 / x + d.

[0166] Alternatively, y = a / x + b.

[0167] Where a, b, c, and d are distance mapping parameters.

[0168] For ease of distinction, in this embodiment, the mapped face distance calculated by the electronic device based on the interpupillary distance and distance mapping parameters a, b, c, and d in the AO image is referred to as the first mapped face distance. Alternatively, the mapped face distance calculated by the electronic device based on the interpupillary distance and distance mapping parameters a and b in the AO image is referred to as the first mapped face distance.

[0169] For example, Figure 5 A schematic diagram illustrating the principle of geometric imaging is shown.

[0170] refer to Figure 5In the geometric imaging process, since the interpupillary distance x1 in the image, the distance d1 between the pupils, the actual interpupillary distance, and the face distance d2 are always related as: x1 / d1 = x2 / d2. Therefore, it can be seen that the face distance d2 to be calculated is inversely proportional to the interpupillary distance x1 in the image.

[0171] Based on this, the measured face distance and the interpupillary distance in the image can be fitted using an inverse proportional relationship to obtain the corresponding distance mapping parameters. As described in expression (1) above, the fitting of the mapping relationship can be a binary quartic relation, thus determining the distance mapping parameters a, b, c, and d. The fitting of the mapping relationship can also be a binary linear relation, thus determining the distance mapping parameters a and b.

[0172] Then, regardless of whether the distance mapping parameters are a, b, c and d, or a and b, the determined distance mapping parameters are fixed after the fitting is completed, thus obtaining the above calculation expression (1). According to actual tests, in the field of face distance calculation, the binary quartic relation is more stable than the binary linear relation.

[0173] In some embodiments, the interpupillary distance can be calculated using the coordinates of key points on the eyes. That is, when an electronic device performs face detection on an AO image, if a face is present in the AO image, it can further perform facial key point detection to obtain the coordinates of these key points. These facial key point coordinates can include the coordinates of five locations: the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth. Then, in this embodiment, the distance between the left and right eyes can be calculated using the two coordinates corresponding to the left and right eyes, thus obtaining the interpupillary distance. For example, Figure 6 A schematic diagram of human interpupillary distance is shown.

[0174] S404, the electronic device determines whether the distance to the first mapped face is less than or equal to a distance threshold.

[0175] When the distance to the first mapped face is less than or equal to a distance threshold, meaning the user is too close to the electronic device, the eye strain is greater, easily causing eye fatigue and increasing the probability of myopia. Therefore, to reduce eye strain, the electronic device can switch the display mode to defocus display mode. That is, the electronic device executes S405.

[0176] When the distance to the first mapped face is greater than the distance threshold, meaning the user is at an appropriate distance from the electronic device and not too close, the eye strain is relatively light. Therefore, the electronic device can determine that it does not need to activate the defocus display mode for the time being. The electronic device can then directly end the current process and wait for the next acquired AO image.

[0177] It should be noted that, in this embodiment of the application, the distance threshold can be set according to the actual test situation and business needs, and this embodiment of the application does not impose any limitations on it.

[0178] For example, the distance threshold could be 20cm. Then, when the distance to the first mapped face is less than or equal to 20cm, the electronic device decides to activate the off-focus display mode. When the distance to the first mapped face is greater than 20cm, the electronic device decides not to activate the off-focus display mode. Alternatively, the distance threshold could also be 10cm or 30cm. Then, when the distance to the first mapped face is less than or equal to 10cm or 30cm, the electronic device decides to activate the off-focus display mode.

[0179] S405, the electronic device activates the off-focus display mode and displays a prompt message.

[0180] When an electronic device determines that a user is too close to the device based on the first mapped facial distance, it can activate an off-focus display mode, switching the current display mode to a more eye-friendly one. Once activated, the device can adjust the displayed content to simulate the effect of the user wearing off-focus lenses. Furthermore, the device can also adjust display parameters such as brightness and contrast.

[0181] Furthermore, after the display mode is switched to off-focus display mode, the electronic device can also display corresponding prompts to inform the user that the current display mode of the electronic device is off-focus display.

[0182] In some embodiments, the display method of the prompt information can be specifically set according to actual needs, and this application embodiment does not impose any limitations on it. For example, the prompt information can be displayed in a drop-down notification bar. Another example is that the prompt information can be displayed in the form of a notification capsule. Yet another example is that the prompt information can be displayed as a pop-up window.

[0183] For example, Figure 7 A schematic diagram of an interface for displaying a defocus indicator is shown. For example... Figure 7 As shown, Figure 7 The prompt message "Off-focus display" in the interface shown in (1) is displayed in the form of a notification capsule. Figure 7 The prompt message "Off-focus display enabled" in the interface shown in (2) is displayed in the drop-down notification bar.

[0184] Therefore, in this embodiment of the application, since the off-focus display mode has a certain eye protection effect compared with other display modes, when the electronic device determines that the distance of the face is close and the eye burden is heavy by the image captured by the front camera, switching to the off-focus display mode can reduce the eye burden and thus reduce the probability of eye fatigue and myopia.

[0185] Furthermore, since electronic devices use AO images to determine whether to enable the off-focus display mode, and AO images are acquired based on the low-power camera mode of AO, this avoids increasing the power consumption of electronic devices compared to acquiring RGB images or directly using a TOF camera to measure face distance.

[0186] In some embodiments, the position and state of the human eyes in the AO image may vary depending on the user's posture when using the electronic device, and correspondingly, the interpupillary distance will also differ. Therefore, to ensure the accuracy of the mapped face distance calculation, the electronic device can first determine whether the face in the AO image is a frontal face before calculating the mapped face distance.

[0187] Furthermore, if the face in the AO image is a frontal view, the electronic device calculates the first mapped face distance based on the interpupillary distance and distance mapping parameters in the AO image. However, if the face in the AO image is not a frontal view, the mapped face distance calculated based on the interpupillary distance of that face may contain errors. Therefore, the electronic device may not calculate the mapped face distance in this case and wait for the next AO image acquisition.

[0188] In other embodiments, if the face in the AO image is not a frontal view, the electronic device can additionally calculate the distance to the mapped face based on the size of the face.

[0189] For example, Figure 8 A flowchart of a display method is shown.

[0190] like Figure 8 As shown, when the electronic device detects eye gaze in the AO image, it further determines whether the face deflection angle in the AO image is less than or equal to an angle threshold. If the face deflection angle is less than or equal to the angle threshold, the electronic device determines that the face in the current OA image is a frontal face. Therefore, the electronic device calculates the first mapped face distance based on the interpupillary distance and distance mapping parameters in the AO image, as described in the above expression (1).

[0191] When the face deflection angle is greater than the angle threshold, the electronic device determines that the face in the current OA image is not a frontal face. Therefore, the electronic device can calculate the first mapped face distance based on the face size and device resolution. The expression for the face distance (2) is as follows:

[0192]

[0193] Where dist is the calculated face distance. Width w and height h are the width and height of the face in the AO image. w0 and h0 are the resolution of the electronic device, i.e., the device resolution. In this way, when the face is not frontal, the mapped face distance can be calculated using the face size and device resolution, thus ensuring the accuracy of the calculated face distance.

[0194] In this embodiment, the face deflection angle can be calculated using any existing head pose estimation method. For example, an electronic device can use a point-to-point (PNP) pose algorithm to obtain the face deflection angle of the face in the AO image.

[0195] Specifically, the electronic device acquires facial landmarks obtained from face detection. Then, these facial landmarks are used as input to the PNP pose algorithm to obtain the head pose, i.e., the face rotation angle. In this embodiment, the face rotation angle includes pitch, yaw, and roll, corresponding to head tilting, head shaking, and head turning, respectively. Furthermore, when the face rotation angles pitch, yaw, and roll are all less than or equal to an angle threshold, the face is determined to be frontal. Conversely, if any of pitch, yaw, or roll has an angle value greater than the angle threshold, the face is not frontal.

[0196] The angle threshold can be set according to actual needs, and this application embodiment does not impose any limitation on it. For example, the angle threshold can be set to 5 degrees. When the pitch angle, yaw angle, and roll angle are all ≤5 degrees, the face is determined to be a frontal face; otherwise, the face is determined to be a non-frontal face.

[0197] In some embodiments, since off-focus display is not supported in certain application scenarios, after determining that the distance to the first mapped face is less than a distance threshold, it can be further determined whether the current application scenario of the electronic device belongs to the target effective scenario. The electronic device only activates the off-focus display mode when the current application scenario is the target effective scenario. It is understood that the target effective scenario can be set based on actual conditions, and this application embodiment does not impose any limitations on it.

[0198] For example, the target application scenario can include video scenarios, reading scenarios, etc. Therefore, the electronic device will only activate the off-focus display mode when the current application scenario of the electronic device is a game application, video application, or reading application. If the current application scenario of the electronic device is not a game application, video scenario, or reading scenario, the electronic device will not activate the off-focus display mode even if the distance to the first mapped face is less than or equal to the distance threshold.

[0199] In one specific embodiment, the electronic device can determine whether the current application scenario is the target effective scenario based on the current focused window. For example, if "xx video application" is the set target effective scenario, and the current focused window of the electronic device is the window of "xx video application", the electronic device can determine that it is currently in the target effective scenario.

[0200] In some embodiments, since the key to whether to enable the off-focus display mode lies in whether the calculated first mapped face distance is less than or equal to a distance threshold, the accuracy of the first mapped face distance detection directly affects the precision of the decision on whether to enable the off-focus display mode. Furthermore, since the first mapped face distance is obtained by mapping the face distance based on the distance mapping parameters in the expression corresponding to the mapping relationship, given that the interpupillary distance in the image is known, the accuracy of the distance mapping parameters directly affects the accuracy of the first mapped face distance.

[0201] Meanwhile, in practical applications, the same electronic device may be used by different users, or the same user may use the device in different environments. Therefore, the pre-configured distance mapping parameters a, b, c, and d may contain errors, easily affecting the accuracy of the initial face distance mapping and consequently the precision of the decision on whether to enable the off-focus display mode. Therefore, to ensure the accuracy of the distance mapping parameters and improve the accuracy of the face distance mapping to ensure the precision of the decision on whether to enable the off-focus display mode, the electronic device can periodically calibrate the distance mapping parameters to reduce errors.

[0202] For example, Figure 9 A flowchart illustrating another display method is shown below. Figure 9 The calibration process for the distance mapping parameters is explained in detail.

[0203] S901, electronic equipment acquires AO images.

[0204] S902, Electronic devices perform human eye gaze detection.

[0205] S903, the electronic device calculates the first mapped face distance based on the human eye pupil distance in the AO image and the first distance mapping parameters.

[0206] In this embodiment, the first distance mapping parameter is the distance mapping parameter before parameter calibration. For explanations of S901-S903, please refer to the descriptions of S401-S403 above; the principle is the same, and will not be repeated here.

[0207] S904, the electronic device determines whether the face deflection angle is less than or equal to the angle threshold.

[0208] The determination of the facial deflection angle can be referenced from the above.Figure 8 The records are as described above, so I will not go into detail here.

[0209] S905, the electronic device determines whether parameter calibration is required.

[0210] Before determining whether the distance to the first mapped face is less than or equal to a distance threshold, the electronic device first determines whether parameter calibration is required. In some embodiments, parameter calibration requires turning on the TOF camera, which is a high-power device and not suitable for prolonged operation. Therefore, considering the power consumption of the electronic device, it can perform parameter calibration according to a preset calibration cycle, thereby saving device power consumption.

[0211] For example, electronic devices may undergo parameter calibration once a week or once a month. It should be noted that the preset calibration cycle can be set according to actual business needs and power consumption requirements of the equipment. The above-mentioned once a week or once a month does not constitute any limitation on the parameter calibration cycle.

[0212] Thus, when the electronic device determines that parameter calibration is needed based on the preset calibration cycle, it needs to further determine whether the parameter calibration requirements are met, i.e., the electronic device executes S909. However, when it is determined that parameter calibration is not needed at present based on the preset calibration cycle, the electronic device can directly determine whether to enable the defocus display mode based on the calculated first mapped face distance, i.e., the electronic device executes S906.

[0213] S906, the electronic device determines whether the distance between the first mapped face and the distance between the second mapped face are less than or equal to a distance threshold.

[0214] When parameter calibration is not required, the electronic device determines whether the distance to the first mapped face is less than or equal to a preset distance threshold. When parameter calibration is required, the electronic device recalculates the face distance based on the calibrated second distance mapping parameters, i.e., the second mapped face distance in this embodiment. Therefore, if parameter calibration is complete, the electronic device determines whether the distance to the second mapped face is less than or equal to the preset distance threshold.

[0215] The determination process for the distance between the first mapped face and the distance between the second mapped face can be referred to in S404 above, as the principle is the same, and will not be repeated here. Therefore, when the electronic device determines that the distance between the first mapped face and the distance between the second mapped face are less than or equal to the distance threshold, the electronic device executes S907. If the electronic device determines that the distance between the first mapped face and the distance between the second mapped face are greater than the distance threshold, the electronic device ends the execution of this process and waits for the next AO image to be acquired.

[0216] S907, the electronic device determines whether the current application scenario is the target effective scenario.

[0217] After the electronic device determines that the distance between the first mapped face and the distance between the second mapped face are less than or equal to a distance threshold, in order to avoid enabling defocus display in application scenarios that do not support defocus display, the electronic device further determines whether the current application scenario is the target effective scenario. If the current application scenario is the target effective scenario, the electronic device executes S908.

[0218] If the current application scenario is not the target effective scenario, the electronic device will end the execution of this process and wait for the next AO image to be acquired.

[0219] S908: When the electronic device activates the off-focus display mode, a prompt message is displayed.

[0220] For an explanation of S908, please refer to the description of S405 above. The principle is the same, so it will not be repeated here.

[0221] S909, the electronic device collects the first TOF face distance and calculates the difference between the first TOF face distance and the first mapped face distance.

[0222] S910, the electronic device determines whether the distance difference is greater than or equal to the error threshold.

[0223] When further determination is needed to determine whether parameter calibration is required because the preset calibration cycle is met, the electronic device turns on the TOF camera (i.e., the second front camera) to measure the actual distance between the electronic device and the user, and obtains the first TOF face distance (i.e., the first measured face distance).

[0224] Then, the electronic device calculates the distance difference between the first TOF face distance and the first mapped face distance. This distance difference represents the error between the face distance calculated based on the first distance mapping parameters and the actual face distance. That is, the larger the distance difference between the first mapped face distance and the first TOF face distance, the larger the error of the first distance mapping parameters. The smaller the distance difference between the first mapped face distance and the first TOF face distance, the smaller the error of the first distance mapping parameters.

[0225] Therefore, when the distance difference is less than the preset error threshold, the face distance (i.e., the first mapped face distance) obtained based on the current distance mapping parameters (i.e., the first distance mapping parameters) is accurate, and parameter calibration is not required for the time being. The electronic device executes S906.

[0226] When the distance difference is greater than or equal to the preset error threshold, it indicates that the face distance (i.e., the first mapped face distance) obtained based on the current distance mapping parameters (i.e., the first distance mapping parameters) is deviated. Therefore, it is necessary to calibrate and obtain accurate distance mapping parameters (i.e., the second distance mapping parameters). Then the electronic device executes S911.

[0227] In summary, the embodiments of this application determine whether to perform parameter calibration on the distance mapping parameters based on two conditions: whether the current time has reached the preset calibration period, and whether the calculated distance difference is less than the preset error threshold.

[0228] In this embodiment, the electronic device only begins calibrating the distance mapping parameters in the corresponding expression when the current time reaches a preset calibration period and the calculated distance difference is less than a preset error threshold. For ease of distinction, the distance mapping parameters before calibration are referred to as the first distance mapping parameters in this embodiment. The distance mapping parameters after calibration are referred to as the second distance mapping parameters in this embodiment.

[0229] The error threshold can be set based on empirical values ​​or actual business needs, and this application embodiment does not impose any limitations on it. For example, the error threshold can be determined based on the determined distance mapping parameters. Specifically, the error threshold can be determined by the difference between the mapped face distance calculated from the distance mapping parameters and the actual measured face distance.

[0230] S911, the electronic device acquires n frontal IR images, the IR images including the corresponding second TOF face distance.

[0231] After determining that the distance mapping parameters need to be calibrated, the electronic device starts the TOF camera to acquire n IR images. Since the IR images are images acquired by the TOF camera, each of the n IR images has a corresponding TOF face distance.

[0232] For ease of distinction, in the embodiments of this application, the TOF face distance corresponding to the acquired IR image is referred to as the second TOF face distance (i.e., the second measured face distance).

[0233] Where n is a positive integer greater than or equal to i, and i is determined based on the expression corresponding to the mapping relationship between human pupil distance and face distance. For example, taking the binary quartic relation in the above expression (1) as an example, at least 4 sets of (x, y) are needed to map to the constants a, b, c and d, so in this case n≥4. That is, i in the embodiments of this application can be understood as the highest power in the expression corresponding to the mapping relationship.

[0234] In some embodiments, the second TOF face distance may include the first TOF face distance, and correspondingly, the IR image corresponding to the second TOF face distance includes the IR image corresponding to the first TOF face distance. That is, the first TOF face distance acquired based on the error threshold to determine whether parameter calibration is needed, and the IR image corresponding to the first TOF face distance, can also be used together for subsequent parameter calibration.

[0235] For example, if n=4, after determining that parameter calibration is needed based on the distance difference obtained from the first TOF face distance, the electronic device can acquire 3 more IR images since one IR image has already been acquired.

[0236] It should also be noted that, in order to ensure the accuracy of face distance calculation, the IR image used for parameter calibration should also be an image of the face as frontal.

[0237] S912, the electronic device determines the second distance mapping parameter based on the human eye pupil distance in the IR image and the second TOF face distance.

[0238] After the electronic device obtains n IR images and the corresponding n second TOF face distances, it can refit the mapping relationship using the human eye pupil distance in the IR images and the corresponding second TOF face distances, that is, refit to obtain the above expression (1), thereby obtaining the second distance mapping parameters.

[0239] In other words, the electronic device uses the interpupillary distance in the IR image as x, and the second TOF face distance corresponding to the interpupillary distance in the IR image as y, thus generating n sets of (x, y). Then, the electronic device refits the mapping relationship based on these n sets of (x, y) to obtain a new set of distance mapping parameters, namely the second distance mapping parameters.

[0240] S913, the electronic device calculates the second mapped face distance based on the human eye pupil distance and the second distance mapping parameters in the AO image.

[0241] After the electronic device completes parameter calibration and obtains the second distance mapping parameters, it can use these parameters to recalculate the face distance by comparing it with the interpupillary distance in the AO image, thus obtaining the second mapped face distance. Then, it further determines whether to enable the defocus display mode by comparing the second mapped face distance with a distance threshold.

[0242] S914, the electronic device updates the error threshold based on the difference between the second mapped face distance and the second TOF face distance.

[0243] Meanwhile, after the electronic device completes parameter calibration, the distance mapping parameters are updated. Therefore, the error threshold determined based on the first distance mapping parameters before the update is no longer consistent with the currently determined second distance mapping parameters. Continuing to use this error threshold may affect the accuracy of the judgment. Therefore, the electronic device needs to update the error threshold accordingly.

[0244] Specifically, the updated error threshold can be determined based on the difference between the recalculated second mapped face distance and the second TOF face distance. For example, the difference between the second mapped face distance and the average of all second TOF face distances can be used as the new error threshold. Alternatively, the difference between the second mapped face distance and the median of all second TOF face distances can be used as the new error threshold. Or, the difference between the second mapped face distance and any single second TOF face distance can be used as the new error threshold.

[0245] Therefore, in this embodiment of the application, by periodically activating the TOF camera to update the distance mapping parameters, the accuracy of face distance calculation can be ensured, thereby improving the precision of activating the off-focus display mode.

[0246] Furthermore, since the embodiments of this application periodically turn on the TOF camera, the frequency of use of the TOF camera can be reduced compared to using a constantly turned-on TOF camera for ranging, thereby reducing the power consumption of the device.

[0247] In some embodiments, in low-light usage scenarios (such as dark scenes), such as when users use electronic devices with the lights off, the acquired AO images may be unclear, such as not including clear faces, which may affect the accuracy of face distance calculation. Therefore, in dark scenes, electronic devices can directly activate the TOF camera to measure face distance, thereby ensuring the accuracy of face distance detection.

[0248] For example, Figure 10 A flowchart of a display method is shown.

[0249] like Figure 10 As shown, before acquiring the AO image, the electronic device can first determine whether it is in a low-light scene by using the ambient light sensor to collect the light intensity L. If it is determined to be in a low-light scene, the electronic device directly activates the TOF camera and uses the face distance measured by the TOF camera as the first mapped face distance. Then, the electronic device determines whether to activate the off-focus display mode based on the first mapped face distance measured by the TOF camera.

[0250] Understandably, in low-light scenarios, since the first-mapped face distance is a direct measurement of the face distance, it doesn't involve distance mapping parameters, and therefore doesn't require calibration of those parameters. In other words, in low-light scenarios, electronic devices don't need to consider whether to calibrate the distance mapping parameters.

[0251] In bright lighting conditions (i.e., not dark lighting), electronic devices can capture AO (Area of ​​Focus) images and use these images to calculate the distance to the face, deciding whether to enable the off-focus display mode. Figure 4 , Figure 8 or Figure 9 The process is shown below, and will not be described in detail here.

[0252] In some embodiments, the illumination intensity L collected by the ambient light sensor is shown in Table 1 below.

[0253] Table 1

[0254] Dark light scene Medium light scene High light scene L < 100 lx 100 lx < L < 500 lx 500 lx < L < 1000 lx

[0255] As shown in Table 1 above, when the ambient light intensity L collected by the ambient light sensor is less than or equal to 100 lx, the electronic device can determine that it is currently in a low-light scene. It is understood that the range of ambient light intensity corresponding to each scene in Table 1 is an example of an embodiment of this application, and Table 1 does not constitute any limitation on the range of ambient light intensity corresponding to each scene. The specific ambient light intensity L corresponding to a low-light scene can be set according to actual needs, and can be larger or smaller than the 100 shown in Table 1.

[0256] In some embodiments, considering the user's actual eye protection needs, the electronic device can provide a corresponding off-focus display switch in the settings. Furthermore, when the off-focus display switch is turned on, it indicates that the user has an eye protection need. Therefore, the electronic device can activate the AO mode of the first front-facing camera, thereby acquiring AO images to detect face distance, and further determine whether to switch to off-focus display mode based on the detected face distance.

[0257] That is, when the defocus display switch is turned on, the electronic device can perform... Figure 4 , Figure 8 , Figure 9 or Figure 10 The process shown is to activate the off-focus display mode. When the off-focus display switch is off, it indicates that the user has no need for eye protection, therefore the electronic device does not need to perform this step. Figure 4 , Figure 8 , Figure 9 or Figure 10 The process is shown below.

[0258] For example, taking a mobile phone as an example, Figure 11 A schematic diagram of the interface of a defocus display switch is shown.

[0259] The phone can respond to user clicks and display the eye protection settings interface 1101. For example... Figure 11 As shown, the eye protection settings interface 1101 includes an off-focus display control option 1102, which includes a control switch 1103 corresponding to the off-focus display mode. The mobile phone can respond to the user clicking the switch 1103, allowing the electronic device to activate the off-focus display mode when the conditions for activating the off-focus display mode are met.

[0260] In some embodiments, the electronic device can also continuously collect relevant data on the user's use of the electronic device, and then push personalized suggestions to the user based on the collected data, such as pushing personalized eye protection tips to the user.

[0261] For example, Figure 12 A block diagram illustrating the principle of a display method is shown.

[0262] like Figure 12 As shown, in addition to short-term defocus display mode based on calculating face distance from AO images, electronic devices can also collect inertial measurement unit (IMU) data, light intensity, and application scenarios when users use electronic devices.

[0263] The IMU includes an accelerometer and / or a gyroscope sensor, and therefore the IMU data can include acceleration data collected by the accelerometer and / or gyroscope data collected by the gyroscope. Illumination intensity can be obtained from an ambient light sensor. Therefore, the electronic device can determine its usage scenario using the IMU data and the illumination intensity.

[0264] The application scenario for an electronic device can be determined by analyzing the focused window when the user uses the device. For example, the focused window can be used to determine whether the application scenario is a game, video, or reading application.

[0265] Then, as Figure 12 As shown, electronic devices can determine the user's activity status under different lighting conditions based on the determined usage and application scenarios, and then combine the calculated mapped face distance to statistically analyze the user's poor eye behavior.

[0266] Examples of poor eye habits include users playing games while lying on their side in bright light, watching videos at close range in bright light, and reading novels in a swaying vehicle.

[0267] Finally, as Figure 12As shown, electronic devices combine statistically analyzed users' poor eye-use behaviors with their basic personal information, such as age, gender, and occupation, to predict the user's risk of myopia. The user's basic personal information is obtained with the user's authorization, for example, through a questionnaire. That is, the electronic device can send a questionnaire to the user and then compile the user's basic personal information based on the questionnaire responses.

[0268] Electronic devices can predict the risk level of a user's myopia using a neural network model. For example, an ophthalmologist can be invited to score the risk level of myopia based on poor eye habits and personal information. This scored data can then be used to train the neural network model. When it's necessary to predict the user's myopia risk level, the electronic device inputs the user's poor eye habits and personal information into the trained neural network model, obtaining the score output by the model. The electronic device determines the user's myopia risk level based on the score. The relationship between the score and the risk level can be set according to actual needs, and this embodiment does not impose any limitations on this. For example, the score can be on a 100-point scale, with higher scores indicating a higher risk of myopia.

[0269] Subsequently, such as Figure 12 As shown, the electronic device can push personalized eye care tips to the user based on the predicted risk level of myopia. For example, personalized eye care tips may include messages such as "Long-term poor eye habits, pay attention to preventing myopia." It is understood that the specific personalized eye care tips can be set according to actual business needs, and this application embodiment does not impose any limitations on this.

[0270] Another embodiment of this application provides an electronic device, including: a display screen, a front-facing camera module, one or more processors, and a memory. The display screen, the front-facing camera module, and the memory are respectively coupled to the processor;

[0271] The front-facing camera module is used to capture images or measure the distance to a face; the memory stores one or more computer program codes, which include computer instructions; when the processor executes the computer instructions, the electronic device implements the display method described in any of the above embodiments.

[0272] In some embodiments, the front-facing camera module includes a first front-facing camera and a second front-facing camera; the first front-facing camera is configured to include an AO mode for acquiring AO images; the second front-facing camera includes a TOF camera for measuring face distance.

[0273] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in an electronic device, causes the electronic device to implement the display method described in any of the above embodiments.

[0274] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0275] This application also provides a chip system, such as... Figure 13 As shown, the chip system 1300 includes at least one processor 1301 and at least one interface circuit 1302. The processor 1301 and the interface circuit 1302 are interconnected via lines. For example, the interface circuit 1302 can be used to receive signals from other devices (e.g., a computer's memory). As another example, the interface circuit 1302 can be used to send signals to other devices (e.g., the processor 1301).

[0276] For example, interface circuit 1302 can read instructions stored in memory and send those instructions to processor 1301. When the instructions are executed by processor 1301, the computer can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0277] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0278] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0279] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0280] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0281] If the function of the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0282] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display method characterized by comprising: Be applied to electronic equipment The electronic equipment includes a front camera module; the front camera module includes a first front camera; the method includes: Collecting a first image, the first image being collected by the first front camera; Determining a first mapped face distance according to a mapping relationship between a human eye pupil distance and a human face distance in the first image; the mapping relationship being obtained by fitting a measured human face distance and a human eye pupil distance; When the first mapped face distance is less than or equal to a preset distance threshold, starting a defocus display mode and displaying prompt information; wherein the prompt information is used to indicate that the display mode of the electronic equipment is the defocus display mode.

2. The method of claim 1, wherein, The first front camera includes an always-on AO mode, and the first image includes an AO image collected by the first front camera in the AO mode.

3. The method according to claim 1 or 2, characterized in that, When the first mapped face distance is less than or equal to a preset distance threshold, starting a defocus display mode and displaying prompt information, including: If the calibration condition is not met, it is determined that there is no need to calibrate the distance mapping parameter, and when the first mapped face distance is less than or equal to a preset distance threshold, the defocus display mode is started and the prompt information is displayed; wherein the distance mapping parameter is a constant in the expression corresponding to the mapping relationship.

4. The method of claim 3, wherein, The front camera module further includes a second front camera, and the second front camera is used to measure a distance; the method further includes: When the current time does not reach a preset calibration period or the distance difference value is less than a preset error threshold, it is determined that the calibration condition is not met; When the current time reaches the preset calibration period and the distance difference value is greater than or equal to the preset error threshold, it is determined that the calibration condition is met; Wherein the distance difference value is the difference between the first measured face distance and the first mapped face distance; the first measured face distance is measured by the second front camera.

5. The method according to claim 3 or 4, characterized in that, The method further includes: When the calibration condition is met, n infrared IR images are obtained; the n IR images are collected by the second front camera, the n IR images include corresponding second measured face distances, and the face deflection angle of the face in the n IR images is less than or equal to a preset angle threshold; The distance mapping parameter in the expression corresponding to the mapping relationship is calibrated by using the human eye pupil distance and the second measured face distance in the n IR images; n is a positive integer greater than or equal to i, and i is the highest order in the expression corresponding to the mapping relationship; A second mapped face distance is determined according to the human eye pupil distance in the first image and the calibrated expression; When the second mapped face distance is less than or equal to a preset distance threshold, the defocus display mode is started and the prompt information is displayed.

6. The method of claim 5, wherein, The method further includes: The error threshold is updated, and the updated error threshold is determined based on the difference between the second mapped face distance and the second measured face distance.

7. The method according to any one of claims 1 to 6, characterized in that, The first mapped face distance is determined according to the mapping relationship between the human eye pupil distance and the human face distance in the first image, including: performing human eye gaze detection on the first image to determine whether a face in the first image gazes at a display screen of the electronic device; when the face in the first image gazes at the display screen of the electronic device, determining a first mapped face distance according to a mapping relationship between a human eye pupil distance and a face distance in the first image.

8. The method according to any one of claims 1-7, characterized in that, The determining the first mapped face distance according to the mapping relationship between the human eye pupil distance and the face distance in the first image comprises: detecting a face yaw angle of the face in the first image; when the face yaw angle is less than or equal to a preset angle threshold, determining the first mapped face distance according to the mapping relationship between the human eye pupil distance and the face distance in the first image; The method further comprises: when the face yaw angle is greater than the preset angle threshold, calculating the first mapped face distance according to a face size in the first image and a device resolution without determining whether the calibration condition is met.

9. The method according to any one of claims 1-8, characterized in that, The front camera module further comprises a second front camera, and the second front camera is configured to measure a distance; The acquiring the first image comprises: acquiring an illumination intensity, and determining whether a use scenario of the electronic device is a dark light scenario according to the illumination intensity; when the use scenario of the electronic device is not the dark light scenario and a duration of a display screen of the electronic device being in a bright screen state is greater than or equal to a preset duration, acquiring the first image at a preset time interval; a time difference between acquisition times of adjacent two first images is equal to the time interval; The method further comprises: when the use scenario of the electronic device is the dark light scenario, starting the second front camera to measure the first mapped face distance without determining whether the calibration condition is met.

10. The method according to any one of claims 1-9, characterized in that, The method further comprises: when a current application scenario is a preset target effective scenario, starting the defocus display mode and displaying the prompt information.

11. The method according to any one of claims 3-10, characterized in that, An expression corresponding to the mapping relationship is: y = a * 1 / x3 + b * 1 / x2 + c * 1 / x + d wherein a, b, c and d are distance mapping parameters, x is a human eye pupil distance, and y is a mapped face distance, and the mapped face distance comprises the first mapped face distance or the second mapped face distance.

12. The method according to any one of claims 1-11, characterized in that, The method further comprises: statistically collecting an unhealthy eye use behavior and personal basic information of a user; the personal basic information is obtained after authorization of the user; predicting a risk degree of myopia of the user according to the unhealthy eye use behavior and the personal basic information, and pushing personalized eye protection prompt information based on the risk degree; wherein the unhealthy eye use behavior is determined according to a use scenario, an application scenario and a target face mapping distance of the electronic device; the use scenario is determined according to inertial measurement unit data and an illumination intensity; and the target face mapping distance comprises the first mapped face distance or the second mapped face distance.

13. An electronic device, comprising: comprise: a display screen, a front camera module, one or more processors and a memory; the front camera module, the display screen and the memory are respectively coupled with the processor; the front camera module is configured to acquire an image or measure a face distance; The memory stores one or more computer program codes including computer instructions; when the processor executes the computer instructions, the electronic device performs the display method as claimed in any one of claims 1-12.

14. The electronic device of claim 13, wherein, The front camera module includes a first front camera and a second front camera; the first front camera includes an AO mode for collecting an AO image; The second front camera includes a time-of-flight (TOF) camera for measuring a face distance.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device performs the display method as claimed in any one of claims 1-12.

16. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor in the electronic device, and the electronic device performs the display method as claimed in any one of claims 1-12.

Images