A device control method and apparatus, an electronic device, and a storage medium

Abstract

The embodiment of the application discloses a device control method and device, electronic equipment and storage medium. The device control method is applied to a terminal, the terminal is provided with a display screen, and the method comprises the following steps: acquiring a first eye parameter of a user's eye, the first eye parameter being used for representing the position of the iris of the user's eye; determining viewing information between the user's eye and the display screen according to the first eye parameter, wherein the viewing information comprises a first viewing distance, a viewing angle and a viewing time; determining whether the user's eye is gazing at the display screen according to the first eye parameter to obtain a judgment result; and determining the working mode of the display screen according to the viewing information when the judgment result represents that the user is gazing at the display screen. When the display screen is controlled, the control can be performed only when the user is gazing at the display screen, and the working mode of the display screen is determined according to the viewing information, so that the user experience is improved, and myopia prevention and control are facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display control, in particular to a device control method and device, an electronic device and a storage medium. BACKGROUND

[0002] Long time close plane fixation is the first major cause of myopia formation, and plane fixation means that the eyes of a person watch two-dimensional plane objects at a relatively fixed viewing distance.

[0003] At present, in order to avoid the user watching the display screen for a long time, the display screen is controlled by the watching time of the page content of the display screen, such as the fixed time of video playing, the playing time of the game or the lighting time of the screen. However, the display screen is controlled by the watching time obtained from the above page content, which is easy to misjudge, such as the user is not necessarily watching the screen when the display screen is displaying, and the control of the display screen at this time will affect the user experience. SUMMARY

[0004] The embodiments of the present application provide a device control method and device, an electronic device and a storage medium, which can control the device according to whether the user's eyes watch the display screen, improve the user experience, and help prevent and control myopia.

[0005] In a first aspect, the embodiments of the present application provide a device control method applied in a terminal, wherein the terminal is provided with a display screen, and the method comprises: acquiring a first eye parameter of a user's eye, the first eye parameter being used to represent the position of the iris of the user's eye; determining viewing information between the user's eye and the display screen according to the first eye parameter, wherein the viewing information comprises a first viewing distance, a viewing angle and a viewing time; judging whether the user's eye is gazing at the display screen according to the first eye parameter to obtain a judgment result; and determining a working mode of the display screen according to the viewing information when the judgment result represents that the user is gazing at the display screen.

[0006] In some embodiments, the terminal is provided with a front camera; the acquiring of the first eye parameter of the user's eye comprises: acquiring a face image of the user, the face image being obtained by the front camera; obtaining first eye socket position information and first iris position information from the face image, wherein the first eye socket position information is used to represent the position of the eye socket of the user, and the first iris position information is used to represent the position of the iris of the user's eye; and obtaining the first eye parameter according to the first eye socket position information and the first iris position information.

[0007] In some embodiments, the viewing information includes a first viewing distance; the determining whether the user's eye gazes at the display screen according to the first eye parameter to obtain a determination result includes: controlling the display screen to display; acquiring a first light shadow of the display screen on the user's eye; determining whether the user's eye gazes at the display screen according to the first viewing distance, the first eye parameter and the first light shadow to obtain the determination result.

[0008] In some embodiments, the determining whether the user's eye gazes at the display screen according to the first viewing distance, the first eye parameter and the first light shadow to obtain a determination result includes: matching a coordinate mapping relationship table corresponding to the distance according to the first viewing distance; calculating a light shadow coordinate of the first light shadow; looking up the coordinate mapping relationship table according to the first eye parameter and the light shadow coordinate to obtain a lookup result; and obtaining the determination result according to whether there is a gaze coordinate of the user's eye gazing at the display screen in the lookup result.

[0009] In some embodiments, the method further includes: displaying a sample cursor on the display screen; acquiring a second eye parameter and a second light shadow when the user's eye gazes at the sample cursor, wherein the second eye parameter is used to represent the position of the iris of the user's eye, and the second light shadow is a light shadow of the display screen or the sample cursor on the user's eye; acquiring a second viewing distance from the display screen when the user's eye gazes at the sample cursor; and recording a corresponding relationship between the second eye parameter and the second light shadow at the second viewing distance to establish a coordinate mapping relationship table of the sample cursor.

[0010] In some embodiments, the displaying a sample cursor on the display screen includes: uniformly dividing the display screen into a plurality of display regions; and displaying a sample cursor on each of the display regions; and the recording a corresponding relationship between the second eye parameter and the second light shadow at the second viewing distance to establish a coordinate mapping relationship table of the sample cursor includes: displaying the sample cursor on each of the display regions in sequence; and recording a corresponding relationship between a plurality of the second eye parameters and a plurality of the second light shadows corresponding to the user's viewing of the sample cursor on each of the display regions at the second viewing distance when the viewing position of the user is unchanged to establish the coordinate mapping relationship table of the sample cursor.

[0011] In some embodiments, the acquiring the second eye parameter and the second light shadow of the user's eyes when the user's eyes gaze at the sample cursor includes: when the second viewing distance has multiple, acquiring the second eye parameter and the second light shadow of the user's eyes when the user's eyes gaze at the sample cursor at multiple different second viewing distances; or, acquiring the second eye parameter and the second light shadow of the user's eyes when the user's eyes gaze at the sample cursor at different viewing angles under the second viewing distance; wherein the viewing angle is an angle formed by a plane where the user's face is located and the display screen.

[0012] In some embodiments, the first eye parameter is obtained according to a face image of the user; and the determining the viewing information between the user's eyes and the display screen according to the first eye parameter includes: determining left eye pupil position information and right eye pupil position information of the user's eyes according to the first eye parameter, calculating a picture interpupillary distance of the user in the face image according to the left eye pupil position information and the right eye pupil position information, calculating a first viewing distance from the display screen of the user's eyes according to the picture interpupillary distance; or, determining left eye pupil position information and right eye pupil position information of the user's eyes according to the first eye parameter, calculating a picture face distance of the user in the face image according to the left eye pupil position information and the right eye pupil position information, and obtaining a picture face ratio according to the picture face distance; obtaining a preset standard face ratio; obtaining a viewing angle between the user and the display screen according to the picture face ratio and the standard face ratio; or, calculating a sum of time of the user's eyes gazing at the display screen according to the first eye parameter to obtain a viewing time of the user's eyes gazing at the display screen.

[0013] In some embodiments, the calculating the first viewing distance from the display screen of the user's eyes according to the picture interpupillary distance includes: obtaining a preset standard interpupillary distance; obtaining a focal length of the face image captured by the front camera, and obtaining an initial distance from the face image to an imaging point according to the focal length; obtaining a first ratio according to the picture interpupillary distance and the standard interpupillary distance, and obtaining the first viewing distance from the display screen of the user's eyes according to the first ratio and the initial distance; or, obtaining a preset distance query table; obtaining the first viewing distance from the display screen of the user's eyes from the distance query table according to the picture interpupillary distance; or, obtaining a reference distance, a reference object size, and a picture size of a reference object corresponding to the face image captured by the front camera; obtaining a preset standard interpupillary distance; and obtaining the first viewing distance from the display screen of the user's eyes according to the reference distance, the reference object size, the picture size, the picture interpupillary distance, and the standard interpupillary distance.

[0014] In some embodiments, the determining the working mode of the display screen according to the viewing information comprises: when the information size represented by the viewing information is within a preset first threshold range, controlling the display screen to enter a first working mode, the first working mode comprising at least one of controlling the display screen to display prompt content and adjusting the display brightness of the display screen; and when the information size represented by the viewing information is within a preset second threshold range, controlling the display screen to enter a second working mode, the second working mode comprising turning off the display screen.

[0015] In some embodiments, when the information size represented by the viewing information is within a preset first threshold range, the method further comprises: controlling the terminal to at least one of emit a first sound prompt and emit a first vibration prompt; and when the information size represented by the viewing information is within a preset second threshold range, the method further comprises: controlling the terminal to at least one of emit a second sound prompt and emit a second vibration prompt.

[0016] In some embodiments, the viewing information comprises a viewing time; and the determining the working mode of the display screen according to the viewing information comprises: obtaining age information of the user; and when the age information represents that the user is a child, controlling the display screen to enter a third working mode according to the viewing time, the third working mode comprising controlling the display screen to be turned on or turned off according to the viewing time interval.

[0017] In some embodiments, the front camera is an under-screen camera, and the under-screen camera is arranged at a center position of the display screen.

[0018] In a second aspect, an embodiment of the present application further provides a device control apparatus, comprising: a first module configured to obtain a first eye parameter of a user's eye, the first eye parameter being used to represent a position of an iris of the user's eye; a second module configured to determine viewing information between the user's eye and the display screen according to the first eye parameter, wherein the viewing information comprises a first viewing distance, a viewing angle and a viewing time; a third module configured to determine whether the user's eye gazes at the display screen according to the first eye parameter, and obtain a determination result; and a fourth module configured to determine a working mode of the display screen according to the viewing information when the determination result represents that the user gazes at the display screen.

[0019] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the device control method according to the first aspect of the present application.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, the storage medium storing a program, and the program is executed by a processor to implement the device control method according to the first aspect of the present application.

[0021] The embodiments of the present application have at least the following beneficial effects: The embodiments of the present application provide a device control method, device, electronic device and storage medium. The device control method is applied to a terminal, and the terminal is provided with a display screen. By executing the device control method, the position of the iris of the eyes of a user can be obtained, and a first eye parameter can be obtained. The first viewing distance, viewing angle and viewing time between the user and the display screen can be determined according to the position of the iris of the user. Whether the user gazes at the display screen can be determined according to the position of the iris of the user. When the display screen is controlled, the control can be performed only when the user gazes at the display screen. The working mode of the display screen can be determined according to the viewing information, so as to improve the user experience and facilitate myopia prevention and control. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a schematic diagram of the internal structure of a human eye provided by an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a terminal provided by an embodiment of the present application;

[0024] Figure 3 is a flowchart of a device control method provided by an embodiment of the present application;

[0025] Figure 4 is a flowchart of a device control method provided by another embodiment of the present application;

[0026] Figure 5 is a flowchart of a device control method provided by another embodiment of the present application;

[0027] Figure 6a is a schematic diagram of light and shadow in an actual image provided by an embodiment of the present application;

[0028] Figure 6b is a schematic diagram of the positions of light and shadow when looking in different directions provided by an embodiment of the present application;

[0029] Figure 7 is a flowchart of a device control method provided by another embodiment of the present application;

[0030] Figure 8 is a flowchart of a device control method provided by another embodiment of the present application;

[0031] Figure 9 is an application scenario diagram of a device control method provided by an embodiment of the present application;

[0032] Figure 10 is a flowchart of a device control method according to another embodiment of the present application;

[0033] Figure 11 is a flowchart of a device control method according to another embodiment of the present application;

[0034] Figure 12 is a flowchart of a device control method according to another embodiment of the present application;

[0035] Figure 13 is a flowchart of a device control method according to another embodiment of the present application;

[0036] Figure 14a is a schematic diagram of a face image according to an embodiment of the present application;

[0037] Figure 14b is a schematic diagram of a face image according to another embodiment of the present application;

[0038] Figure 15 is a flowchart of a device control method according to another embodiment of the present application;

[0039] Figure 16 is a schematic diagram of relative lens (imaging point) imaging according to an embodiment of the present application;

[0040] Figure 17 is a schematic diagram of calculating a first viewing distance by a triangle principle according to an embodiment of the present application;

[0041] Figure 18 is a schematic diagram of obtaining a first viewing distance according to a reference system according to an embodiment of the present application;

[0042] Figure 19 is a schematic diagram of obtaining a first viewing distance according to a reference system according to another embodiment of the present application;

[0043] Figure 20 is a flowchart of a device control method according to another embodiment of the present application;

[0044] Figure 21 is a flowchart of a device control method according to another embodiment of the present application;

[0045] Figure 22 is a schematic diagram of a device control apparatus according to an embodiment of the present application;

[0046] Figure 23 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the present application clearer, the following further describes the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] It should be understood that, in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, more than, and the like are not included in the number, and above, below, and the like are included in the number. If it is described as "first", "second", and the like, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features.

[0049] Long time close plane fixation is the first major cause of myopia formation, and plane fixation means that the person's eyes, the eyes, watch two-dimensional plane things, such as books, newspapers, mobile phones, televisions, and non-three-dimensional things, for a long time, plane fixation, will make our eyes ciliary muscle unable to adjust the lens for a long time, and over time, the contraction and stretching function of the ciliary muscle of the eye and the adjusting ability of the lens will be weakened.

[0050] It can be understood that the human eye is a spherical lens, and the lens of the eye is usually spherical, filled with transparent gelatinous material, with a focusing lens, and an iris that can control the amount of light entering the eye. Figure 1 The human eye has three layers of outerwear, composed of three transparent structures, the outermost layer is composed of the cornea, the middle layer is composed of the ciliary body and the iris, and the outerwear is the aqueous humor, the vitreous body, and the flexible lens, the water-like liquid is a clear liquid, contained in two areas: the area exposed by the lens, the anterior chamber between the cornea and the iris, and the lens is suspended by the ciliary zonular suspensory ligament (zona ciliaris) composed of transparent fine fibers. The vitreous body, the posterior chamber of the eye is a clear gel larger than the anterior chamber of the eye, located behind the lens and the rest of the area, wrapped around the zonular and the lens.

[0051] When the lens of our eyes cannot normally adjust the viewing distance, blurred vision will be presented, which is called myopia. Currently, in order to avoid long time watching of the display screen, the display screen is controlled according to the watching time of the page content of the display screen, such as according to the fixed time of video playing, the playing time of game or the lighting time of the screen. However, the applicant finds that the control of the display screen according to the watching time of the page content is easy to misjudge, and the user is not necessarily watching the screen when the display screen is displaying, for example, when the user is watching e-book or playing game through the terminal, the user may put the terminal on the desktop and leave to deal with other things, and then come back to continue watching the display screen after finishing dealing with the other things. At this time, if the control method in the related art is used, the accumulated time is not the real watching time of the user, and the control of the display screen will affect the user experience.

[0052] Based on this, the embodiment of the application provides a device control method, device, electronic device and storage medium. The device control method can be applied to a terminal, and the terminal is provided with a display screen. By executing the device control method, the device can be controlled according to the condition that the user's eyes watch the display screen, so as to improve the user experience and facilitate myopia prevention and control.

[0053] In the embodiment of the application, the position of the user's iris can be acquired, the first viewing distance, viewing angle and viewing time between the user and the display screen can be determined according to the position of the user's iris, and it can be judged whether the user is staring at the display screen according to the position of the user's iris. When the display screen is controlled, the control can be executed only when the user is staring at the display screen, and the working mode of the display screen can be determined according to the viewing information, so as to improve the user experience and facilitate myopia prevention and control.

[0054] The terminal in the embodiment of the application can be a mobile terminal device or a non-mobile terminal device. The mobile terminal device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal device, a wearable device, a super mobile personal computer, a netbook, a personal digital assistant, etc. The non-mobile terminal device can be a personal computer, a television, a cashier machine or a self-service machine, etc. The application does not make specific limitation.

[0055] The terminal can include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a mobile communication module, a wireless communication module, an audio module, a speaker, a receiver, a microphone, a headset jack, a sensor module, a key, a motor, an indicator, a front camera, a rear camera, a display screen, and a subscriber identification module (SIM) card interface, etc. The terminal can implement a photographing function through the front camera, the rear camera, a video codec, a GPU, a display screen, and an application processor, etc.

[0056] The front camera or the rear camera is used to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to an ISP (image signal processor) to convert into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, etc. format image signal. In some embodiments, the terminal can include one or N front cameras, where N is a positive integer greater than 1.

[0057] The terminal implements a display function through a GPU, a display screen, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor can include one or more GPUs that execute program instructions to generate or change display information.

[0058] The display screen is used to display images, videos, etc. The display screen comprises a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0059] In an embodiment, the display screen can be a naked-eye 3D display screen. Through the naked-eye 3D display screen, multimedia data can be processed, split into two parts, for example, a 2D video is cropped into two parts, and the light refraction directions of the two parts are changed. After the user's eyes watch, a 3D picture is formed, the formed 3D picture has a negative parallax, can be displayed between the user and the display screen, and naked-eye 3D viewing effect is achieved. Alternatively, the display screen is a 2D display screen, and the terminal can externally attach a 3D grating film to refract the outgoing light of the 2D display screen. After the user watches the display screen through the 3D grating film, a 3D display effect is observed.

[0060] For example, Figure 2 In an embodiment, the terminal is taken as a mobile phone as an example. For example, the front panel 10 of the mobile phone is provided with a front camera 11, which can acquire image information. The front panel 10 is also provided with a display screen 12, which can display a picture. It can be understood that when the terminal is a 3D visual training terminal or a mobile phone capable of displaying a 3D picture, the display screen 12 can display a 3D picture.

[0061] It should be noted that the front camera 11 in the embodiment of the present application can be arranged on the same plane as the display screen 12, and the position of the front camera 11 is fixed. In an embodiment, the front camera 11 can be perpendicular to the display screen 12 or not perpendicular to the display screen 12. The front camera 11 can be located inside the display screen 12 or located at the periphery of the display screen 12. Figure 2 In addition, the front camera 11 can have a vertical distance from the display screen 12, so that the front camera 11 and the display screen 12 are not on the same plane. The terminal can perform parameter calibration according to the different settings of the front camera 11 to achieve the device control method in the embodiment of the present application.

[0062] The device control method, device, electronic device, and storage medium are described below. First, the device control method is described.

[0063] With reference to Figure 3 The device control method provided in the embodiments of the present application can be applied in a terminal. The device control method can include, but is not limited to, steps S101-S104.

[0064] In step S101, a first eye parameter of a user's eye is obtained. The first eye parameter is used to represent the position of the iris of the user's eye.

[0065] In step S102, viewing information between the user's eye and a display screen is determined according to the first eye parameter. The viewing information includes a first viewing distance, a viewing angle, and a viewing time.

[0066] In step S103, it is determined whether the user's eye gazes at the display screen according to the first eye parameter, and a determination result is obtained.

[0067] In step S104, when the determination result indicates that the user gazes at the display screen, the working mode of the display screen is determined according to the viewing information.

[0068] It should be noted that the device control method provided in the embodiments of the present application can be applied in a terminal or the like. The terminal is provided with a display screen. By executing the device control method provided in the embodiments of the present application, the device control can be performed according to the user's eye viewing the display screen, and the working mode of the display screen can be determined according to the user's viewing, which is conducive to myopia prevention and control. Specifically, the terminal can obtain the position of the iris of the user's eye to obtain the first eye parameter, determine the viewing information between the user's eye and the display screen according to the first eye parameter, and determine whether the user's eye gazes at the display screen according to the first eye parameter to obtain a determination result. After it is determined that the user is viewing the display screen, the working mode of the display screen can be determined according to the user's gazing at the display screen, for example, the working mode of the display screen can be determined according to the actual viewing time of the user viewing the display screen, or the working mode of the display screen can be determined according to the distance between the user and the display screen when the user is viewing the display screen, or the working mode of the display screen can be determined according to the viewing angle of the user viewing the display screen. It can be understood that the device control method provided in the embodiments of the present application is performed after the position of the user's iris is obtained and it is determined that the user is viewing the display screen according to the first eye parameter. The display of the display screen can be controlled according to the actual viewing of the user, which is conducive to myopia prevention and control.

[0069] It can be understood that when the user views something at different angles, the eyes also rotate, and the position of the iris in the human eye moves with the human eye viewing different directions, and thus the embodiment of the present application obtains the position of the iris of the human eye to obtain the first eye parameter to judge the viewing condition of the human eye.

[0070] With reference to Figure 4 In an embodiment, the terminal is provided with a front camera, and the step S101 can further include, but is not limited to, steps S201 to S204.

[0071] In step S201, a face image of the user is obtained, which is obtained by the front camera.

[0072] In step S202, first eye socket position information and first iris position information are obtained by analyzing the face image, wherein the first eye socket position information is used to represent the position of the eye socket of the user, and the first iris position information is used to represent the position of the iris of the eye of the user.

[0073] In step S203, the first eye parameter is obtained according to the first eye socket position information and the first iris position information.

[0074] It should be noted that in the embodiment of the present application, the front camera provided on the terminal is used to obtain the image, and finally the first eye parameter required in the embodiment of the present application is obtained. Specifically, in the embodiment of the present application, the face image of the user is first obtained by the front camera, and the face image is analyzed to obtain the first eye socket position information and the first iris position information of the user. The first eye socket position information represents the position of the eye socket of the user, and the eye socket refers to the frame formed by the edge of the eyelid, which is a bone cavity similar to a four-sided cone that accommodates the eyeball and other tissues, and there is one on the left and one on the right, which are symmetrical to each other, and the adult orbit depth is about 4 to 5 cm. The first iris position information represents the position of the iris in the eye of the user. The position of the iris in the eye of the user can be obtained by the position relationship between the first iris position information and the first eye socket position information. It can be understood that in the embodiment of the present application, the first eye socket position information and the first iris position information of both eyes of the user can be obtained, or only the first eye socket position information and the first iris position information of one eye can be obtained, and no specific limitation is made to this. In addition, the center of the iris is the center of the pupil, and the position of the pupil of the user can also be obtained according to the first iris position information.

[0075] It should be noted that the device control method in the embodiment of the present application can obtain the face image of the user through the front camera provided on the terminal, so that it is not necessary to additionally provide a sensor or a camera, and the present application can realize image acquisition through the front camera provided on the terminal, and then the position of the iris of the eyes of the user is obtained according to the recognized face image. The device control method in the embodiment of the present application can be realized only by using a terminal provided with a front camera, and the cost is low, and the device control method is applicable to the existing smart phones, tablet computers, 3D training terminals and other devices.

[0076] It should be noted that the face image in the embodiment of the present application can be directly obtained by the front camera recognizing the face of the user. In an embodiment, the face image is obtained by cropping the image obtained by the front camera. For example, the terminal obtains an image through the front camera, and the image contains the face of the user and may also contain some other sundries, which may interfere with iris recognition. Therefore, the embodiment of the present application crops the face region of the user to obtain the face image through image cropping, so as to improve the accuracy of recognition.

[0077] Reference Figure 5 In an embodiment, the viewing information includes a first viewing distance, and the step S103 can further include but is not limited to steps S301 to S303.

[0078] In step S301, the display screen is controlled to display.

[0079] In step S302, the first light shadow of the display screen on the eyes of the user is obtained.

[0080] In step S303, whether the eyes of the user gaze at the display screen is judged according to the first viewing distance, the first eye parameter and the first light shadow, and a judgment result is obtained.

[0081] It should be noted that the device control method in the embodiment of the present application controls the display screen to display first, and the display screen displays light after being turned on. Then, the first eye parameter and the first light shadow of the eyes of the user are obtained. The first eye parameter is parameter information for representing the position of the iris of the eyes of the user. The position of the iris of the user in the eyes can be known through the first eye parameter. The first light shadow is the light shadow of the display screen on the eyes of the user. After the display screen displays light, the light shadow is formed in the optical mirror of the human eye based on the characteristics of the human eye. Then, the first viewing distance from the eyes of the user to the display screen is obtained. Whether the eyes of the user gaze at the display screen is judged according to the first viewing distance, the first eye parameter and the first light shadow, and a judgment result is obtained. Therefore, whether the display screen is being watched can be judged according to the gazing condition of the eyes of the user in the embodiment of the present application.

[0082] Specifically, since the human eye is a spherical structure, the light rays emitted by the display screen will be refracted when passing through the cornea of the human eye, and the refracted light rays will form a light shadow on the optical mirror of the human eye. Figure 6aBased on the physical property of the human eye as an optical mirror, when a human eye views a luminous display screen, the screen's reflection is projected into the eye. The position of this reflection in the eye varies depending on the angle and position of the eye facing the screen. This is manifested in how the reflection appears different depending on the viewing angle and position. Figure 6b The relative position between light and shadow and the iris of the eye will also be different. Therefore, in this embodiment of the invention, after obtaining the first light and shadow in the human eye, the first viewing distance and the first eye parameters can be combined to determine whether the user's eyes are focused on the display screen. It can be understood that... Figure 6a and Figure 6b The light and shadow in the above embodiment can be the first light and shadow.

[0083] It should be noted that in this embodiment of the invention, facial image recognition is achieved through image processing to obtain the required first orbital position information, first iris position information, and first light and shadow. Specifically, this embodiment of the invention performs face eye region detection on the facial image based on a preset detector to obtain the first orbital position information, converts the facial image into a grayscale image, and performs binarization processing on the grayscale image to obtain a first preprocessed image. The first light and shadow is obtained based on the rectangular or circular noise image in the first preprocessed image, wherein the rectangular or circular noise image is the light and shadow of the display screen in the human eye after image processing. Further, the first preprocessed image is subjected to erosion and dilation processing, and the noise in the image is removed to obtain a second preprocessed image. Since the iris of the human eye is circular, the position of the circular region representing the user's iris in the second preprocessed image is extracted using circular structural elements to obtain the first iris position information of the user's eye.

[0084] In an embodiment, the embodiment of the present application realizes processing of a face image based on opencv, and uses a cascade classifier as a detector to perform face recognition, the cascade classifier is realized based on Local Binary Pattern (LBP) features and Haar-like features (HAAR), and the classifier data trained based on the LBP features and the HAAR features for a specific target can be saved, loaded, and effectively used for object recognition. An LBP code can be obtained for each pixel point in an image by using an LBP operator. After the original LBP operator is extracted from the image, the obtained original LBP features are still an image. Therefore, the embodiment of the present application is based on a trained LBP feature cascade detector. When used, relevant face detection cascade analyzer data is called. After a face region is intercepted, the upper half of the face region is taken, and then the upper half is evenly divided into left and right parts, which are left and right parts of eyes. Then, an eye region is intercepted according to a proportion of the eye region in the upper half, and selection and calibration of the eye region are completed. Furthermore, eye detection is realized by using an eye cascade detector in opencv. A detected eye object sub-image is cached as a template, so that when the detector cannot detect the eye region, the above-mentioned template image is used to complete matching of the eye region. When an iris is positioned, the calibrated eye region is binarized, so that an outline of the eye can be obtained, and first eye socket position information can be obtained. To obtain a position of the iris, a circular structure element is used to perform an opening operation (first erosion and then expansion) on the image. At this time, noise exists in a central circular region, and the noise needs to be removed, including rectangular or circular light and shadow. Therefore, the first light and shadow can be obtained. Then, a circular structure element is used to extract a position of the iris, and first iris position information is obtained. For example, a center position of the circular structure can be used as the position of the iris, that is, the center position of the circular structure is used as the first iris position information.

[0085] It can be understood that the embodiment of the present application takes the realization of processing of a face image based on opencv to obtain the first eye socket position information, the first iris position information, and the first light and shadow as an example, and does not represent a limitation on the embodiment of the present application.

[0086] Reference Figure 7 In an embodiment, the step S303 can further include, but is not limited to, steps S401 to S404.

[0087] In step S401, a coordinate mapping relationship table under a corresponding distance is matched according to the first viewing distance.

[0088] In step S402, a light and shadow coordinate of the first light and shadow is calculated.

[0089] Step S403: According to the first eye parameter and the light shadow coordinate, a table lookup result is obtained in the coordinate mapping relationship table.

[0090] Step S404: According to whether the gaze coordinate of the user's eye gaze on the display screen exists in the table lookup result, a judgment result is obtained.

[0091] It should be noted that in the embodiment of the present application, the angle of the user's view can be obtained according to the first viewing distance, the first eye parameter and the first light shadow, and the angle is the angle of the eye relative to the normal line of the center of the display screen, so that whether the user views on the display screen is obtained according to the different angles. In another embodiment, whether the user gazes in the coordinate range of the display screen can be determined according to the preset coordinate mapping relationship table.

[0092] Specifically, in the embodiment of the present application, different coordinate mapping relationship tables are established according to different viewing distances. The coordinate mapping relationship table is a mapping table for obtaining the corresponding gaze coordinate according to the first eye parameter and the first light shadow. In the embodiment of the present application, the coordinate mapping relationship table corresponding to the distance is matched according to the value of the first viewing distance, and the light shadow coordinate is calculated according to the first light shadow. It can be understood that the light shadow coordinate can be the geometric center point or the average point of the first light shadow, which can be set according to the actual situation, and the shape of the first light shadow can be rectangular or circular, which represents the shape of the light shadow of the display screen after image processing. In the embodiment of the present application, the gaze coordinate of the user's eye gaze on the display screen is obtained in the coordinate mapping relationship table according to the first eye parameter and the light shadow coordinate. It can be understood that the judgment result is obtained by determining whether the gaze coordinate corresponds to the display screen, which can represent a specific position on the display screen. The judgment result can also be obtained by determining whether the gaze coordinate is in the corresponding coordinate range. The coordinate mapping relationship table can represent the coordinate range of the display screen. When it represents the display range, if the coordinate obtained by the table lookup result is in the coordinate range, it means that the user gazes on the display screen, without the need to calculate which coordinate point. Therefore, the device control method in the embodiment of the present application can obtain whether the user's eye gazes on the display screen through the coordinate mapping relationship table to obtain the judgment result, which is simple and efficient, without the need for the terminal to perform complex calculation and without the need to consume a large amount of calculation resources.

[0093] Reference Figure 8 In an embodiment, the device control method can further include, but is not limited to, steps S501 to S504.

[0094] Step S501: A sample cursor is displayed on the display screen.

[0095] In step S502, a second eye parameter and a second light shadow when the user's eyes gaze at the sample cursor are obtained, wherein the second eye parameter is used to represent the position of the user's eye iris, and the second light shadow is the light shadow of the display screen or the sample cursor on the user's eyes.

[0096] In step S503, a second viewing distance to the display screen when the user's eyes gaze at the sample cursor is obtained.

[0097] In step S504, a corresponding relationship between the second eye parameter and the second light shadow is recorded at the second viewing distance, so as to establish a coordinate mapping relationship table of the sample cursor.

[0098] It should be noted that in the embodiment of the present application, the coordinate mapping relationship table can be established in advance according to sample data, so that whether the gaze coordinate is within the coordinate range of the display screen can be directly obtained according to the coordinate mapping relationship table established in advance in the process of table lookup. In the process of establishing the coordinate mapping relationship table in advance, the sample cursor is first displayed on the display screen. The sample cursor can be a bright spot or a bright ring existing in the display screen, and the sample cursor can be displayed in a flashing manner so as to be observed and paid attention to by the user. Then, the second eye parameter and the second light shadow when the user's eyes gaze at the sample cursor are obtained, wherein the second eye parameter is used to represent the position of the user's eye iris, and the second light shadow is the light shadow of the display screen or the sample cursor on the user's eyes. It can be understood that the second eye parameter is similar to the first eye parameter in the above embodiment, and the second light shadow is similar to the first light shadow in the above embodiment, which can be understood as data obtained according to samples and data obtained according to actual conditions, and is not limited specifically here. Then, the second viewing distance to the display screen when the user's eyes gaze at the sample cursor is obtained. The second viewing distance is the distance between the user and the display screen when the user watches the sample cursor. When the user is at different positions, the second viewing distance can have multiple values. Finally, the corresponding relationship between the second eye parameter and the second light shadow is recorded at the second viewing distance, so as to establish the coordinate mapping relationship table of the sample cursor. Therefore, the coordinate mapping relationship table established is a mapping table at different viewing distances.

[0099] As Figure 9 It can be understood that when the judgment result is obtained by judging whether the gaze coordinate is within the corresponding coordinate range, the coordinate mapping relationship table can represent the coordinate range of the display screen. The sample cursor in the embodiment of the present application can be displayed around the display screen, representing the coordinate range of the display screen.

[0100] Referring to Figure 10 In an embodiment, the step S501 can further include but is not limited to steps S601 to S602.

[0101] In step S601, the display screen is evenly divided into a plurality of display regions.

[0102] Step S602, display the sample cursors on each display area respectively.

[0103] It should be noted that the coordinate mapping relationship table needs to be established for the coordinate mapping relationship of the plurality of display areas on the display screen in the embodiment of the present application, therefore the display screen is evenly divided into a plurality of display areas, and the sample cursors are displayed on each display area respectively, it can be understood that the terminal can display the sample cursors on each display area in turn, or display the sample cursors on any display area randomly, in an embodiment, the terminal divides the display screen into 100 areas, changes the fixation point by the eyeball rotation, collects the eye parameters by the fixation point of the eye attention, divides the screen into a plurality of areas, for example, 100 areas, labels each area of the eye fixation, for example, the labeled area flashes when being labeled, the eye fixates on this area at this time, labels the eyeball position characteristics of the left and right eyes, thereby obtaining the corresponding second eye parameters and second light shadows.

[0104] Referring to Figure 11 In an embodiment, the step S504 can further include but is not limited to steps S701 to S702.

[0105] Step S701, display the sample cursors on each display area in turn.

[0106] Step S702, when the viewing position of the user is unchanged, record the corresponding relationship between the plurality of second eye parameters and the plurality of second light shadows corresponding to each sample cursor viewed by the user at the second viewing distance respectively, to establish the coordinate mapping relationship table of the sample cursor.

[0107] It should be noted that the coordinate mapping relationship table of the sample cursor is established according to the displayed sample cursor, and the corresponding relationship between the plurality of second eye parameters and the plurality of second light shadows corresponding to each sample cursor is collected in turn under the condition that the fixed distance is unchanged, the established coordinate mapping relationship table can more represent the second eye parameters and the second light shadows of the user viewing the display screen when the position is unchanged.

[0108] Specifically, assuming that the display screen is in front of the user, the user's eyes gaze at the sample cursor in the center of the display screen, there is a light shadow of the sample cursor in the center of the eyeball, at this time, the sample cursor is on the right of the left eye, the left eye looks right, the light shadow of the sample cursor in the left eye is on the left of the center of the eyeball, the sample cursor is on the left of the right eye, the right eye looks left, the light shadow of the sample cursor in the right eye is on the right of the center of the eyeball, and then the positions of the light shadows (the shadows of the sample cursor on the eyeball) on the eyeball are marked, so that the second light shadow (the shadow of the cursor on the eyeball) when gazing at the center of the screen is marked. When looking up, the light shadow is below the eyeball, and when looking up to the left, the light shadow is below and to the right of the eyeball (the light shadow of the right eye is more to the right), the positions of the light shadows of the cursor on the eyeballs of the left and right eyes are marked, and the rest is not repeated. The eye conditions of the user when watching the 100 sample cursors on the display screen up, down, left and right are marked, and the corresponding relationship between the multiple second eye parameters and the multiple second light shadows corresponding to each sample cursor watched by the user in turn is recorded, so as to establish the coordinate mapping relationship table of the sample cursor.

[0109] Reference Figure 12 In an embodiment, the step S502 can further include but is not limited to steps S801-S802.

[0110] In step S801, when there are multiple second viewing distances, the second eye parameters and the second light shadows of the user's eyes when gazing at the sample cursor are obtained under multiple different second viewing distances.

[0111] In step S802, or, under the second viewing distance, the second eye parameters and the second light shadows of the user's eyes when gazing at the sample cursor at different viewing angles are obtained; wherein the viewing angle is the angle formed by the plane where the user's face is located and the display screen.

[0112] It can be understood that when the user is at different distances, the image captured by the front camera on the terminal is different, and the rotation of the user's eyes when watching the display screen is different. For example, when the user's eyes are close to the display screen, the rotation of the eyes is larger when watching the uppermost and lowermost positions of the display screen, and when the user's eyes are far from the display screen, the rotation of the eyes will be much smaller when watching the uppermost and lowermost positions of the display screen. In the embodiment of the present application, the second viewing distance is multiple, and the user gazes at the sample cursor at different viewing distances, so that the corresponding relationship between the second eye parameters and the second light shadows is recorded under multiple different second viewing distances, so as to establish the coordinate mapping relationship table of the sample cursor. Therefore, the coordinate mapping relationship table established is according to the mapping table under multiple different viewing distances, so when the first viewing distance of the user's eyes to the display screen is obtained, the coordinate mapping relationship table under the corresponding second viewing distance can be obtained, and the gazing condition of the user's eyes at this distance can be more accurately recognized.

[0113] In an embodiment, the embodiment of the present application respectively labels and establishes the correspondence between the sample cursor of the gaze point and the eye parameters and the light shadow in the human eye under different distances according to the 30 cm, 40 cm, 90 cm, 100 cm of the human eye in front of the display screen. In the actual use process, the user can call the coordinate mapping relationship table under the current distance according to the distance between the screen and the human eye, and can determine the gaze of the user under different distances.

[0114] It should be noted that when the angle of the display screen is rotated, for example, the screen is rotated 5° to the right rear, at this time, the shadow of the display screen on the eyeball is larger on the left and smaller on the right. In the embodiment of the present application, the coordinate mapping relationship table is established according to the different angles formed by the plane where the user's face is located and the display screen, for example, when the distance between the user and the display screen remains unchanged at the second viewing distance, the table is established once when the user faces the display screen, and the table is established at angles of 5°, 10° or 30°, which can be set according to actual needs. The established coordinate mapping relationship table can still obtain an accurate gaze when the user views the display screen at different angles. When the user views the display screen at different angles, the front camera can obtain images of the user viewing the display screen at different angles, so that the finally established coordinate mapping relationship table can still obtain an accurate gaze when the user views at different angles.

[0115] For example, the above-mentioned methods can be combined to measure the parameters of the user's eyes when viewing the sample cursor under different distances and angles, and a coordinate mapping relationship table based on the user's gaze under different angles and distances can be established to meet the actual application when the user views the display screen at different angles and distances. It can be understood that the more sample cursors corresponding to the parameters of the human eye are measured, the higher the accuracy of the table established, and the more accurate the judgment result obtained. Similarly, the more parameters of the human eye are measured under different distances and angles, the higher the accuracy of the table established, and the more accurate the judgment result obtained.

[0116] It should be noted that in addition to directly searching the coordinate mapping relationship table under the first viewing distance according to the first eye parameter, the first light shadow to obtain the gaze coordinate in the embodiment of the present application, the preset fitting coordinate model or neural network model can also be obtained, and the first viewing distance, the first eye parameter and the first light shadow are input into the fitting coordinate model or the neural network model to obtain the gaze coordinate of the user's eye on the display screen, and then the judgment result is obtained, which is not limited here.

[0117] Reference Figure 13 In an embodiment, the first eye parameter is obtained by analyzing the face image of the user, and the above-mentioned step S102 can further include but is not limited to steps S901 to S903.

[0118] In step S901, the left eye pupil position information and the right eye pupil position information of the user's eyes are determined according to the first eye parameter, the picture interpupillary distance of the user is calculated according to the left eye pupil position information and the right eye pupil position information, and the first viewing distance of the user's eyes to the display screen is calculated according to the picture interpupillary distance.

[0119] In step S902, alternatively, the left eye pupil position information and the right eye pupil position information of the user's eyes are determined according to the first eye parameter, the picture face distance of the user is calculated from the face image according to the left eye pupil position information and the right eye pupil position information, the picture face ratio is obtained according to the picture face distance, the preset standard face ratio is obtained, and the viewing angle between the user and the display screen is obtained according to the picture face ratio and the standard face ratio.

[0120] In step S903, alternatively, the sum of the time of the user's eyes staring at the display screen is calculated according to the first eye parameter, and the viewing time of the user's eyes staring at the display screen is obtained.

[0121] It should be noted that the device control method in the embodiment of the application can measure the distance based on the camera, obtain the face image of the user, the face image is obtained by the front camera, perform pupil recognition on the face image, the left eye pupil position information and the right eye pupil position information of the user's eyes can be obtained according to the center points of the left eye iris position information and the right eye iris position information, the picture interpupillary distance of the user in the face image is calculated according to the left eye pupil position information and the right eye pupil position information of the user's eyes, the viewing distance of the user's eyes to the display screen can be determined according to the pixel number of the display screen, and finally the first viewing distance is calculated according to the picture interpupillary distance. The embodiment of the application can realize distance measurement through the front camera arranged on the terminal, obtain the pupil position of the user through image analysis of the front camera, calculate the required picture interpupillary distance, the picture interpupillary distance represents the interpupillary distance of the user in the obtained image, and the viewing distance of the user's eyes to the display screen can be calculated according to the picture interpupillary distance. The distance measurement cost is low, and other sensors do not need to be additionally arranged to realize distance measurement.

[0122] It can be understood that the terminal obtains the face image through the front camera, the interpupillary distance of the user on the face image is the picture interpupillary distance, in the process of photographing by the front camera, the position of the user relative to the camera can change at any time, the image size formed by the user at different distances is different through camera imaging, such as Figure 14a and Figure 14b , in Figure 14aIn the case that the user is close to the camera, the size of the user's face in the image is large, and thus the pupil distance is large, while in the case that the user is far from the camera, the size of the user's face in the image is small, and thus the pupil distance is small. Figure 14b In the case that the user is close to the camera, the size of the user's face in the image is large, and thus the pupil distance is large, while in the case that the user is far from the camera, the size of the user's face in the image is small, and thus the pupil distance is small.

[0123] For example, there is a 10 cm reference object (such as a ruler) at a distance of 50 cm in front of the display screen, according to the parameter characteristics of the front camera, the photographed photo, the 10 cm object in the image obtained by the front camera will become a certain size (which can be determined according to the number of pixels), and now the size of the target object (i.e., the two pupils) in the obtained image is 6.3 cm, the size of the target object in the photo is determined, and thus the distance of the target object to the display screen can be calculated.

[0124] It should be noted that the pupil distance is the distance between the pupils of the user's eyes, which can also be referred to as the interpupillary distance, and refers to the length between the centers of the pupils of the eyes, and the normal value of the interpupillary distance of an adult is in the range of 58-64 mm, and the interpupillary distance itself is determined by the individual's genetics and development, and thus the interpupillary distance of different ages is different, and for a certain user, the interpupillary distance is certain, and thus the distance of the user to the terminal can be determined according to the size of the image interpupillary distance in the face image, and the first viewing distance of the user's eyes to the display screen is calculated.

[0125] It should be noted that in the embodiment of the present application, the image of the user is recognized by the front camera, and no additional sensor device needs to be set, and the distance can be measured, the design cost is low, no additional hardware setting is needed, and the method can be applied to the terminal with the front camera, and it can be understood that the processor of the terminal can execute the method in the embodiment of the present application, the image is obtained by the front camera, and finally the processor is calculated, and thus the accurate distance measurement can be realized.

[0126] It should be noted that when the user watches the display screen, the user can watch at a certain angle, for the front camera, the obtained image is a two-dimensional planar image, and the angle of rotation of the user's face cannot be distinguished according to the image alone. Based on this, the embodiments of the present application obtain the viewing angle of the user by judging the face ratio, first determine the left eye pupil position information and the right eye pupil position information of the user's eyes according to the first eye parameter, calculate the picture face distance of the user from the face image according to the left eye pupil position information and the right eye pupil position information, at this time, the left eye pupil position information and the right eye pupil position information play a positioning role, then combine face recognition, frame the face area of the user, calculate the length and width of the face, and thus obtain the picture face ratio, then obtain the preset standard face ratio, the standard face ratio is the face ratio of the user in reality, that is, the actual face ratio, the standard face ratio can be a default setting or input by the user, which will not be described here. It can be understood that the embodiments of the present application can calculate the length and width of the user's face in the face image to obtain the picture face ratio. When the user's face is directly opposite the front camera, the ratio of the width to the length of the user's face is maximum, that is, the ratio is maximum, and as the user's face rotates, the ratio of the width to the length of the user's face in the face image obtained by the front camera will decrease, and the viewing angle of the user can be obtained according to the picture face ratio and the standard face ratio.

[0127] It should be noted that in the embodiments of the present application, the rotation angle is obtained by comparing the picture face ratio with the standard face ratio. The basis for comparison and analysis can be pre-established according to experiments to obtain a mapping relationship with the angle. For example, in an embodiment, according to the comparison relationship between the sample picture face ratio and the standard face ratio, the viewing angle under each condition is recorded, so that the rotation angle of the user's face can be obtained directly according to the picture face ratio and the standard face ratio in the subsequent process. In another embodiment, the values of the picture face ratio and the standard face ratio can be input into a neural network model to obtain the viewing angle. The neural network model can be established according to the data in the sample, and the embodiments of the present application do not make specific limitations.

[0128] It should be noted that the viewing time obtained in the embodiments of the present application is accumulated. Specifically, since the viewing condition of the user can be judged according to the iris position of the user in the embodiments of the present application, the time during which the user's eyes gaze at the display screen can be accumulated. For example, it is assumed that when the user starts to watch the display screen and first reaches 15 minutes, the user turns his head to chat with others. At this time, according to the recognized pupil position information, it is judged that the user is in the shooting range of the front camera, but is not watching the display screen, so the time accumulation is paused. When the user watches the display screen again after chatting with others for 1 minute, the time accumulation is continued, and thus the real viewing time is obtained.

[0129] Referring toFigure 15 In an embodiment, the calculating the first viewing distance from the eyes of the user to the display screen according to the interocular distance in the picture in step S902 can further include, but is not limited to, steps 1001 to 1003.

[0130] In step 1001, a preset standard interocular distance is obtained; a focal length of a face image captured by the front camera is obtained, and an initial distance of the face image corresponding to an imaging point is obtained according to the focal length; a first ratio is obtained according to the interocular distance in the picture and the standard interocular distance, and the first viewing distance from the eyes of the user to the display screen is obtained according to the first ratio and the initial distance.

[0131] In step 1002, alternatively, a preset distance query table is obtained; the first viewing distance from the eyes of the user to the display screen is obtained by looking up the distance query table according to the interocular distance in the picture.

[0132] In step 1003, further alternatively, a reference distance, a reference object size, and a picture size of a reference object corresponding to the front camera are obtained; a preset standard interocular distance is obtained; the first viewing distance from the eyes of the user to the display screen is obtained according to the reference distance, the reference object size, the picture size, the interocular distance in the picture, and the standard interocular distance.

[0133] It should be noted that in the calculation of the first viewing distance from the eyes of the user to the display screen according to the interocular distance in the picture, specifically, the preset standard interocular distance is first obtained, the standard interocular distance is the interocular distance of the user in reality, and the standard interocular distance can be a default setting, for example, set to 63 mm, or the standard interocular distance can be input by the user, so that the user can accurately input the interocular distance, or the interocular distance of different age groups and different genders can be analyzed by big data and artificial intelligence, and the interocular distance of a 63 mm adult can be replaced by the conclusion of the analysis to obtain a more accurate interocular distance and a more accurate first viewing distance. Subsequently, the focal length of the face image captured by the front camera is obtained, and the initial distance of the face image corresponding to the imaging point is obtained according to the focal length, and finally the first ratio is obtained according to the interocular distance in the picture and the standard interocular distance, and the first viewing distance from the eyes of the user to the display screen can be obtained according to the first ratio and the initial distance.

[0134] It can be understood that each camera should have a certain field of view angle (FOV) and focal length when shooting, and the focal length and field of view angle of each camera are one-to-one corresponding, which can be obtained by a public way or measured, the field of view angle is the angle between the two ends of the camera cone, and the focal length is the distance from the lens of the camera to the internal “sensor”, but in actual camera, the sensor is behind the lens, in order to simplify, it can be assumed that the lens is in front of the sensor, and the focal length can be obtained relative to the lens mirror image. Figure 16The plane where the picture is shown, for example, the plane where the sensor is located, is the plane where the face image is located, and the formed face image is equivalent to being above the plane where the lens is located. The position of the lens can be described as the imaging point in the embodiment of the application. The plane where the imaging point is located is below the plane where the face image is located and is arranged in parallel. Therefore, the position of the plane where the face image is located relative to the plane where the imaging point is located can be obtained according to the focal length. In an embodiment, the distance between the plane where the face image is located and the plane where the imaging point is located can be obtained according to the focal length, which is defined as the initial distance.

[0135] It can be understood that the plane where the face image is located corresponds to the plane where the display screen is located, which is determined according to the wide angle and focal length of the front camera. In an embodiment, the plane where the face image is located is the plane where the display screen is located. Alternatively, the plane where the face image is located plus or minus a small distance can also obtain the plane where the display screen is located. This can be pre-calculated according to the physical parameters of the front camera used and applied in subsequent processing. The embodiment of the application takes the plane where the face image is located as the plane where the display screen is located as an example.

[0136] It should be noted that in the embodiment of the application, the initial distance is obtained according to the focal length, and the initial distance can also be obtained by acquiring the field of view of the shot. However, since the field of view and the focal length have a one-to-one correspondence, the processing is taken as an example by acquiring the focal length. It should be noted that the initial distance can be calculated according to the characteristics of camera imaging or can be pre-calculated. However, it can be understood that each different focal length will correspond to an initial distance, which is not specifically limited here.

[0137] It can be understood that, as Figure 17 , according to the characteristics of camera imaging, the line segment of the actual interpupillary distance of the user and the imaging point form a triangle, and the line segment of the picture interpupillary distance is located in the triangle and is parallel to the line segment of the actual interpupillary distance. In an embodiment, the triangle formed by the line segment where the picture interpupillary distance is located and the imaging point is a similar triangle to the triangle formed by the line segment where the actual interpupillary distance is located and the imaging point. Since the initial distance is known and the picture interpupillary distance and the actual interpupillary distance can obtain the first proportion, the first viewing distance between the user's eyes and the display screen can be obtained according to the initial distance and the first proportion.

[0138] It should be noted that in the process of obtaining the first viewing distance from the user's eyes to the display screen, the calculation is performed according to the characteristics of the triangle. In an embodiment, the triangle formed by the line segment where the picture interpupillary distance is located and the imaging point is defined as the first triangle, and the triangle formed by the line segment where the actual interpupillary distance is located and the imaging point is defined as the second triangle. The first triangle and the second triangle are similar triangles. Figure 17 Figure 17 It should be noted that in the process of obtaining the first viewing distance from the user's eyes to the display screen, the calculation is performed according to the characteristics of the triangle. In an embodiment, the triangle formed by the line segment where the picture interpupillary distance is located and the imaging point is defined as the first triangle, and the triangle formed by the line segment where the actual interpupillary distance is located and the imaging point is defined as the second triangle. The first triangle and the second triangle are similar triangles. ​

[0139] As Figure 17 In the embodiment, the distance from the user's eyes to the imaging point can be obtained according to the first ratio and the initial distance, the first ratio is the picture interpupillary distance Q divided by the actual interpupillary distance K, then the initial distance H0 is divided by the first ratio to obtain the distance H1, and finally the distance H1 is subtracted from the initial distance H0 to obtain the first viewing distance H of the user's eyes to the display screen, and the calculation formula of H is as follows:

[0140] H = H1 - H0 (1)

[0141] H1 = H0 / (Q / K) (2)

[0142] In an embodiment, the embodiment of the present application can obtain a more accurate first viewing distance according to the rotation angle of the user's face, and correct it. Specifically, when the user watches the display screen, he may watch at a certain angle. For the front camera, the image obtained is a two-dimensional planar image, and the rotation angle of the user's face cannot be distinguished simply according to the image. If the interpupillary distance is calculated directly at this time, an error will be caused, thereby causing inaccurate distance measurement. Therefore, geometric calculation can be performed according to the specific position of the front camera to correct the parameters. In addition, when the pupils are not in front of the front camera, correction can also be made through geometric principles. When the display screen and the front camera are not in the same plane, correction can also be made according to the distance difference. The embodiment of the present application is not limited specifically.

[0143] In addition, it should be noted that the first viewing distance in the embodiment of the present application can also be obtained by querying a preset distance query table. Specifically, a mapping relationship table of picture interpupillary distance to first viewing distance can be established in advance in the embodiment of the present application. In the process of calculating the first viewing distance, the preset distance query table is obtained first, and the first viewing distance of the user's eyes to the display screen is obtained from the distance query table according to the measured picture interpupillary distance.

[0144] It should be noted that the distance query table in the above embodiment can be calculated according to the data in the sample. It can be understood that when it is necessary to reduce errors by measuring the face ratio of the user, the distance query table can also be established based on the rotation angle, which is not limited specifically herein.

[0145] Further, it needs to be explained that the first viewing distance can also be obtained according to the establishment of the reference system in the embodiment of the application, specifically, the embodiment of the application first places a reference object in front of the terminal, measures the reference distance from the reference object to the display screen and the object size of the reference object, captures the imaging of the reference object through the front camera, calculates the size of the reference object in the image in the captured image to obtain the picture size, and then the reference system can be established accordingly, so that the first viewing distance from the eyes of the user to the display screen can be obtained according to the reference distance, the object size of the reference object, the picture size, the picture interpupillary distance and the standard interpupillary distance.

[0146] Specifically, the first coefficient is obtained by dividing the standard interpupillary distance by the picture interpupillary distance, the second coefficient is obtained by dividing the object size of the reference object by the picture size, the third coefficient is obtained by dividing the reference distance by the second coefficient, and finally the first viewing distance from the eyes of the user to the display screen is obtained according to the product of the third coefficient and the first coefficient. In addition, the picture size and the picture interpupillary distance can be calculated according to the pixel points of the display screen, for example, as shown in Figure 18 and Figure 19 When the object size of the standard reference object is 10 cm, the reference distance is 50 cm, the picture size is AB, and the standard interpupillary distance is taken as an example, the picture interpupillary distance is ab, the first coefficient is 6.3÷ab, the second coefficient is 10÷AB, and finally the formula of the first viewing distance h can be established as follows:

[0147] 50÷(10÷AB)=h÷(6.3÷ab) (3)

[0148] Since the picture size AB and the picture interpupillary distance ab are known, the first viewing distance h can be obtained according to the formula (3).

[0149] Referring to Figure 20 In an embodiment, the step S104 can further include but is not limited to steps S1101 to S1102.

[0150] In step S1101, when the information size represented by the viewing information is within a preset first threshold range, the display screen is controlled to enter a first working mode, and the first working mode includes at least one of controlling the display screen to display prompt content and adjusting the display brightness of the display screen.

[0151] In step S1102, when the information size represented by the viewing information is within a preset second threshold range, the display screen is controlled to enter a second working mode, and the second working mode includes turning off the display screen.

[0152] It should be noted that the embodiment of the present application performs different controls according to the size of the viewing information, so that the display screen can be in different working modes. Specifically, when the information size represented by the viewing information is within a preset first threshold range, the display screen is controlled to enter a first working mode, the first working mode including at least one of displaying prompt content and adjusting the display brightness of the display screen; when the information size represented by the viewing information is within a preset second threshold range, the display screen is controlled to enter a second working mode, the second working mode including turning off the display screen.

[0153] When the viewing information is the first viewing distance, the first threshold range is a first distance threshold range, and the second threshold range is a second distance threshold range. For example, the first distance threshold range is greater than the second distance threshold range. When the user is within the first distance threshold range, it indicates that the distance to the display screen has approached, and a prompt needs to be issued. Therefore, at least one of the display screen displays prompt content and adjusts the display brightness of the display screen. A prompt sign can be displayed on the display screen, or the brightness of the display screen is lowered. When the user continues to approach the display screen and is within the second distance threshold range, it indicates that the distance is very close at this time, and continued viewing is not conducive to eye health. Therefore, the display screen needs to be forcibly turned off to protect the user's eyes and ensure eye health, which is conducive to myopia prevention and control.

[0154] When the viewing information is the viewing angle, the first threshold range is a first angle threshold range, and the second threshold range is a second distance angle range. For example, the first angle threshold range is less than the second angle threshold range. When the user is within the first angle threshold range, it indicates that the viewing angle to the display screen has tilted to a certain extent, and a prompt needs to be issued. Therefore, at least one of the display screen displays prompt content and adjusts the display brightness of the display screen. A prompt sign can be displayed on the display screen, or the brightness of the display screen is increased. When the user continues to expand the viewing angle and is within the second angle threshold range, it indicates that the angle is very large at this time, and continued viewing is not conducive to eye health. Therefore, the display screen needs to be forcibly turned off to protect the user's eyes and ensure eye health, which is conducive to myopia prevention and control.

[0155] When the viewing information is the viewing time, the first threshold range is a first time threshold range, and the second threshold range is a second time threshold range. For example, the first time threshold range is smaller than the second time threshold range. When the user is within the first time threshold range, it indicates that the viewing time has been long, and a reminder needs to be sent. Therefore, the display screen is controlled to display at least one of the prompt content and the display brightness of the display screen is adjusted. The reminder mark can be displayed on the display screen, or the brightness of the display screen is lowered. When the user continues to watch the display screen and is within the second time threshold range, it indicates that the viewing time has been long, and it is not good for the eye health to continue to watch. Therefore, the display screen needs to be forcibly closed to protect the eyes of the user and ensure the eye health, which is conducive to myopia prevention and control.

[0156] It can be understood that when the viewing information includes other information, different controls can be performed according to different viewing information, which is not specifically limited herein.

[0157] In an embodiment, when the information size represented by the viewing information is within a preset first threshold range, the device control method can further include but is not limited to the following:

[0158] At least one of the following is controlled: the terminal sends a first sound prompt and the terminal sends a first vibration prompt.

[0159] When the information size represented by the viewing information is within a preset second threshold range, the device control method can further include but is not limited to the following:

[0160] At least one of the following is controlled: the terminal sends a second sound prompt and the terminal sends a second vibration prompt.

[0161] It should be noted that in the embodiments of the present application, prompt information can be sent according to the viewing situation of the user watching the display screen. Specifically, sound prompt information, graphical prompt information, vibration prompt information, etc. can be sent. For example, when it is judged that the user is too close to the display screen, the prompt information is sent to prompt the user to watch too close by sound, graphics or vibration, and the user needs to move away from the display screen. For another example, when it is judged that the viewing time of the user is too long, the prompt information is sent to prompt the user that the viewing time exceeds the healthy time length by sound, graphics or vibration, and the user needs to stop watching or close the display screen for rest. For another example, when it is judged that the viewing angle of the user is not correct, the prompt information is sent to prompt the user that the viewing angle is not correct by sound, graphics or vibration, and the user needs to correct the viewing angle to avoid damage to the eyes caused by the angle, such as causing astigmatism, etc. Thus, it is conducive to myopia prevention and control.

[0162] It should be noted that the viewing time calculated in the embodiment of the application is accumulated, and the display screen can be controlled according to the viewing time interval of the user, for example, when the accumulated viewing time reaches 20 minutes, the display screen is controlled to be closed, a sound and vibration prompt are issued, the display screen is closed for 20 seconds, and then the display screen is opened again, so that the harm to the vision caused by the long viewing time of the user at a time is avoided, and it can be understood that the time for closing and the time for closing for a long time can be set according to the health regulations or set according to the user demand.

[0163] With reference to Figure 21 In an embodiment, the viewing information includes the viewing time, and the step S104 can further include but is not limited to steps S1201 to S1202.

[0164] In step S1201, the age information of the user is acquired.

[0165] In step S1202, when the age information indicates that the user is a child, the display screen is controlled to enter a third working mode according to the viewing time, and the third working mode includes controlling the opening or closing of the display screen according to the viewing time interval.

[0166] It should be noted that in the embodiment of the application, the user can be determined to be a child user, and the age information of the user can be acquired to determine the child control mode, and the opening or closing of the display screen is controlled according to the viewing time interval in the child control mode, for example, when the accumulated viewing time reaches 10 minutes, the display screen is controlled to be closed, a sound and vibration prompt are issued, the display screen is closed for 1 minute, and then the display screen is opened again, so that the harm to the vision caused by the long viewing time of the user at a time is avoided, and it can be understood that the time for closing and the time for closing for a long time can be set according to the health regulations or set according to the user demand, but the viewing time in the child control mode is shorter than that in the adult mode, and the rest time is required to be longer.

[0167] In an embodiment, the terminal can identify the age information of the user by the acquired face image of the user, for example, the terminal inputs the photographed face image into a preset age prediction network (such as a caffe model), and outputs the age information of the user through the age prediction network; or, the face (region of interest, ROI) of the person is extracted from the face image, and the age information is obtained by predicting the age of the person through the algorithm of the age detector; or, the inter-pupillary distance and the face ratio of the user in the face image are identified, and the age information is obtained by comparison with a preset comparison library, the inter-pupillary distance of users in different age groups is different, and the face ratio of children is smaller; or, the skin smoothness of the face of the user in the face image is identified, and the age information of the user is obtained according to the smoothness. In addition, the age information can also be obtained through the input of the user, which is not limited here.

[0168] In an embodiment, the front camera is an under-screen camera, and the display screen is an OLED screen, so that the front camera can be arranged below the display screen. Specifically, the under-screen camera is arranged at the center position of the display screen. By arranging the under-screen camera at the center position of the display screen, the face image of the user can be acquired at this position, so that the inter-pupillary distance of the user can be more accurately measured, the distance measurement can be more accurately measured, and the gaze of the eyes of the user can be more accurately determined.

[0169] Reference Figure 22 In the embodiment of the present application, a device control apparatus is also provided, and the apparatus comprises:

[0170] The first module 2201 is configured to acquire a first eye parameter of the eyes of the user, and the first eye parameter is used to represent the position of the iris of the eyes of the user.

[0171] The second module 2202 is configured to determine viewing information between the eyes of the user and the display screen according to the first eye parameter, wherein the viewing information comprises a first viewing distance, a viewing angle and a viewing time.

[0172] The third module 2203 is configured to determine whether the eyes of the user gaze at the display screen according to the first eye parameter, and obtain a determination result.

[0173] The fourth module 2204 is configured to determine a working mode of the display screen according to the viewing information when the determination result represents that the eyes of the user gaze at the display screen.

[0174] It should be noted that the device control apparatus in the embodiments of the present application can implement the device control method in any one of the above embodiments. The device control apparatus can be a terminal device such as a mobile phone, a tablet computer, a 3D visual training terminal, etc. The device control apparatus can obtain the position of the user's eye iris, obtain the first eye parameter, determine the first viewing distance, the viewing angle and the viewing time between the user and the display screen according to the position of the user's eye iris, and determine whether the user is gazing at the display screen according to the position of the user's eye iris. When the display screen is controlled, the control can be performed only when the user is gazing at the display screen, and the working mode of the display screen can be determined according to the viewing information, thereby improving the user experience and being conducive to myopia prevention and control.

[0175] It should be noted that the first module 2201, the second module 2202, the third module 2203 and the fourth module 2204 can be various functional modules on the terminal. In an embodiment, the modules can be various functional modules in the processor, and can be executed by the processor arranged on the terminal. In the embodiments of the present application, the hardware structure of the device control apparatus is only embodied in the form of functional modules, and does not represent a limitation on the embodiments of the present application.

[0176] Figure 23 An electronic device 2300 provided by the embodiments of the present application is shown. The electronic device 2300 includes a processor 2301, a memory 2302, and a computer program stored in the memory 2302 and executable on the processor 2301, and the computer program is used to execute the device control method described above when running.

[0177] The processor 2301 and the memory 2302 can be connected through a bus or other means.

[0178] The memory 2302 is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs, such as the device control method described in the embodiments of the present application. The processor 2301 runs the non-transitory software programs and instructions stored in the memory 2302, thereby implementing the device control method described above.

[0179] The memory 2302 can include a program storage area and a data storage area, where the program storage area can store an operating system, at least one application program required by the at least one function, and the data storage area can store data required for executing the above-mentioned device control method. In addition, the memory 2302 can include a high-speed random access memory 2302, and can also include a non-transitory memory 2302, such as at least one storage device memory, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 2302 can optionally include a memory 2302 disposed remotely from the processor 2301, and these remote memories 2302 can be connected to the electronic device 2300 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0180] The non-transitory software programs and instructions required to implement the above-mentioned device control method are stored in the memory 2302, and when executed by one or more processors 2301, the above-mentioned device control method is executed, for example, the method steps S101 to S104 in Figure 3 , the method steps S201 to S203 in Figure 4 , the method steps S301 to S303 in Figure 5 , the method steps S401 to S404 in Figure 7 , the method steps S501 to S504 in Figure 8 , the method steps S601 to S602 in Figure 10 , the method steps S701 to S702 in Figure 11 , the method steps S801 to S802 in Figure 12 , the method steps S901 to S903 in Figure 13 , the method steps S1001 to S1003 in Figure 15 , the method steps S1101 to S1102 in Figure 20 , and the method steps S1201 to S1202 in Figure 21 .

[0181] The above-described device embodiments are only illustrative, and units described as separate components can or can not be physically separated, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0182] As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the methods disclosed above can be embodied in software, firmware, hardware, and / or suitable combinations thereof. Some or all of the physical components can be implemented with software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented with hardware, or can be implemented with an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to a person of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to a person of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0183] It should also be appreciated that various embodiments provided by the embodiments of the present application can be arbitrarily combined to achieve different technical effects. The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application.

Claims

1. A device control method, applied in a terminal, said terminal being equipped with a display screen, characterized in that, The method includes: Obtain the first eye parameters of the user's eye, which are used to characterize the position of the iris of the user's eye; The viewing information between the user's eyes and the display screen is determined based on the first eye parameters, wherein the viewing information includes a first viewing distance, a viewing angle, and a viewing time; Control the display screen to display; acquire the first light and shadow of the display screen on the user's eyes; match the coordinate mapping relationship table at the corresponding distance according to the first viewing distance, wherein the coordinate mapping relationship table represents the coordinate range of the display screen; Calculate the light and shadow coordinates of the first light and shadow; based on the first eye parameters and the light and shadow coordinates, look up the coordinate mapping table to obtain the lookup result; based on whether there are gaze coordinates of the user's eyes on the display screen in the lookup result, obtain a judgment result; When the judgment result indicates that the user is looking at the display screen, the working mode of the display screen is determined based on the viewing information; The method further includes: displaying a sample cursor on the display screen; acquiring a second eye parameter and a second shadow when the user's eyes are focused on the sample cursor, wherein the second eye parameter is used to characterize the position of the user's iris, and the second shadow is the shadow of the sample cursor on the user's eyes; acquiring a second viewing distance from the user's eyes to the display screen when the user's eyes are focused on the sample cursor; and recording the correspondence between the second eye parameter and the second shadow at the second viewing distance to establish a coordinate mapping table of the sample cursor.

2. The equipment control method according to claim 1, characterized in that, The terminal is equipped with a front-facing camera; the acquisition of the user's first eye parameters includes: The user's facial image is acquired by the front-facing camera. The first eye socket position information and the first iris position information are obtained by parsing the facial image, wherein the first eye socket position information is used to characterize the position of the user's eye socket, and the first iris position information is used to characterize the position of the user's iris. The first eye parameters are obtained based on the first orbital position information and the first iris position information.

3. The equipment control method according to claim 1, characterized in that, The step of displaying the sample cursor on the display screen includes: The display screen is evenly divided into multiple display areas; The sample cursor is displayed on each of the aforementioned display areas; The step of recording the correspondence between the second eye parameters and the second light and shadow at the second viewing distance to establish a coordinate mapping table for the sample cursor includes: The sample cursor is displayed sequentially on each of the aforementioned display areas; When the user's viewing position remains unchanged, at the second viewing distance, the correspondence between multiple second eye parameters and multiple second light and shadow corresponding to each of the sample cursors viewed by the user in sequence is recorded to establish a coordinate mapping table of the sample cursors.

4. The equipment control method according to claim 1, characterized in that, The acquisition of the second eye parameters and second light and shadow when the user's eyes are focused on the sample cursor includes: When there are multiple second viewing distances, acquire the second eye parameters and second light and shadow when the user's eyes are looking at the sample cursor at multiple different second viewing distances; Alternatively, at the second viewing distance, acquire the second eye parameters and the second light and shadow when the user's eyes are looking at the sample cursor at different viewing angles; wherein, the viewing angle is the angle formed by the plane where the user's face is located and the display screen.

5. The equipment control method according to claim 1, characterized in that, The first eye parameter is obtained by parsing the user's facial image; determining the viewing information between the user's eyes and the display screen based on the first eye parameter includes: The position information of the user's left and right pupils is determined based on the first eye parameters. The interpupillary distance of the user in the face image is calculated based on the interpupillary distance of the left and right pupils. The first viewing distance from the user's eyes to the display screen is calculated based on the interpupillary distance of the face image. Alternatively, the position information of the user's left and right pupils can be determined based on the first eye parameters; the user's facial spacing can be calculated from the facial image based on the left and right pupil positions, and the facial proportion can be obtained based on the facial spacing; a preset standard facial proportion can be obtained; and the viewing angle between the user and the display screen can be obtained based on the facial proportion and the standard facial proportion. Alternatively, the sum of the time the user's eyes spend looking at the display screen can be calculated based on the first eye parameters to obtain the viewing time of the user's eyes looking at the display screen.

6. The equipment control method according to claim 5, characterized in that, The step of calculating the first viewing distance from the user's eyes to the display screen based on the interpupillary distance of the image includes: Obtain a preset standard interpupillary distance; obtain the focal length of the facial image captured by the front camera, and obtain the initial distance from the facial image to the imaging point based on the focal length; obtain a first ratio based on the interpupillary distance in the image and the standard interpupillary distance, and obtain a first viewing distance from the user's eyes to the display screen based on the first ratio and the initial distance; Alternatively, a preset distance lookup table can be obtained; the first viewing distance from the user's eyes to the display screen can be obtained by looking up the distance lookup table based on the interpupillary distance of the image. Alternatively, obtain a reference distance, a reference object size, and the image size corresponding to the reference object captured by the front-facing camera; obtain a preset standard interpupillary distance; and obtain the first viewing distance from the user's eyes to the display screen based on the reference distance, the reference object size, the image size, the image interpupillary distance, and the standard interpupillary distance.

7. The equipment control method according to claim 1 or 5, characterized in that, Determining the operating mode of the display screen based on the viewing information includes: When the size of the information represented by the viewing information is within a preset first threshold range, the display screen is controlled to enter a first working mode. The first working mode includes at least one of controlling the display screen to display prompt content and adjusting the display brightness of the display screen. When the size of the information represented by the viewing information is within a preset second threshold range, the display screen is controlled to enter a second working mode, the second working mode including turning off the display screen.

8. The equipment control method according to claim 7, characterized in that, When the size of the information represented by the viewing information is within a preset first threshold range, the method further includes: Controlling the terminal to emit a first sound prompt and controlling the terminal to emit a first vibration prompt, at least one of the following: When the size of the information represented by the viewing information is within a preset second threshold range, the method further includes: Controlling the terminal to emit a second sound prompt and controlling the terminal to emit a second vibration prompt are at least one of the following:

9. The equipment control method according to claim 1 or 5, characterized in that, Determining the operating mode of the display screen based on the viewing information includes: Obtain the user's age information; When the age information indicates that the user is a child, the display screen is controlled to enter a third working mode based on the viewing time. The third working mode includes controlling the display screen to turn on or off based on the viewing time interval.

10. The equipment control method according to claim 2, characterized in that, The front-facing camera is an under-display camera, and the under-display camera is located at the center of the display screen.

11. A device control apparatus, characterized in that, include: The first module is used to obtain the first eye parameters of the user's eye, which are used to characterize the position of the iris of the user's eye; The second module is used to determine the viewing information between the user's eyes and the display screen based on the first eye parameters, wherein the viewing information includes a first viewing distance, a viewing angle, and a viewing time; The third module is used to control the display screen to display; acquire the first light and shadow of the display screen on the user's eyes; match the coordinate mapping relationship table at the corresponding distance according to the first viewing distance, wherein the coordinate mapping relationship table represents the coordinate range of the display screen; calculate the light and shadow coordinates of the first light and shadow; look up the table in the coordinate mapping relationship table according to the first eye parameters and the light and shadow coordinates to obtain the lookup result; and obtain a judgment result based on whether there is a gaze coordinate of the user's eyes on the display screen in the lookup result. The fourth module is used to determine the working mode of the display screen based on the viewing information when the judgment result indicates that the user is looking at the display screen. The third module is further configured to: display a sample cursor on the display screen; acquire second eye parameters and second light and shadow when the user's eyes are focused on the sample cursor, wherein the second eye parameters are used to characterize the position of the user's iris, and the second light and shadow is the light and shadow of the sample cursor on the user's eyes; acquire a second viewing distance from the user's eyes to the display screen when the user's eyes are focused on the sample cursor; and record the correspondence between the second eye parameters and the second light and shadow at the second viewing distance to establish a coordinate mapping table of the sample cursor.

12. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the device control method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the device control method as described in any one of claims 1 to 10.

Images