Control method, intelligent terminal and storage medium

By using eye-tracking technology to divide the screen into visual zones and adjust the refresh rate, the problem of unintelligent screen frame rate control and poor power saving in existing technologies is solved, thus improving the user experience.

CN118942360APending Publication Date: 2024-11-12CHONGQING TRANSSION COMM TECH LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410781747.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing screen frame rate control technology is only applicable to LCD screens and cannot intelligently divide areas, resulting in a fragmented feel in fast-moving scenes, poor power saving, and a poor user experience.

Method used

Visual partitions are determined by eye-tracking, and different areas of the screen are divided according to the range of eye focus, with different refresh rates used to display data.

Benefits of technology

It achieves smarter screen frame rate control, reduces power consumption, minimizes visual disjointedness, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942360A_ABST
    Figure CN118942360A_ABST
Patent Text Reader

Abstract

The invention provides a control method, an intelligent terminal and a storage medium, and the control method comprises the steps: responding to a condition meeting a first preset condition, dividing visual partitions of a screen according to a sight line focusing range, and updating the display data of each visual partition according to a preset refreshing frame rate. According to the technical scheme, the visual partitions can be determined in combination with the sight capturing function, the screen data can be displayed at different refreshing frame rates for different visual partitions, the method is more intelligent and / or power-saving, and the user experience is better.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a control method, an intelligent terminal and a storage medium. Background Art

[0002] The existing screen frame rate control technology divides the GOA circuit of the screen into two areas with fixed positions, and then adds a frame rate switching switch in the GOA circuit to perform frame rate switching control according to the instructions of the AP end.

[0003] During the process of conceiving and implementing this application, the inventors discovered the following problems: the existing screen frame rate control is only applicable to LCD screens and can only control the division of row pixels, resulting in the screen being divided into areas only horizontally; and / or, the application scenarios are limited and are only applicable to low-speed scenarios. For example, when playing a video in a small window, the upper video area is refreshed at a high speed and the lower comment area is refreshed at a low speed. When the refresh rates of the two refresh areas appear in a fast dynamic scene at the same time, the human eye will perceive it very clearly and there will be a certain sense of fragmentation.

[0004] The above solutions are not intelligent and / or power-saving, which affects the user experience.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0006] The present application provides a control method, an intelligent terminal and a storage medium, which can determine visual partitions in combination with a line of sight capture function, and support displaying screen data at different refresh frame rates for different visual partitions, which is more intelligent and / or power-saving, and provides a better user experience.

[0007] The present application provides a control method, comprising the steps of:

[0008] S1: In response to satisfying a first preset condition, dividing the visual partitions of the screen according to the sight focus range;

[0009] S2: Update the display data of each visual partition according to the preset refresh frame rate.

[0010] Optionally, step S1 comprises the steps of:

[0011] S11: in response to the screen state satisfying the first preset condition, acquiring image information within the field of view angle;

[0012] S12: capturing human eye information on the image information to obtain a sight focus range;

[0013] S13: Divide the visual partitions of the screen according to the sight focus range.

[0014] Optionally, step S12 includes:

[0015] The human eye information in the image information is captured based on the structure of the human eye, and the visual focus range is determined by utilizing the positional relationship between the human eye information and the screen.

[0016] Optionally, step S12 includes:

[0017] The line of sight focusing range is determined according to the first position information of the first feature of the human eye information relative to the screen and the eye line of sight direction determined by the relative position relationship between the second feature and the third feature of the human eye information.

[0018] Optionally, satisfying the first preset condition includes at least one of the following:

[0019] The screen is on;

[0020] The screen is in use;

[0021] The camera’s field of view contains facial information;

[0022] The camera's field of view contains information about the human eye.

[0023] Optionally, the visual partition of the screen includes at least one of a first visual area, a second visual area and a third visual area.

[0024] Optionally, the preset refresh frame rate includes at least one of a first refresh frame rate, a second refresh frame rate and a third refresh frame rate.

[0025] Optionally, the control method further includes at least one of the following:

[0026] The second visual area includes the first visual area;

[0027] The third visual area includes the first visual area and the second visual area;

[0028] The first visual area is refreshed using a first refresh frame rate;

[0029] The second visual area is refreshed using a second refresh frame rate;

[0030] The third visual area is refreshed using a third refresh frame rate;

[0031] The first refresh frame rate is higher than the second refresh frame rate;

[0032] The second refresh frame rate is higher than the third refresh frame rate.

[0033] Optionally, the control method further includes: in response to satisfying a second preset condition, updating the display data using a fourth refresh frame rate.

[0034] The present application also provides an intelligent terminal, comprising: a memory and a processor, wherein a control program is stored in the memory, and when the control program is executed by the processor, the steps of any of the control methods described above are implemented.

[0035] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the control methods described above are implemented.

[0036] As described above, the control method of the present application includes the steps of: S1: in response to satisfying the first preset condition, dividing the visual partitions of the screen according to the sight focus range; S2: updating the display data of each visual partition according to the preset refresh frame rate. The technical solution of the present application can determine the visual partitions in combination with the sight capture function, and support displaying screen data at different refresh frame rates for different visual partitions, which is more intelligent and / or power-saving, and provides a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0038] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application;

[0039] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0040] Figure 3 It is a flow chart of the control method shown in the embodiment of the present application;

[0041] Figure 4 is a schematic diagram of pupil changes shown in an embodiment of the present application;

[0042] Figure 5 is a schematic diagram of a calibration rule screen shown in an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of visual partitioning shown in an embodiment of the present application;

[0044] Figure 7 It is a schematic diagram of updating data in each visual area according to the frame rate shown in the embodiment of the present application;

[0045] Figure 8This is a schematic diagram of the global update of the screen off state shown in an embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0049] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0050] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0051] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0052] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0053] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0055] The smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0056] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0057] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0058] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0059] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution), 5G and 6G, etc.

[0060] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0061] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0062] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0063] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0064] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0065] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0066] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0067] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0068] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0069] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0070] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0071] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0072] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0073] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0074] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0075] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0076] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0077] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0078] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation here.

[0079] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0080] Embodiment 1

[0081] See also Figure 3 , which is a flow chart of a control method for implementing an embodiment of the present application. The control method can be applied to a smart terminal, comprising the steps of:

[0082] S1: In response to satisfying a first preset condition, dividing the visual partitions of the screen according to the sight focus range;

[0083] S2: Update the display data of each visual partition according to the preset refresh frame rate.

[0084] Optionally, when dividing the visual partitions of the screen according to the sight line focus range in step S1, the visual partitions of the screen are divided based on the sight line focus range.

[0085] Optionally, step S1 comprises the steps of:

[0086] S11: in response to the screen state satisfying the first preset condition, acquiring image information within the field of view angle;

[0087] S12: capturing human eye information on the image information to obtain a sight focus range;

[0088] S13: Divide the visual partitions of the screen according to the sight focus range.

[0089] Optionally, satisfying the first preset condition includes at least one of the following:

[0090] The screen is on;

[0091] The screen is in use;

[0092] The camera’s field of view contains facial information;

[0093] The camera's field of view contains information about the human eye.

[0094] Optionally, the camera may be a front camera, or a camera of an associated device.

[0095] Optionally, the associated device may be any device connected to the smart terminal, or may be any device with the same login account or user as the smart terminal.

[0096] Optionally, the associated device may be in the form of a mobile phone, tablet, television, car, etc.

[0097] Optionally, the working state of the screen includes at least one of the following: a screen-on state, a screen-in-use state, and a screen-off state.

[0098] In one embodiment, when the smart terminal enters the camera application to shoot, the captured image information will be rendered, and the power consumption of the rendering process is relatively high. However, in the technical solution of the embodiment of the present application, the smart terminal does not enter the camera application, and can directly capture the image information of the preset structure or form when the screen is on or in use, thereby lowering the power consumption.

[0099] Optionally, the camera is used to capture image information of a preset structure or form, such as a diagram of the structure of a human eye (eg, white of the eye, black of the eye), so as to achieve the purpose of eye tracking.

[0100] Optionally, step S12 includes: capturing human eye information in the image information based on the structure of the human eye, and determining the line of sight focusing range by utilizing the positional relationship between the human eye information and the screen.

[0101] Optionally, step S12 includes:

[0102] Capture the first, second and third features of human eye information;

[0103] Acquire first position information of the first feature relative to the screen;

[0104] Determine the eye sight direction, where the eye sight direction is obtained based on the relative position relationship between the second feature and the third feature;

[0105] The sight focusing range is determined, where the sight focusing range is obtained based on the eye sight direction and the first position information.

[0106] For example, the first feature is the eyeball, the second feature is the pupil, and the third feature is Purkinje's spot.

[0107] The technical solution of this embodiment can use eye tracking technology to identify human eye features. For example, the recognition technology based on pupil-corneal reflection spot, please refer to Figure 4 , which is a schematic diagram of pupil changes in an embodiment of the present application. The light entering the pupil is reflected on the outer surface of the cornea to form a bright spot, which is regarded as a Purkinje spot on the outer surface of the cornea of ​​the eyeball. The absolute position of the Purkinje spot does not change with the rotation of the eyeball.

[0108] Optionally, see Figure 5 , which is a screen diagram of the calibration rule for implementing an embodiment of the present application. Based on specific points, eye calibration can be performed using a 6-point calibration rule to obtain first position information of the eye relative to the screen. Based on the positional relationship between the pupil and the Purkinje spot, the reflection vector of the cornea is calculated, a stereoscopic geometric model is established using the reflection vector, and the eye line of sight direction is estimated through the stereoscopic geometric model. According to the eye line of sight direction and the first position information (i.e., the position of the eye relative to the screen), the current line of sight focus range (i.e., the range of eye gaze) is determined.

[0109] Optionally, the eye tracking method may be at least one of the following:

[0110] (1) Pupillary corneal reflection method, which tracks eye movements through the line connecting the center of the cornea and the center of the pupil;

[0111] (2) Retinal imaging, based on the irregular patterns of capillaries, fovea and other physiological structures on the retina, eye tracking is performed by calculating changes in retinal images;

[0112] (3) After modeling the eye, the visual center is calculated, and the eye is modeled using structured light, or the eye is modeled using a light field camera;

[0113] (4) Retinal reflected light intensity: The laser passes through the cornea, pupil, and lens and finally hits the retina. The retina reflects the incident laser. The external photosensor determines the direction of eye movement by detecting the intensity of the retinal reflected light.

[0114] (5) The intensity of light reflected from the cornea. The outer wall of the cornea is relatively smooth. When a laser beam is emitted from one side of the eye to the cornea, it will be reflected by the cornea. When the laser beam hits the center of the cornea (the center of the cornea is the highest point), the intensity of the laser reflection is the highest. The direction of eye movement is determined by detecting the intensity of the light reflected from the cornea through a photosensor.

[0115] In one embodiment, the visual partition of the screen includes at least one of a first visual area, a second visual area, and a third visual area.

[0116] Optionally, the preset refresh frame rate includes at least one of a first refresh frame rate, a second refresh frame rate and a third refresh frame rate.

[0117] Optionally, the control method further includes at least one of the following:

[0118] The second visual area includes the first visual area;

[0119] The third visual area includes the first visual area and the second visual area;

[0120] The first visual area is refreshed using a first refresh frame rate;

[0121] The second visual area is refreshed using a second refresh frame rate;

[0122] The third visual area is refreshed using a third refresh frame rate;

[0123] The first refresh frame rate is higher than the second refresh frame rate;

[0124] The second refresh frame rate is higher than the third refresh frame rate.

[0125] Optionally, the third visual area includes the first visual area and the second visual area.

[0126] Optionally, the second visual area includes the first visual area.

[0127] See also Figure 6 , which is a schematic diagram of visual partitioning for implementing the embodiment of the present application. The smart terminal tracks and captures the sight focus position through the camera. When the sight focus range (the range is an irregular range) is captured, the sight focus range is further divided into regions, including the first visual area within the sight focus range where the screen needs to be refreshed high, the third visual area outside the sight focus range where the screen needs to be refreshed low, and the second visual area that needs to be transitioned between the first visual area and the third visual area.

[0128] Optionally, the shape of the visual partition may be a rectangle, a square, or the like.

[0129] Optionally, a preset first sub-frame rate, a second sub-frame rate, and a third sub-frame rate are allocated to each visual partition.

[0130] Optionally, the first subframe rate is higher than the second subframe rate.

[0131] Optionally, the second subframe rate is higher than the third subframe rate.

[0132] Optionally, the first sub-frame rate is allocated to the first visual area.

[0133] Optionally, a second sub-frame rate is allocated to a second visual area.

[0134] Optionally, a third subframe rate is allocated to the third visual area.

[0135] See also Figure 7 , which is a schematic diagram of updating data according to the frame rate in each visual area of ​​the embodiment of the present application. This embodiment pre-configures the refresh frame rate for each visual partition with unknown position: A represents the first visual area, B represents the third visual area, and C represents the second visual area. Taking 120HZ-1HZ as an example, the first sub-frame rate of high refresh is 120Hz, the third sub-frame rate of low refresh is 1Hz, and the second sub-frame rate of transition is 60Hz. Area A is always updated and maintained at 120HZ refresh, while areas B and C execute the LTPO refresh rate reduction solution, such as 60HZ or 1HZ, during the time period when no data is transmitted.

[0136] Optionally, the control method also includes: determining the visual partitions that need to be updated (for example, which visual partitions need to be updated and when which visual partitions need to be updated), and confirming the transmission status of the display data of the first visual area, the second visual area, and the third visual area, and displaying the refresh effect of each visual partition after receiving the display data.

[0137] In one embodiment, the embodiment of the present application further includes: in response to satisfying the second preset condition, updating the display data using a fourth refresh frame rate. Further, the fourth refresh frame rate is allocated to the entire screen so that the screen updates the display data according to the fourth refresh frame rate.

[0138] See also Figure 8 , which is a schematic diagram for implementing the global update of the screen-off state in the embodiment of the present application. The second preset condition is that the screen is in the screen-off state, at which time the smart terminal is in the standby state, and based on the preset fourth refresh frame rate, for example, 1HZ, the screen globally updates the display data at a refresh frame rate of 1HZ.

[0139] The technical solution of the embodiment of the present application can be combined with the line of sight capture function to determine the visual partition, and support displaying screen data at different refresh frame rates for different visual partitions, which is more intelligent and / or power-saving and provides a better user experience.

[0140] The control method provided by the present application adopts the line of sight tracking technology, tracks the position of the eyeball through the camera, determines the line of sight focusing range, constitutes a new type of human-computer interaction design, and brings users a novel experience; dynamically adjusts the visual partition according to the line of sight focusing range (i.e., the focus range of the eyeball), defines the line of sight focusing range as a high refresh rate, and defines the line of sight focusing range as a low refresh rate, and in order to reduce the visual sense of separation between adjacent areas, an intermediate transition zone is added; visual partitions are used to refresh the display data, thereby reducing the power consumption of the screen and the whole machine; due to the partition processing, the power consumption is greatly reduced, and an ultra-high refresh rate of 144HZ or even 240HZ or more can be achieved within the line of sight focusing range, and it can be reduced to 1HZ outside the focus, which is adapted to the focus range of the human eye, and the ultra-high refresh rate is maintained to enhance the visual effect.

[0141] The present application also provides an intelligent terminal, including a memory and a processor. The memory stores a control program, and when the control program is executed by the processor, the steps of the control method in any of the above embodiments are implemented.

[0142] The present application also provides a computer-readable storage medium, on which a control program is stored. When the control program is executed by a processor, the steps of the control method in any of the above embodiments are implemented.

[0143] In the embodiments of the smart terminal and computer-readable storage medium provided in the present application, all technical features of any of the above-mentioned control method embodiments may be included, and the expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods, and will not be repeated here.

[0144] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0145] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0146] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0147] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0148] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0149] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0150] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0151] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0154] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0155] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control method, characterized in that: Includes steps: S1: In response to satisfying a first preset condition, dividing the visual partitions of the screen according to the sight focus range; S2: Update the display data of each visual partition according to the preset refresh frame rate.

2. The control method according to claim 1, characterized in that: Step S1, comprising the steps of: S11: in response to the screen state satisfying the first preset condition, acquiring image information within the field of view angle; S12: capturing human eye information on the image information to obtain a sight focus range; S13: Divide the visual partitions of the screen according to the sight focus range.

3. The control method according to claim 2, characterized in that: Step S12 includes: The human eye information in the image information is captured based on the structure of the human eye, and the visual focus range is determined using the positional relationship between the human eye information and the screen.

4. The control method according to claim 3, characterized in that: Step S12 includes: The line of sight focusing range is determined according to the first position information of the first feature of the human eye information relative to the screen and the eye line of sight direction determined by the relative position relationship between the second feature and the third feature of the human eye information.

5. The control method according to any one of claims 1 to 4, characterized in that: The first precondition is met, including at least one of the following: The screen is on; The screen is in use; The camera’s field of view contains facial information; The camera's field of view contains information about the human eye.

6. The control method according to any one of claims 1 to 4, characterized in that: The visual partitioning of the screen includes at least one of a first visual area, a second visual area and a third visual area; and / or the preset refresh frame rate includes at least one of a first refresh frame rate, a second refresh frame rate and a third refresh frame rate.

7. The control method according to claim 6, characterized in that: Also includes at least one of the following: The second visual area includes the first visual area; The third visual area includes the first visual area and the second visual area; The first visual area is refreshed using a first refresh frame rate; The second visual area is refreshed using a second refresh frame rate; The third visual area is refreshed using a third refresh frame rate; The first refresh frame rate is higher than the second refresh frame rate; The second refresh frame rate is higher than the third refresh frame rate.

8. The control method according to any one of claims 1 to 4, characterized in that: Also includes: In response to satisfying the second preset condition, the display data is updated using a fourth refresh frame rate.

9. An intelligent terminal, characterized in that: include: A memory and a processor, wherein a control program is stored in the memory, and when the control program is executed by the processor, the steps of the control method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the control method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Catalytic dehydrohalogenation process

    EP0002021A1