Method and device for saving energy

Abstract

Mobile phone comprising: a camera (20); a display (30); a first logic for detecting a user's eye via the camera (20) and for taking a power reduction action for the display (30) of the mobile phone if the logic determines that the user's eye is not viewing the display (30); and a second logic for distinguishing between an active touch and an inactive touch of the display (30) and for taking a first action for the display (30) when the touch is inactive and for taking a second action for the display (30) when the touch is active.

Description

Technical field

[0001] The present disclosure relates generally to data processing systems. More particularly, the present disclosure relates to conserving energy in a data processing system. background

[0002] For the purposes of this disclosure, the noun "display" refers to the hardware component that a computing system uses to present visual information to a person (the "user"). When a conventional computing system is turned on ("booted up"), the display is turned on regardless of whether the user is actively interacting with the computing system. The power or energy consumed by the display when it is turned on may be wasted in cases where the user does not need the display to be turned on. For example, the user may leave the area where the computing system is located, and the display may remain on unless the user specifically instructs the computing system to turn off the display. For the purposes of this disclosure, the term "turn off" refers to turning off the power.

[0003] With the advent of mobile computing systems such as smartphones, laptops, and other devices that rely on battery power, power conservation has become even more desirable. A user can conserve power by disabling certain features, such as a module that scans for wireless connections; components that enable various communication channels, such as the protocol known under the brand BLUETOOTH; and so-called third-generation (3G) and fourth-generation (4G) or other high-speed Internet connections; vibrating alerts; multiple email accounts; options for performing automatic email or web browser updates; and so on. A user can turn off the entire device when not in use or, if available, put it into a mode that suspends signal transmission functions (“airplane mode” or “offline mode”).A user can close applications that may be running in the background or avoid applications that require large amounts of power, such as games or music applications. A user can also save power consumed only by the display by reducing the display brightness, setting a timeout function to a low value that causes the display to turn off quickly after the user stops providing input, avoiding screen savers, or manually turning off the display. Energy storage solutions can also be implemented by developers. For example, a developer can create a software application that turns off the display when certain activities occur, such as answering a phone call on a smartphone. A developer can also create technology that reduces the amount of power consumed by a display.For example, some liquid crystal displays (LCDs) use display processing technology marketed under the PIXCALE brand to reduce power consumption. A company called Unipixel has also proposed a certain display technology that claims to consume less power than conventional displays. However, if a user interacts with a device intermittently, energy can still be wasted by leaving the display on when it's not necessary.

[0004] US 2004 / 0175020 A1 describes a method and apparatus comprising an image capture device connected to a computer system, wherein the image captured by the image capture device is analyzed using a face tracking technique to learn the user's behavior.

[0005] US 2004 / 0073827 A1 describes a computer that is put into power saving mode by detecting the presence of a user, where a camera is connected to the computer and the output of the camera is analyzed to determine whether the user is present.

[0006] US 2009 / 0082066 A1 describes a portable electronic device with automatic control to keep the device switched on. Short description of the drawings

[0007] The features and advantages of the present invention will become apparent from the appended claims, the following detailed description of one or more exemplary embodiments and the corresponding figures, wherein: Fig. 1 is a block diagram of the data processing system with a low-power eye detection service according to at least one exemplary embodiment, Fig. 2 illustrates a flowchart of a process for turning off a display when a user is not looking at the display or actively using the display, according to at least one exemplary embodiment, and Fig. 3A and Fig. 3B are schematic diagrams illustrating the use of a camera to detect whether a user is looking at the display, according to at least one example embodiment. Description of the embodiments

[0008] The invention provides a mobile telephone according to claim 1, a data processing system according to claim 9, and a method according to claim 16; the dependent claims describe embodiments of the invention. It is an object of the invention to improve the decision regarding the implementation of a power reduction measure of a device.

[0009] Turning off a display is useful for saving energy and can extend a device's battery life. When a user is interacting with a device, such as a mobile smartphone, the display may be turned on by default. When the user no longer interacts with the device in a way that requires the display to be turned on, the display may remain turned on for a predetermined period of inactivity unless the user or a software application specifically instructs the display to turn off. Allowing the display to remain turned on when not needed reduces energy efficiency and can decrease a device's battery life.

[0010] According to the present disclosure, in at least one embodiment, a power-saving eye detection service uses a camera to detect whether a user is looking at the display during periods of inactivity to determine whether the display should remain on. When the user is not looking at the display, the eye detection service instructs the display to turn off, thereby saving power and potentially extending the device's battery life.

[0011] Fig. 1 is a block diagram of data processing system 8 having an eye detection service 62 according to an exemplary embodiment. In the illustrated embodiment, the data processing system is a device 10 having at least a central processing unit (CPU) or processor 50, as well as data storage 60 and a display 30 responsive to the processor. The display may include a touch sensor 32 covering part or all of the display area. The touch sensor may serve as an input interface. As noted below, the device may also include other input devices. In the illustrated embodiment, the eye detection service is implemented as software stored in data storage and executed by the processor to provide the functions described below.An OS 65, a context framework 63, a computer vision library 64, and one or more applications 61 are also stored in memory, and when these components execute on the processor, they can invoke the eye detection service. The context framework may also include an eye sensor 66. The processor may also be connected to a camera 20 and one or more peripheral devices 40 (e.g., a keyboard or mouse).

[0012] In the embodiment of Fig. 1, the context framework can describe a framework that enables the development of context-aware applications. For example, a device with a context framework can use Intel® Context Aware Computing Technology to determine a user's individual needs and personalize the user experience through applications and services. More information about Intel®'s Context Aware Computing Technology can be found at http: / / techresearch.intel.com / ResearchAreaDetails.aspx?Id=37.

[0013] Fig. 1 also refers to a computer vision library located in the data store. This may refer to an Open Computer Vision (OpenCV) library that contains program functions directed towards real-time computer vision. Computer vision (CV) describes the procedures for acquiring, processing, analyzing, and understanding images to produce numerical or symbolic information. For example, a computer may be able to acquire images of a person in real time, and it may be able to determine whether there is a human eye in the images, and also whether that eye is moving. Further information on the OpenCV library can be found at http: / / opencv.willowgarage.com / wiki / . As described below with respect to Fig. 2, the eye detection service may use information obtained from the CV library to determine whether the user is looking at the display. For the purposes of this disclosure, information reported by the CV library may be referred to as "vision results."

[0014] In one embodiment, the eye detection service may be implemented as an OS service. In another embodiment, the eye detection service may be a background application. The eye detection service may interact with the context framework. For example, when a user is not actively touching a user interface of the device, as described below, the eye detection service may connect to the context framework to detect whether the user is looking at the display. The context framework may load an eye sensor. The eye sensor may turn on a camera and use vision results from the CV library to determine whether the user is looking at the display, and the eye sensor may generate eye sensor data indicating whether the user is looking at the display based on the vision result from the CV library. The eye detection service may then receive the eye sensor data from the eye sensor.The eye sensor data can be in a simple format such as “. <lookingatscreen> false < / lookingatscreen> In response to the eye sensor data, the eye detection service may turn off the display if it detects that the user is not looking at the display. If it detects that the user is looking at the display, the eye detection service may leave the display on. If the user is actively touching a user interface of the device while the eye detection service is determining whether the user is looking at the display, the eye detection service may receive touch sensor data indicating the active touching and suspend itself.

[0015] Different programming languages or combinations of programming languages, etc., can be used to implement the eye detection service, context framework, and CV library in different environments. For example, a combination of C and C++ can be used, as well as Java or JavaScript. Implementation details may also differ for implementations intended for different platforms or devices, such as devices using the Android OS, devices using the iPhone OS, devices using the Windows CE OS, etc.

[0016] Fig. Figure 2 illustrates a flow diagram of a process for turning off a display when a user is not looking at the display or actively using the display, according to at least one example embodiment. The illustrated process may include a device such as the one shown in Fig. 1, and the process may begin with the display turned on. As illustrated at block 100, the eye detection service determines whether the user is actively using or actively interacting with the device, either by actively touching a user interface of the device or other use that indicates active interaction or use. The device may be configured to interpret various types of activity for indications of whether the user is actively interacting with a device. For example, if the device is a smartphone, the display may have a touch sensor integrated into the display. The touch sensor may be capable of distinguishing between active touching, such as entering data into an input interface (e.g., an on-screen keyboard), and inactive touching, such as the user's cheek touching the screen when using the smartphone for phone calls.If the device detects only inactive touching, the device may determine that the user is not actively using or interacting with the device. Additionally, the device may be used in other ways to indicate active interaction. In one or more embodiments, the types of activities that the device may interpret as active interaction may include, for example, without limitation, typing on a peripheral keyboard, moving a mouse, providing audio input via a microphone, providing input via a game controller, and displaying a presentation or video on a second display or projector.

[0017] When the eye detection service receives touch sensor data or other data indicating active interaction, the eye detection sensor may suspend itself, as illustrated at block 102, so that the eye detection service does not continuously monitor the eyes but simply waits for inactivity. If there is no active interaction, the context frame loads an eye sensor, which turns on the camera, as illustrated at blocks 106 and 108. As a result, the eye detection service is no longer suspended, as indicated by dashed box 107, but connects to the context frame to retrieve eye sensor data from the eye sensor and provides power management operations for the display, as indicated below. As illustrated in block 110, after the camera is turned on, the eye detection service determines whether the user is looking at the display.For example, as noted above, the eye detection service may use eye sensor data from the eye sensor to determine whether the user is looking at the display. In one embodiment, when the eye sensor detects a face in front of the camera (based on the vision results from the CV library), the eye sensor filters at least one of the eyes as points. The eye sensor then returns the corresponding eye sensor data to the caller, the eye detection service.

[0018] In one embodiment, the eye detection service is configured to determine whether the user is looking at the display in response to receiving eye sensor data indicating detection of a face facing the display. Alternatively, the eye detection service may be configured to determine whether the user is looking at the display in response to receiving eye sensor data indicating detection of at least one eye within the camera's field of view facing the display. Alternatively, the eye detection service may determine whether the user is looking at the display in response to receiving eye sensor data indicating detection of a pair of eyes in a face within the camera's field of view. In another embodiment, the eye detection service may interpret eye sensor data resulting from certain combinations of the vision results that indicate that the user is looking at the display.For example, in some situations (e.g., if the display has been turned off and the user subsequently returns to the computer), facial recognition alone, without any indication of the direction the face is facing, may determine that the user is looking at the display. The eye detection service may also be configured to determine whether the user is looking at the display in response to eye sensor data indicating that an eye is detected, even if no face is detected (to account for the possibility that facial recognition may have failed due to, for example, the angle of the user's face).

[0019] In response to a determination that the user is looking at the display, the eye detection service may keep the display on, as indicated by the "Yes" arrow in block 110. In response to a determination that the user is not looking at the display, the eye detection service may automatically turn off the display, as illustrated at block 112. As illustrated at block 114, the eye detection service may then also start a sleep timer. While the sleep timer is running, the eye detection service may continue to operate to determine whether the user returns their gaze to the display, as illustrated at block 120. If the user returns their gaze to the display before a predetermined inactivity threshold is reached, the eye detection service may automatically turn the display back on, as illustrated at block 122.In one embodiment, the eye detection service may cause the display to turn off within one second of detecting that the user is not looking at the display, and the eye detection service may cause the display to turn on within one second of detecting that the user is looking at the display again. In other embodiments, the eye detection service may implement other response times for turning off the display, for example, without limitation, turn-off times of less than or equal to 2000 milliseconds or greater than or equal to 300 milliseconds. In one embodiment, the user may configure the turn-off time to be greater than or equal to the minimum time. Similarly, in some embodiments, the eye detection service may implement turn-on times of less than or equal to 800 milliseconds or greater than or equal to 100 milliseconds.In one embodiment, the user may configure the on-time to a value greater than or equal to the minimum time. Furthermore, various embodiments may use various combinations of off-times and on-times, including, without limitation, any combination of the off-times and on-times described above.

[0020] When the eye detection service turns the display back on, the eye detection service may also clear the sleep timer, as illustrated in block 124. The eye detection service may then continue to operate to determine whether the user continues to look at the display, as indicated by the arrow returning from block 124 to block 110.

[0021] However, referring again to 120, if the user is not looking at the display after the sleep timer has started, the eye detection service may determine whether the sleep timer equals or exceeds a predetermined threshold, as illustrated at block 130. As a result, if the user is inactive for a period of time greater than or equal to the predetermined inactivity threshold, as measured by the sleep timer, the eye detection service may then turn off the camera to further reduce power consumption, as illustrated at block 132. The eye detection service may then suspend itself, as illustrated at block 134.

[0022] Subsequently, as illustrated at block 136, the OS may receive a wake event, for example, in response to user interaction with a mouse or keyboard. In response to the wake event, the OS may turn on the display, as illustrated in block 138. Wake events may be the same as or similar to those that wake a device from "sleep" mode, and in one embodiment, the types of activities that constitute a wake event may be specified by the user. The process may then return to block 100, where the device determines whether the user is actively interacting with the device or looking at the display and responds accordingly, as described above. Thus, the eye detection service is active (not suspended) for the operations illustrated in dashed box 107.

[0023] In another embodiment, an application may override the eye detection service if the application requires the display to remain in a particular state, i.e., on or off. For example, a video player may suspend the eye detection service while a video is playing to avoid turning off the display at an inappropriate time. Similarly, a service that downloads a file may turn on the display in response to the download completing. Furthermore, these and other types of turning on and off operations may be controlled by settings that can be changed by an application, user, administrator, etc.

[0024] Fig. 3A and Fig. 3B illustrate the use of a camera to detect whether a user is looking at the display. In the illustrated embodiment, the eye detection service 62 in device 10 may be Fig. 1 use the camera 20 to determine whether the user 80 is looking at the display 30. As in Fig. 3A, a dashed line 200 indicates that the eye detection service may determine whether the user is looking at the display based on vision results such as those described above. For example, the eye detection service may determine that the user is looking at the display based on vision results from the CV library indicating that the user's face was detected within the camera's field of view and facing the display. The eye detection service may also determine that the user is looking at the display based on vision results indicating that the CV library detected at least one eye 90 within the camera's field of view. In response to a determination that the user is looking at the display, the eye detection service may turn on the display, as indicated above.

[0025] On the other hand, as in Fig. 3B, dashed line 300 indicates that the eye detection service cannot detect a user looking at the display. For example, as indicated above, the eye detection service may determine that no user is looking at the display after receiving eye sensor data indicating that the CV library could not detect the presence of a human face or a human eye within a human face in the camera's field of view. In response to determining that no user is looking at the display, the eye detection service may turn off the display, as indicated above.

[0026] In view of the principles and exemplary embodiments described and illustrated herein, it is to be understood that the illustrated embodiments may be modified in arrangement and details without departing from such principles. Furthermore, the foregoing discussion has focused on particular embodiments, but other configurations are contemplated. Furthermore, although terms such as "in one embodiment," "in another embodiment," or the like are used herein, these terms are intended as general referents of embodiments and are not intended to limit the invention to particular embodiment configurations. As used herein, these terms may refer to the same or different embodiments that may be combined in other embodiments.

[0027] As used herein, the terms "processing system" and "data processing system" are intended to encompass, in the broadest sense, a single machine or a system of communicatively coupled machines or devices operating together. Exemplary processing systems include, without limitation, distributed computing systems, supercomputers, high-performance computing systems, computer clusters, mainframe computers, minicomputers, client-server systems, personal computers (PCs), workstations, servers, portable computers, laptop computers, tablet computers, personal digital assistants (PDAs), telephones, handheld devices, entertainment devices such as audio and / or video devices, and other devices for processing or transmitting information.

[0028] Furthermore, components described as coupled, communicating, responsive, or similar do not necessarily have to be in constant communication with each other or directly coupled to each other unless explicitly stated otherwise. Furthermore, some components of the data processing system may be implemented as adapter cards with interfaces (e.g., a connector) for communicating with a bus. Alternatively, devices may be implemented as embedded controllers using components such as programmable or non-programmable logic devices, application-specific integrated circuits (ASICs), embedded computers, smart cards, and the like.

[0029] It is also to be understood that the hardware and software components illustrated herein represent functional elements that are reasonably self-contained such that each may be designed, constructed, or updated substantially independently of the others. In alternative embodiments, many of the components may be hardware, software, or combinations of hardware and software for providing the functions described and illustrated herein. For example, alternative embodiments include machine-accessible media encryption instructions or control logic for performing the operations of the invention. Such embodiments may also be referred to as program products. Such machine-accessible media may be, for example, without limitation, tangible storage media such as magnetic disks, optical disks, RAM, ROM, flash memory, etc.Alternatively, some or all of the control logic for implementing the described operations may be implemented in hardware logic (e.g., as part of an integrated circuit chip, a programmable gate array (PGA), an ASIC, etc.). Instructions may also be used in a distributed environment and stored locally and / or remotely for access by single- or multi-processor machines.

[0030] Furthermore, although one or more example processes have been described with respect to particular operations performed in a particular order, numerous modifications could be applied to these processes to derive numerous alternative embodiments of the present invention. For example, alternative embodiments may include processes that utilize fewer than all of the disclosed operations, processes that utilize additional operations, and processes in which the individual operations disclosed herein are combined, split, rearranged, or otherwise altered.

[0031] Given the multitude of useful permutations that can be readily derived from the exemplary embodiments described herein, this detailed description is intended for illustrative purposes only and should not be considered limiting the scope of the invention. Therefore, the invention is claimed to include all implementations falling within the scope of the following claims and all equivalents of such implementations.

[0032] Embodiments may include, without limitation, some or all of the features described in the following paragraphs.

[0033] A method for controlling a display of a data processing system may include using a camera of the data processing system to automatically determine whether a user of the data processing system is viewing a display of the data processing system. In response to determining that the user is not viewing the display, the display may be automatically turned off.

[0034] A method for controlling a display of a data processing system may include automatically determining whether a user is actively using the data processing system before using a camera to automatically determine whether the user is viewing the display. In response to determining that the user is not actively using the data processing system, the camera may be automatically turned on.

[0035] A method for determining whether a user is actively using a computing system may include determining whether the user is actively touching an input interface. The input interface may include a touch sensor integrated into the display of the computing system.

[0036] A method for controlling a display of a data processing system may include using a camera of the data processing system to automatically determine whether a user continues viewing the display after the display has been automatically turned off. In response to determining that the user has not continued viewing the display, the display may be automatically turned off.

[0037] A method for controlling a display of a data processing system may include determining whether a user is viewing the display by detecting the presence of the user's face within the field of view of a camera of the data processing system. Determining whether the user is viewing the display may include determining the direction in which the user's face is pointing. In response to determining that the user's face is pointing at the display, determining whether the user is viewing the display may be performed.

[0038] A method for controlling a display of a data processing system may include determining whether a user is viewing the display by detecting the presence of at least one of the user's eyes within the field of view of a camera of the data processing system. In response to detecting at least one of the user's eyes within the field of view of the camera, determining that the user is viewing the display may be performed.

[0039] A method for controlling a display of a computing system may include determining whether a user has actively used the computing system for at least a predetermined period of time after the display has been automatically turned off. In response to a determination that the user has not actively used the computing system for at least the predetermined period of time, a camera of the computing system may be automatically turned off. After the display has been automatically turned off in response to the determination that the user has not actively used the computing system for at least the predetermined period of time, it may be automatically determined whether the user is actively using the computing system. In response to the determination that the user is actively using the computing system, the display may be automatically turned on.

[0040] At least one non-transitory machine-accessible medium may include instructions that, when executed by a data processing system, enable a data processing system to perform the methods for controlling a display of the data processing system, as described above.

[0041] A data processing system may include at least one non-transitory machine-accessible medium responsive to the processor and instructions in the machine-accessible medium that, when executed by the data processing system, enable the data processing system to perform the methods for controlling a display of the data processing system, as described above.

Claims

[1] A mobile phone comprising: a camera (20); a display (30); a first logic for detecting a user's eye via the camera (20) and for taking a power reduction action for the display (30) of the mobile phone if the logic determines that the user's eye is not viewing the display (30); and a second logic for distinguishing between an active touch and an inactive touch of the display (30) and for taking a first action for the display (30) when the touch is inactive and for taking a second action for the display (30) when the touch is active. [2] A mobile phone according to claim 1, wherein the first measure consists in turning off the display (30) and wherein the second measure consists in turning on the display (30). [3] A mobile phone according to claim 1, wherein the first logic is to take the action automatically when the user's eye is not looking into the camera (20). [4] A mobile phone according to claim 1, wherein the first and second logic are part of an operating system. [5] A mobile phone according to claim 1, wherein the display (30) comprises a touch sensor. [6] A mobile telephone according to claim 1, comprising a third logic which deactivates the first logic. [7] A mobile phone according to claim 1, comprising a working memory communicatively coupled to the camera (20). [8] The mobile phone of claim 1, wherein the first logic is to increase the power consumption of the mobile device when the first logic detects that the user's eye is viewing the display (30). [9] A data processing system (8) comprising: a processor (50); and a memory coupled to the processor (50), the memory having instructions that, when executed, cause the processor (50) to: to capture an eye of a user via a camera (20); to take a power reduction action for a display (30) of a mobile phone if the logic determines that the user's eye is not viewing the display (30); and to distinguish between an active touch and an inactive touch of the display (30) and to take a first action for the display (30) when the touch is inactive and to take a second action for the display (30) when the touch is active. [10] Data processing system (8) according to claim 9, wherein the first measure consists in switching off the display (30) and wherein the second measure consists in switching on the display (30). [11] Data processing system (8) according to claim 9, wherein the first logic is to perform the action automatically when the user's eye is not looking into the camera (20). [12] Data processing system (8) according to claim 9, wherein the first and second logic are part of an operating system and wherein the display (30) comprises a touch sensor. [13] Data processing system (8) according to claim 9, comprising a third logic which deactivates the first logic. [14] Data processing system (8) according to claim 9, wherein the main memory is communicatively coupled to the camera (20). [15] The data processing system (8) of claim 9, wherein the first logic is to increase the power consumption of the mobile device when the first logic determines that the user's eye is viewing the display (30). [16] Method comprising: detecting a user's eye via a camera (20); Taking a power reduction measure for a display (30) of a mobile phone when it is determined that the user's eye is not viewing the display (30); and Distinguishing between an active and an inactive touch of the display (30); taking a first action for the display (30) when the touch is inactive; and Taking a second action for the display (30) when the touch is active. [17] The method of claim 16, wherein taking the first action comprises turning off the display (30), and wherein taking the second action comprises turning on the display (30). [18] The method of claim 16, wherein taking the first action comprises automatically taking the power reduction action when the user's eye is not looking into the camera (20). [19] The method of claim 16, wherein the first and second logic are part of an operating system, wherein the display (30) comprises a touch sensor, and wherein the camera (20) is communicatively coupled to a main memory. [20] The method of claim 16, comprising increasing the power consumption of the mobile device when the first logic determines that the user's eye is viewing the display (30).

Images