ELECTRONIC DEVICES WITH PROXIMITY SENSORS

Abstract

An electronic device may include a proximity sensor and a display. Specifically, the display may be deactivated when the device is near a user's head. However, the proximity sensor may operate through or near the display and detect the user's finger. To distinguish between the user's finger and head, a motion sensor, such as an inertial measurement unit (IMU), may be used. For example, the IMU may measure the device's position, acceleration, and rotational speed, and a machine learning algorithm may use these values ​​as inputs to determine whether the proximity sensor is triggered by the user's finger or head. Based on the determination that the proximity was triggered by the head, the display may be deactivated.Alternatively, the display can remain on in response to the determination that the proximity sensor was triggered by the finger.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 671,116, filed May 22, 2024, which claims the benefit of U.S. Provisional Patent Application No. 63 / 581,534, filed September 8, 2023, which are hereby incorporated by reference in their entirety. AREA

[0002] This refers generally to electronic devices, including electronic devices with sensors. STATE OF THE ART

[0003] Electronic devices such as laptop computers, mobile phones, and other equipment are sometimes provided with various components. The components can be adjusted based on sensor measurements. SUMMARY

[0004] An electronic device may include a proximity sensor and a display. In particular, it may be desirable for the display to be deactivated when the device is near a user's head (e.g., when the user holds the device to their head to make a phone call) to prevent inaccurate inputs on the display. However, the proximity sensor may operate through or near the display and may also be triggered when the user interacts with the display with their finger (or other external object, such as a stylus).

[0005] To distinguish between the user's head and finger, a motion sensor, such as an inertial measurement unit (IMU), can be used. For example, the position, acceleration, and rotational velocity of the device can be measured by the IMU. A machine learning algorithm can use the position, acceleration, and rotational velocity as inputs to determine whether the proximity sensor is triggered by the user's finger or head.

[0006] In response to determining that proximity was triggered by the user's head, the display may be deactivated, for example, by deactivating a touch sensor and / or a pixel array of the display. Alternatively, in response to determining that the proximity sensor was triggered by the finger, the display (e.g., the touch sensor and / or the pixel array) may remain on.

[0007] Other sensor information, such as ambient light measurements and / or touch sensor measurements, may also be used to determine whether the proximity sensor was triggered by the user's finger (or other external object) or the user's head. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic diagram of an illustrative electronic device with a display and sensor components according to some embodiments. Fig. 2 is a perspective view of an electronic device with a display and a proximity sensor according to some embodiments. Fig. 3 is a side view of an electronic device having a proximity sensor operating through a display layer, according to some embodiments. Fig. 4A-4C are diagrams of illustrative relationships between the position, acceleration, and rotational speed of an electronic device over time when a proximity sensor is triggered by a user's finger and when the proximity sensor is triggered by the user's head, according to some embodiments. Fig. 5 is a diagram of illustrative proximity and motion sensor components used to determine whether the proximity sensor is triggered by a finger or the head, according to some embodiments. Fig. 6 is a diagram of illustrative steps that may be used to determine whether the proximity sensor is triggered by a finger or the head, according to some embodiments. Fig. 7 is a flowchart of illustrative method steps that may be used to determine whether the proximity sensor is triggered by a finger or the head, according to some embodiments. DETAILED DESCRIPTION

[0008] Electronic devices such as mobile phones, tablets, and wearable devices may have a display. It may be desirable to incorporate a proximity sensor into the electronic devices to detect when a user brings the device to their head (e.g., to make a call) and to turn off / deactivate the display when the device is brought to the user's head. However, the proximity sensor may be located near or below the display. This allows the proximity sensor to detect not only when the device is against the user's head, but also when an external object, such as the user's finger, is near the proximity sensor (e.g., when the user interacts with the display). Therefore, it may be desirable to determine whether the proximity sensor is triggered by the user's head or by a finger or other external object.

[0009] To determine whether the proximity sensor is triggered by a head or other external object, a motion sensor, such as an inertial measurement unit (IMU), can take measurements both before and after the proximity sensor is triggered. For example, the IMU can take repeated measurements at a specific frequency while the electronic device is powered on, thus providing data before and after the proximity sensor is triggered.

[0010] Using IMU data from a period immediately prior to the proximity sensor triggering, it can be determined whether the proximity sensor was triggered by the user's head or another object. In response to determining that the proximity sensor was triggered by the user's head, the display can be turned off / disabled to prevent inadvertent touch inputs on the display. Alternatively, in response to determining that the proximity sensor was triggered by an external object, such as the user's finger, the display can remain on to allow the user to continue interacting with the display.

[0011] An illustrative electronic device of the type that may be equipped with a proximity sensor and a display is shown in Fig. 1. The electronic device 10 may be a computing device, such as a laptop computer, a computer monitor containing an embedded computer, a tablet computer, a mobile phone, a media player, or other handheld or portable electronic device, a smaller device such as a wristwatch or other device worn on a user's wrist, a pendant device, a headphone or earphone device, a device embedded in eyeglasses or other equipment worn on a user's head, or other wearable or miniature device, a television, a computer display that does not contain an embedded computer, a gaming device, a navigation device, an embedded system such as a system in which electronic equipment with a display is mounted in a kiosk system or automobile, equipment,which implements the functionality of two or more of these devices, or other electronic equipment.

[0012] As in Fig. 1, the electronic device 10 may include a control unit 16 (also referred to herein as control circuitry). The control unit 16 may include storage and processing circuitry to support the operation of the device 10. The storage and processing circuitry may include a storage device, e.g., a hard disk drive storage device, non-volatile memory (e.g., flash memory or other electrically programmable read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random access memory), etc. The processing circuitry in the control unit 16 may be used to control the operation of the device 10.The processing circuitry may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, application-specific integrated circuits, etc. The control unit 16 may include communication circuitry to support wired and / or wireless communication between the device 10 and external equipment. For example, the control unit 16 may include wireless communication circuitry, such as cellular phone communication circuitry and wireless local area network communication circuitry. The communication circuitry may include one or more antennas that transmit and / or receive data or other information from external sources.

[0013] Input / output devices 12 may be used to provide data to device 10 and to provide data from device 10 to devices external to device 10. Input / output devices 12 may include, for example, buttons, joysticks, scroll wheels, touchpads, keypads, keyboards, microphones, speakers, sound generators, vibrators, cameras, light-emitting diodes and other status indicators, data ports, etc. A user may control the operation of device 10 by providing commands via input / output devices 12 and may obtain status information and other outputs from device 10 using the output resources of input / output devices 12.

[0014] Input / output devices 12 may include one or more displays, such as display 14. Display 14 may be a touchscreen display, which may include a touch sensor for detecting touch input from a user (e.g., from a user's finger, a stylus, or other input device), or display 14 may be touch-insensitive. A touch sensor for display 14 may be based on an array of capacitive touch-sensing electrodes, acoustic touch-sensing structures, resistive touch components, force-based touch-sensing structures, a light-based touch sensor, or other suitable touch-sensing devices. Display 14 may include any desired display technology and may be an organic light-emitting diode (OLED) display, a liquid crystal display (LCD), a micro-LED display, or any other desired type of display.

[0015] Sensors 18 may include a light-based proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, an ultrasonic proximity sensor, a photoelectric proximity sensor, a magnetic sensor (e.g., a magnetometer or compass), an inertial measurement sensor, an accelerometer or other motion sensor, a force sensor, a touch sensor, a temperature sensor, a pressure sensor, a microphone, a radio frequency sensor, a three-dimensional image sensor, an ambient light sensor, a camera, a light-based position sensor (e.g., a lidar sensor), and / or other sensors. Sensors 18 may include one or more of these sensors, if desired. A battery in device 10 may store energy used to power display 14, sensors 18, and other components of device 10.

[0016] A perspective view of an illustrative electronic device of the type that may include a proximity sensor and a display is shown in Fig. 2. In the example of Fig. 2, the device 10 includes a display, such as the display 14 mounted in the housing 22. The display 14 may be a liquid crystal display, an electrophoretic display, an organic light-emitting diode display, or other display having an array of light-emitting diodes (e.g., a display including pixels with diodes formed from crystalline semiconductor chips), may be a plasma display, may be an electrowetting display, may be a display based on microelectromechanical systems (MEMs) pixels, or may be any other suitable display. The display 14 may include a pixel array 26 extending across part or all of the front face F of the device 10 and / or other external device surfaces. The pixel array may be rectangular or have another suitable shape. The display 14 may be formed using a display cover layer (e.g.,a transparent front housing layer), such as a layer of transparent glass, clear plastic, sapphire, or other transparent layer. The display cover layer may overlap the pixel array 26.

[0017] The housing 22, sometimes referred to as an enclosure, may be formed from plastic, glass, ceramic, fiber composites, metal (e.g., stainless steel, aluminum, etc.), other suitable materials, or a combination of two or more of these materials. The housing 22 and the display 14 may separate an interior region of the device 10 from an exterior region surrounding the device 10. The housing 22 may be formed using a single-body configuration in which part or all of the housing 22 is machined or formed as a single structure, or it may be formed using multiple structures (e.g., an internal frame structure, one or more structures forming exterior housing surfaces, etc.).

[0018] The pixels 26 may cover substantially the entire front surface F of the device 10, or the display 14 may have inactive areas (e.g., notches, recessed areas, rectangular island areas, or other areas) that are free of pixels 26. The inactive areas may be used to accommodate an opening for a speaker and windows for optical components, such as one or more image sensors, ambient light sensors, optical proximity sensors, three-dimensional image sensors such as 3D structured light image sensors, and / or a camera flash, etc. In an example configuration, the pixels 26 may extend across the front surface F of the device 10 and overlap the proximity sensor 30. In this type of arrangement, the proximity sensor 30 may detect whether an external object (e.g., a user's face or finger) is in close proximity to the display 14 by operating through the display 14.Alternatively, the proximity sensor 30 can also be located in an inactive area of ​​the display 14.

[0019] Device 10 may also include speaker 31. Speaker 31 may be surrounded by pixels 26 of display 14 or formed in an inactive area of ​​display 14. Speaker 31 may generate sound for a user when the user uses device 10, for example, to make a call, listen to a voice message, and / or interact with a virtual assistant.

[0020] During operation, the display 14 may be turned on (activated). When a user holds the device 10 to their head to hear the sound from the speaker 31, it may be desirable to turn off (deactivate) the display 14 to prevent accidental touches of the display. For example, a touch sensor of the display 14 and / or a pixel array of the display 14 may be turned off (and / or a portion of the touch sensor and / or the pixel array may be turned off). Therefore, the proximity sensor 30 may be used to detect the presence of the user's head. However, other external objects, such as the user's finger, a stylus, or other object, may be detected by the proximity sensor 30, such as when the user interacts with the display 14.Therefore, to determine whether proximity sensor 30 has detected the user's head or another object, measurements from an inertial measurement unit (IMU) in device 10 may be used. Specifically, data from the IMU, such as position, acceleration, and / or rotation data, available immediately before the proximity sensor is triggered may be used. If proximity sensor 30 was triggered by the user's head, display 14 may be deactivated. If proximity sensor 30 was triggered by another object, such as the user's finger, display 14 may remain activated.

[0021] Although the user's finger is described throughout as triggering the proximity sensor 30, this is for illustrative purposes only. In general, the proximity sensor 30 can detect any suitable external object, such as a stylus.

[0022] Although Fig. While Figure 2 shows the electronic device 10 as a mobile phone, this arrangement is for illustrative purposes only. In general, the electronic device 10 may be any electronic device. For example, the device 10 may be a wearable device, such as a wristwatch or a head-mounted device.

[0023] Regardless of the device type of device 10, the proximity sensor 30 may be an optical proximity sensor. An illustrative example of an electronic device with an optical proximity sensor is shown in Fig. 3 shown.

[0024] As in Fig. 3, the proximity sensor 30 in the device 10 may operate through the layer 32. The layer 32 may include, for example, a transparent layer of the device 10, such as a transparent cover layer that overlaps a display, a layer with a transparent opening, and / or one or more display layers. In some illustrative examples, the proximity sensor 30 may operate through a display (e.g., the layer 32 may cover the display 14 of the Fig. 1 and Fig. 2).

[0025] In operation, proximity sensor 30 may emit light 36, for example, using a light-emitting diode or other light source. Light 36 may be infrared light, near-infrared light, or other suitable light. Light 36 may be reflected from an external object 34 as light 38. Proximity sensor 30 may detect light 38 after it has passed through layer 32, for example, using a photodetector sensitive to the wavelength of light 36 and light 38. In particular, by determining the time period between proximity sensor 30 emitting light 36 and detecting light 38, control circuitry in device 10 may determine the proximity of external object 34.

[0026] Although the proximity sensor 30 in Fig. 3 is shown as an optical proximity sensor, this is for illustrative purposes only. In general, proximity sensor 30 may be a light-based proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, an ultrasonic proximity sensor, a photoelectric proximity sensor, or any other suitable proximity sensor.

[0027] The external object 34 may be a user's head, a user's finger, or another external object. In some embodiments, it may be desirable to display a display (e.g., display 14 in the Fig. 1 and Fig. 2) to deactivate in the device 10 when the external object 34 is a user's head (e.g., when the user has brought the device 10 to his head / ear), while the indicator remains on / activated when the external object is a user's finger or other external object.

[0028] To determine whether the external object 34 is a user's head or another external object, data from other sensors in the device 10 from the period immediately preceding the detection of the object 34 by the proximity sensor 30 may be used. In some example embodiments, data from one or more motion sensors, such as an inertial measurement unit (IMU), may be used. The IMU may include one or more accelerometers, gyroscopes, magnetometers, and / or other motion sensors. Fig. 4A to 4C show illustrative examples of IMU data that may indicate whether the user has held the device 10 to the head (and the display should be disabled).

[0029] As in Fig. 4A, the graph 40 includes the curve 42. The curve 42 is an illustrative relationship of the position of an electronic device (such as the housing 22 of the device 10 from the Fig. 1-3) over time. The illustrative position of the device housing over time may be the relative device position in a vertical direction (e.g., Z direction) and / or one or more horizontal directions (e.g., an X and / or Y direction). In other words, the changes in curve 42 may be the relative position changes of the device housing with respect to itself.

[0030] The time period 44 may correspond to a period of time during which the user has touched the display with their finger or another external object, such as a stylus. In particular, the time period 44 may extend over 1 / 4 second, 1 / 2 second, 3 / 4 second, 1 second, or another period of time before the proximity sensor 30 detects the finger or other external object. As shown in Figure 40, the relative position of the device 10 may remain relatively constant during the time period 44.

[0031] Time period 46, on the other hand, may correspond to a period in which the user held the electronic device to their head (e.g., to answer a call). In particular, time period 46 may extend over 1 / 4 second, 1 / 2 second, 3 / 4 second, 1 second, or another period before proximity sensor 30 detects the user's head. During time period 46, the relative position of device 10 may change significantly more than during time period 44. For example, the user may need to lift the device perpendicular to their head, and this change in distance may be measured by the IMU during time period 46. Therefore, the change in the relative position of device 10 in a period immediately before a proximity sensor is triggered may indicate whether a user touched the display with their finger or held the device to their head.

[0032] Another illustrative example of IMU data that can be used to distinguish between a user’s finger and the lifting of the device to the user’s head is shown in Fig. 4B. As shown in Fig. 4B, the graph 48 includes the curve 50. The curve 50 is an illustrative relationship of the acceleration of an electronic device (such as the device 10 of the Fig. 1-3) over time. In period 44, the electronic device may exhibit a small acceleration, which may indicate that the user touched the display with their finger or another object. In contrast, in period 46, the electronic device may exhibit a large acceleration, which may indicate that the user raised the device to their head. In particular, the user may accelerate the device when lifting the device to their head, and the acceleration of the device may be measured by the IMU during period 46. Therefore, the change in acceleration of device 10 in a period immediately before a proximity sensor is triggered may indicate whether a user touched the display with their finger or held the device to their head.

[0033] A third illustrative example of IMU data that can be used to distinguish between a user’s finger and the lifting of the device to the user’s head is shown in Fig. 4C. As shown in Fig. 4C, the graph 52 includes the curve 54. The curve 54 is an illustrative relationship of the rotational speed of an electronic device (such as the device 10 of the Fig. 1-3) over time. In period 44, the electronic device may have a low rotation speed, which may indicate that the user touched the display with their finger or another object. In contrast, in period 46, the electronic device may have a high rotation speed, which may indicate that the user raised the device to their head. In particular, the user may rotate the device when they raise the device to their head, and the rotation speed of the device may be measured by the IMU during period 46. Therefore, the rotation speed of device 10 in a period immediately before a proximity sensor is triggered may indicate whether a user touched the display with their finger or held the device to their head.

[0034] The relationships between position, acceleration and / or rotation versus time from the Fig. 4A, Fig. 4B and Fig. 4C can be determined directly from IMU measurements or calculated from IMU data. For example, the position of the device can be determined from multiple position measurements of the electronic device (e.g., a moving average of the position measurements, a standard deviation of the position measurements, etc. can be included). The acceleration of the device can be based on a moving average of acceleration measurements. The rotational speed of the device can be determined using a gyroscope in the IMU, and the relationship of the rotational speed over time can be based on a moving average of acceleration measurements. However, these examples are for illustrative purposes only. In general, any IMU data and / or other motion sensor data can be used in a suitable manner to determine whether a detected external object is a user's finger or head.

[0035] Regardless of the IMU data used, the illustrative schematic diagram 56 in Fig. 5, how proximity information 58 can be obtained from the proximity sensor 30 and motion information 60 from the IMU 62. In particular, the proximity sensor 30 can be activated in response to the detection of an external object (e.g., the external object 34 in Fig. 3) Generate signals, such as by emitting light and detecting light reflected from the external object, or by capacitively sensing the external object. Generally, the proximity sensor 30 can be triggered when an external object comes close enough to the proximity sensor 30 (e.g., within a predetermined threshold).

[0036] The IMU 62 may generate signals in response to movement of the device (e.g., movement of the device's housing), such as device 10 of FIGS. 1-3. The IMU 62 may include one or more accelerometers, gyroscopes, and / or magnetometers to measure the movement of the device. The motion signals and proximity signals may be analog or digital signals and may constitute motion data and proximity data, respectively.

[0037] The sensor analysis 64 may analyze the signals / data generated by the proximity sensor 30 and the IMU 62. In particular, a control unit, such as the control unit 16 in device 10 ( Fig. 1), determine when the proximity sensor 30 is triggered (e.g., when the proximity sensor 30 detects the presence of an external object). In response to the proximity sensor 30 being triggered, the controller may analyze the motion data generated by the IMU 62 in a period immediately prior to the proximity sensor 30 being triggered, for example, a period of 0.25 s, 0.5 s, 0.75 s, 1 s, or 1.5 s, to determine whether the detected external object was the user's finger or head.

[0038] To make this determination, a machine learning algorithm can be used to correlate the motion data from the IMU 62 with whether the proximity sensor 30 was triggered by the user's finger or by raising the device toward the head. For example, a gradient boosted tree algorithm, a random forest algorithm, a decision tree algorithm, a support vector machine algorithm, a multi-layer perceptron algorithm, a convolutional neural network algorithm, or another suitable machine learning algorithm can be used to determine the detected external object based on the IMU data. Any number of features from the IMU data (or other sensor data) can be used to train the machine learning algorithm and make this determination.In some illustrative examples, the position, acceleration, and rotational speed of the device may be used, such as in the . Fig. 4A-4C. Sampled changes in position, acceleration, and / or rotational velocity (and / or other features) when a proximity sensor is covered with a finger and when the device is raised toward the user's head can be used to train the algorithm.

[0039] After the algorithm has been trained with these sampled features, the algorithm can be used to determine, based on the position, acceleration, and rotational speed (and / or other inputs) of the device, whether the proximity sensor is covered by a finger or whether the device is raised toward the user's head. In other words, the finger indication determination 66 can be performed. The finger indication determination 66 can be a determination that determines whether the external object detected by the proximity sensor is a finger, such as a finger over the display 14 and the proximity sensor 30 of Fig. 2 (e.g., as opposed to the user holding the device to their head). This allows the control unit to determine whether a detected external object is the user's finger or another object, or whether the user has brought the phone to their head.

[0040] Although the finger indication determination 66 is described as determining whether a finger has triggered the proximity sensor 30, this is for illustrative purposes only. Generally, the finger indication determination can be used to determine whether a suitable external object, such as a stylus, has triggered the proximity sensor 30.

[0041] Appropriate actions may be taken on the basis of Regulation 66. In particular, an advertisement (e.g. advertisement 14 of Fig. 1) be deactivated when it is determined that the user has brought the device to their head. This can reduce false touch inputs that can occur when the user brings the device to their head / face. To deactivate the display, the controller (or other suitable circuitry) can deactivate a touch sensor in the display (or part of the touch sensor), turn the display off (e.g., prevent a pixel array (or part of the pixel array) of the display from emitting light), or otherwise disable one or more functions of the display. On the other hand, if it is determined that the proximity sensor was triggered by a user's finger or other external object, the display can remain on (e.g.,The display pixels can continue to emit light, and the touch sensor can remain activated to enable user input and other interactions with the display. This allows the display to remain on even when the proximity sensor detects the user's finger, enabling more seamless interaction with the electronic device.

[0042] In Fig. Figure 6 is an illustrative diagram showing the process used to determine whether a proximity sensor is triggered by a finger (or other object) or by holding a device to a user's head. As shown in Fig. 6, process 68 may begin with the determination 70 of whether the device is already on the user's head (e.g., based on the last determination using process 68), with the determination 72 of whether the proximity sensor is covered / triggered (e.g., based on a measurement of a proximity sensor, such as proximity sensor 30 from the Fig. 2 and Fig. 3) and with determination 74, whether the process timeout has been exceeded (e.g., one second, two seconds, or another suitable time frame has passed). If determination 70 determines that the device is not yet at the user's head, determination 72 determines that the approach has been triggered, and determination 74 determines that the process timeout has been exceeded, the process may proceed to finger determination 76. Otherwise, the device (e.g., a control unit within the device) may wait until these determinations have been made before proceeding to finger determination 76.

[0043] Although Fig. Figure 6 shows that all three determinations 70, 72, and 74 must be performed before the process proceeds to finger determination 76; this is for illustrative purposes only. Generally, one or more of determinations 70, 72, and / or 74 (and / or other suitable determinations) may be performed before proceeding to finger determination 76.

[0044] Finger determination 76 may use a machine learning algorithm to correlate motion data from an IMU in the electronic device with whether the proximity sensor was triggered by the user's finger or by the user raising the device to their head. For example, a gradient boosted tree algorithm, a random forest algorithm, a decision tree algorithm, a support vector machine algorithm, a multi-layer perceptron algorithm, a convolutional neural network algorithm, or another suitable machine learning algorithm may be used to determine the detected external object based on the IMU data. Any number of features from the IMU data may be used to train the machine learning algorithm and perform this determination.In some illustrative examples, the position, acceleration, and rotational speed of the device may be used, as shown in the . Fig. 4A-4C. Sampled changes in position, acceleration, and / or rotational velocity (and / or other features) when a proximity sensor is covered with a finger and when the device is raised toward the user's head can be used to train the algorithm.

[0045] After the algorithm has been trained with these sampled features, the algorithm can be used to determine, based on the position, acceleration, and rotational speed (and / or other inputs) of the device, whether the proximity sensor is covered by a finger or whether the device is raised toward the user's head. In other words, a finger determination 76 can be performed. The finger determination 76 can be a determination that determines whether the external object detected by the proximity sensor is a finger, such as a finger over the display 14 and the proximity sensor 30 of Fig. 2 (e.g., as opposed to the user holding the device to their head). This allows the control unit to determine whether a detected external object is the user's finger or another object, or whether the user has brought the phone to their head.

[0046] Although finger detection 76 is described as determining whether a finger has triggered proximity sensor 30, this is for illustrative purposes only. Generally, finger detection 76 can determine whether a suitable external object, such as a stylus, has triggered proximity sensor 30.

[0047] If it is determined that the external object that triggered the proximity sensor was a finger (or other external object, such as a stylus), process 68 may leave the display on, as indicated in block 78.

[0048] In other words, the display pixels can continue to emit light and the touch sensor can remain enabled to allow user input and other interactions with the display.

[0049] Alternatively, if it is determined that the external object is not a finger, a second algorithm may be used to perform off-head finger detection 80. In particular, off-head finger detection 80 may determine whether the user has briefly removed the device from their head to interact with the display. Off-head finger detection 80 may be similar to finger determination 76 and may use motion data from the IMU to determine whether the user is briefly interacting with the display with their finger (e.g., using a machine learning algorithm). If it is determined that the user is attempting to briefly interact with the display, process 68 may continue to keep the display on, as indicated in block 78.

[0050] Alternatively, if it is determined that the user is not attempting to briefly interact with the display using their finger, it may be determined that the user is holding the device to their head (e.g., to make a call), and the display may be turned off / deactivated, as indicated in block 82. For example, a touch sensor (or a portion of the touch sensor) in the display may be deactivated, the display may be turned off entirely (e.g., a pixel array (or a portion of the pixel array) of the display may stop emitting light), or one or more functions of the display may be deactivated in some other way. In this way, the display may be turned off / deactivated when it is determined that the user has raised the phone to their head, and false touch inputs that may otherwise occur may be reduced or eliminated.

[0051] Although in the Fig. 5 and Fig. 6, the use of proximity information (e.g., data from proximity sensor 30) and motion information (e.g., motion data from IMU 62) was shown and described, other sensor information may also be used in determining whether the proximity sensor was triggered based on a user's finger or in response to the device being raised toward the user's head. In general, data from other sensors may also be used in combination with the motion data, if desired. For example, data from motion sensors, data from ambient light sensors, data from display touch sensors, and / or other desired data may be used to determine whether the proximity sensor was triggered by a user's finger or head.

[0052] As an illustrative example, the ambient light sensor readings may be lower when the device is held against the user's head (e.g., the ambient light sensor may be at least partially covered, reducing the amount of ambient light reaching the sensor) than when the user is touching the display. Therefore, the ambient light sensor measurements, in combination with the IMU data, can be used to determine whether the display should remain on or be disabled.

[0053] Alternatively or additionally, the display's touch sensor may be used. In particular, the display's touch sensor may receive touch input corresponding to the shape of a user's ear when the user holds the phone to their head (e.g., to make a call). In contrast, the display's touch sensor may receive touch input with a smaller area when the user merely interacts with the display. Therefore, the display's touch sensor measurements, in combination with the IMU data and / or the ambient light sensor measurements, may be used to determine whether the display should remain on or be deactivated.

[0054] Regardless of the sensor(s) used to determine whether the external object has triggered a proximity sensor, Fig. 7 illustrates steps that can be used to make such a determination and to adapt the display based on that determination.

[0055] As in the case 81 of Fig. 7, in step 83, one or more sensors in an electronic device may collect sensor measurements. The sensor(s) may include a proximity sensor, an IMU, an ambient light sensor, a touch sensor, and / or other desired sensors. The sensor measurements may be analog or digital signals and may constitute sensor data. In some embodiments, the sensor data may be collected continuously (e.g., at regular frame rate intervals) while the electronic device is powered on. Alternatively, the sensor measurements may be collected during specific time intervals. In one illustrative embodiment, proximity data may be collected from a proximity sensor (e.g., proximity sensor 30 of Fig. 5) and movement data from an IMU (e.g. IMU 62 from Fig. 5) be captured continuously at regular frame rate intervals while the device (and / or display) is switched on.

[0056] In step 84, the sensor measurements / data may be analyzed. For example, a control unit, such as control unit 16 in device 10 ( Fig. 1), analyze the proximity data and the motion data. In particular, the control unit can first determine whether the proximity sensor has been triggered. In other words, the control unit can determine whether an external object is within a threshold proximity of the proximity sensor.

[0057] In response to determining that the proximity sensor was triggered by an external object, the controller may further analyze the data to determine whether the proximity sensor was triggered in response to the user raising the device to their head (as opposed to simply touching the display). Specifically, the controller may use a machine learning algorithm to correlate the motion data from the IMU in the electronic device with whether the proximity sensor was triggered by the user's finger or by the user raising the device to their head.For example, a gradient boosted tree algorithm, a random forest algorithm, a decision tree algorithm, a support vector machine algorithm, a multi-layer perceptron algorithm, a convolutional neural network algorithm, or another suitable machine learning algorithm may be used to determine the detected external object based on the IMU data. Any number of features from the IMU data may be used to train the machine learning algorithm and perform this determination. In some illustrative examples, the position, acceleration, and rotational velocity of the device may be used, as shown in the . Fig.4A-4C. Sampled changes in position, acceleration, and / or rotational velocity (and / or other features) when a proximity sensor is covered with a finger and when the device is raised toward the user's head can be used to train the algorithm.

[0058] After training the algorithm with these sampled features, the algorithm can be used to determine, based on the device's position, acceleration, and rotational velocity (and / or other inputs), whether the proximity sensor is covered by a finger or whether the device is being raised toward the user's head. This allows the control unit to determine whether a detected external object is the user's finger or another object, or whether the user has raised the phone toward their head.

[0059] If, in step 86, the proximity sensor was triggered by the user's head (e.g., if it was determined that the user had raised the device to their head), the controller (or other circuit logic) may proceed to step 88 and turn off the display. For example, a touch sensor (or part of the touch sensor) in the display may be disabled, the display may be turned off entirely (e.g., a pixel array (or part of the pixel array) of the display may stop emitting light), or one or more functions of the display may be disabled in some other way. In this way, the display may be turned off / deactivated when it is determined that the user has raised the phone to their head, and false touch inputs that might otherwise occur may be reduced or eliminated.

[0060] Alternatively, at step 90, the display may remain on if it is determined that the proximity sensor was triggered by the user touching the display with their finger or another external object (e.g., the user did not raise the device to their head). In other words, the display pixels may continue to emit light, and the touch sensor may remain enabled to enable user input and other interactions with the display. In this way, the display may remain on even if the proximity sensor detects the user's finger, enabling a more seamless interaction with the electronic device.

[0061] According to one embodiment, an electronic device is provided comprising a housing, a display located in the housing having a pixel array configured to output images and a touch sensor, a proximity sensor in the housing configured to be triggered in response to proximity of an external object, an inertial measurement unit in the housing configured to measure movement of the housing, and a control unit in the housing, wherein the control unit is configured to determine, based on the movement of the housing, whether the proximity sensor was triggered in response to a finger or to the housing being held to the head.

[0062] According to another embodiment, the controller is further configured to deactivate the display in response to determining that the proximity sensor was triggered in response to the housing being held to the head.

[0063] According to another embodiment, the control unit is configured to deactivate the display by deactivating the touch sensor in the display.

[0064] According to another embodiment, the controller is further configured to deactivate the display by turning off at least a portion of the pixel array in the display.

[0065] According to another embodiment, the controller is further configured to keep the pixel array and the touch sensor turned on in response to determining that the proximity sensor was triggered in response to the finger.

[0066] According to another embodiment, the inertial measuring unit is configured to measure a position of the housing, an acceleration of the housing, and a rotational speed of the housing.

[0067] According to another embodiment, the control unit is configured to determine whether the proximity sensor was triggered in response to the finger or in response to the housing being held to the head based on the position of the housing, the acceleration of the housing, and the rotational speed of the housing.

[0068] According to another embodiment, the controller is configured to use a machine learning algorithm that uses the position of the housing, the acceleration of the housing, and the rotational speed of the housing as inputs to determine whether the proximity sensor was triggered in response to the finger or in response to the housing being held to the head.

[0069] According to a further embodiment, the proximity sensor is located below the display.

[0070] According to another embodiment, the proximity sensor is an optical proximity sensor.

[0071] According to another embodiment, the controller is further configured to keep the pixel array and the touch sensor turned on in response to determining that the proximity sensor was triggered in response to the finger.

[0072] According to another embodiment, the inertial measurement unit includes an accelerometer, a gyroscope and a magnetometer.

[0073] According to one embodiment, a method of operating an electronic device having a display, a proximity sensor, and an inertial measurement unit is provided, comprising acquiring proximity sensor measurements with the proximity sensor and motion sensor measurements with the inertial measurement unit, detecting an external object with the proximity sensor, determining, in response to detecting an external object, whether the external object is a finger or the user's head based on the motion sensor measurements, and deactivating the display in response to determining that the external object is the user's head.

[0074] According to another embodiment, determining whether the external object is the user's finger or head based on the motion sensor measurements includes inputting a position of the electronic device, an acceleration of the electronic device, and a rotational speed of the electronic device into a machine learning algorithm.

[0075] According to another embodiment, the method comprises training the machine learning algorithm based on known correlations between the position of the electronic device, the acceleration of the electronic device, and the rotational speed of the electronic device, as well as whether the external object is the user's finger or head.

[0076] According to another embodiment, disabling the display comprises disabling a touch sensor and a pixel array of the display.

[0077] According to another embodiment, the method comprises, in response to determining that the external object is the user's finger, leaving the touch sensor and the pixel array of the display turned on.

[0078] According to one embodiment, an electronic device is provided comprising a housing, a display in the housing, a proximity sensor in the housing, wherein the proximity sensor is configured to be triggered in response to an external object, a motion sensor in the housing, wherein the motion sensor is configured to measure movement of the housing, and a controller in the housing, wherein the controller is configured to determine, based on the movement of the housing, whether the proximity sensor was triggered in response to a user's finger or in response to a user's head, and wherein the controller is configured to deactivate the display in response to determining that the proximity sensor was triggered in response to the user's head.

[0079] According to another embodiment, the movement of the housing includes a position of the housing, an acceleration of the housing, and a rotational speed of the housing.

[0080] According to another embodiment, the motion sensor includes an inertial measurement unit including an accelerometer, a gyroscope, and a magnetometer.

[0081] The foregoing is for illustrative purposes only, and various modifications may be made to the described embodiments. The above embodiments may be implemented individually or in any combination. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 18 / 671,116

[0001] US 63 / 581,534

[0001]

Claims

[1] Electronic device comprising: a housing; a display in the housing and comprising a pixel array configured to emit images and a touch sensor; a proximity sensor in the housing configured to be triggered in response to proximity of an external object; an inertial measurement unit in the housing configured to measure a movement of the housing; and a control unit in the housing, the control unit configured to determine, based on the movement of the housing, whether the proximity sensor was triggered in response to a finger or the housing being held to a head. [2] The electronic device of claim 1, wherein the controller is further configured to deactivate the display in response to determining that the proximity sensor was triggered in response to the housing being held to the head. [3] The electronic device of claim 2, wherein the controller is configured to deactivate the display by deactivating the touch sensor in the display. [4] The electronic device of claim 3, wherein the controller is further configured to deactivate the display by turning off at least a portion of the pixel array in the display. [5] The electronic device of claim 4, wherein the controller is further configured to keep the pixel array and the touch sensor powered on in response to determining that the proximity sensor was triggered in response to the finger. [6] The electronic device according to claim 1, wherein the inertial measuring unit is configured to measure a position of the housing, an acceleration of the housing, and a rotational speed of the housing. [7] The electronic device of claim 6, wherein the control unit is configured to determine whether the proximity sensor was triggered in response to the finger or in response to the housing being held to the head based on the position of the housing, the acceleration of the housing, and the rotational speed of the housing. [8] The electronic device of claim 7, wherein the controller is configured to use a machine learning algorithm that uses the position of the housing, the acceleration of the housing, and the rotational speed of the housing as inputs to determine whether the proximity sensor was triggered in response to the finger or in response to the housing being held to the head. [9] The electronic device of claim 1, wherein the proximity sensor is located below the display and wherein the proximity sensor is an optical proximity sensor. [10] The electronic device of claim 1, wherein the controller is further configured to keep the pixel array and the touch sensor powered on in response to determining that the proximity sensor was triggered in response to the finger. [11] The electronic device according to claim 1, wherein the inertial measurement unit comprises an accelerometer, a gyroscope and a magnetometer. [12] A method of operating an electronic device having a display, a proximity sensor and an inertial measurement unit, the method comprising: Capturing proximity sensor measurements with the proximity sensor and motion sensor measurements with the inertial measurement unit; Detecting an external object with the proximity sensor; in response to detecting the external object, determining whether the external object is a finger or a head of the user based on the motion sensor measurements; and in response to determining that the external object is the user's head, disabling the display. [13] The method of claim 12, wherein determining whether the external object is the user's finger or head based on the motion sensor measurements comprises inputting a position of the electronic device, an acceleration of the electronic device, and a rotational speed of the electronic device into a machine learning algorithm. [14] The method of claim 13, further comprising: Training the machine learning algorithm based on known correlations between the position of the electronic device, the acceleration of the electronic device, the rotation speed of the electronic device, and whether the external object is the user's finger or head. [15] The method of claim 12, wherein deactivating the display comprises deactivating a touch sensor and a pixel array of the display, the method further comprising: in response to determining that the external object is the user's finger, leaving the touch sensor and the pixel array of the display turned on.

Images