Ultrasonic touch and force input detection

Abstract

A system includes an ultrasonic input device coupled to a layer of material having an outer surface and one or more data processors. The ultrasonic input device receives a set of reflected ultrasonic signals associated with the transmitted signal, the set of reflected ultrasonic signals associated with a touch event between the object and the outer surface of the material layer. The one or more data processors are configured to: determine an energy signal comprising energy measurements over time, each energy measurement corresponding to a summation value obtained from a portion of the set of reflected ultrasound signals; extracting feature information associated with the energy signal; determining an inference associated with the object based on the extracted feature information; and generating an output signal associated with the determined inference.

Description

This application is a divisional application of a Chinese patent application with application number 201980034705.6 and invention title "Ultrasonic Touch and Force Input Detection", filed on May 21, 2019. Cross - Reference to Related Applications

[0001] This application is non - provisional and claims the benefit of U.S. Provisional Application Serial No. 62 / 674,317, filed on May 21, 2018, titled "ULTRASONIC TOUCH AND FORCE INPUT DETECTION"; U.S. Provisional Application Serial No. 62 / 725,697, filed on August 31, 2018, titled "ULTRASONIC TOUCH AND FORCE INPUT DETECTION"; U.S. Provisional Application Serial No. 62 / 751,053, filed on October 26, 2018, titled "ULTRASONIC TOUCH FEATURE EXTRACTION"; U.S. Provisional Application Serial No. 62 / 784,615, filed on December 24, 2018, titled "ULTRASONIC TOUCH SENSOR AND SYSTEM"; U.S. Provisional Application Serial No. 62 / 810,786, filed on February 26, 2019, titled "ULTRASONIC TOUCH DETECTION AND DECISION"; U.S. Patent Application Serial No. 16 / 396,597, filed on April 26, 2019, titled "ULTRA SONIC TOUCH AND FORCE INPUT DETECTION"; and U.S. Patent Application Serial No. 16 / 417,184, filed on May 20, 2019, titled "ULTRASONIC TOUCH FEATURE EXTRACTION", the entire contents of which are incorporated herein by reference. Background of the Invention

[0002] Capacitive, resistive, and inductive sensing are used in industrial, automotive, medical, and consumer applications to detect touch inputs. In human - machine interface devices (HIDs) such as trackpads and touchscreens, the use of capacitive technology to detect touch inputs has grown rapidly. Consumer and industrial applications have started to adopt touch buttons and sliders using capacitive technology in devices such as mobile phones, TV controls, automotive dashboards, remote controls, or industrial controls. In terms of appearance and reliability, capacitive sensing has proven to be more attractive than mechanical switches and rotary encoders.

[0003] However, due to the challenges of touch input layout and system stack, using capacitive, resistive, or inductive sensing limits creative industrial design. The conflicting priorities between design and robustness further complicate the design. It should also be noted that current input touch sensing methods cannot be implemented on metal surfaces. In addition, current sensing technologies have an inherent property that limits waterproof applications. Pressure sensing technology using strain gauges has emerged as an alternative sensing technology for touch input on metal surfaces. However, the measurement of deflection and strain is usually unreliable, especially in metals. Such sensors are very susceptible to unwanted disturbances that cause surface deflection, and their sensitivity and performance are highly dependent on the overall boundary conditions of the surface to which they are attached. In addition, the surface to which the sensor is attached must be conformal enough to deflect sufficiently when touched by a person so that the sensor can detect it. An additional sensing layer (e.g., capacitive) is required to detect the x-y position of the input touch detected using the strain gauge. The increased complexity of touch input interface materials, the implications of complex interfaces in industrial design, waterproofing, and cost have become key challenges limiting the use of touch input in any environment and on any material. There is a need for improved systems and methods for detecting touch input to a human-machine interface (HMI).

[0004] Embodiments of the present invention, alone and in combination, solve these and other problems. SUMMARY OF THE INVENTION

[0005] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch and force input detection.

[0006] According to some embodiments, a method is provided. A transducer coupled to a first surface of a material layer can transmit an ultrasonic signal directed at a second surface, the material layer having a distance between the first surface and the second surface. The transducer can then detect the reflected ultrasonic signal and then determine the amplitude of the reflected ultrasonic signal. Subsequently, the transducer can determine that the amplitude exceeds a threshold associated with a portion of the ultrasonic signal that penetrates the second surface. In the case where the amplitude exceeds the threshold, the transducer can generate a signal indicating a touch input on the second surface.

[0007] According to other embodiments, a method is provided. A transducer coupled to a first surface of a material layer can transmit an ultrasonic signal directed at a second surface, the material layer having a distance between the first surface and the second surface. Subsequently, the transducer can detect the reflected ultrasonic signal. The method then includes determining an energy value associated with the reflected ultrasonic signal. The method may also include determining that the energy value exceeds a threshold associated with a portion of the ultrasonic signal that penetrates the second surface. In the case where the energy value exceeds the threshold, the method may include generating a signal indicating a touch input on the second surface.

[0008] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch feature extraction. The system may include an ultrasonic input device and one or more data processors.

[0009] The ultrasonic input device may be coupled to a material layer that may have an outer surface. The outer surface may be located at a position of the material layer opposite to the ultrasonic input device. The ultrasonic input device may be coupled to the material layer to transmit an emitted signal through the material layer toward the outer surface and receive a set of reflected ultrasonic signals associated with the emitted signal. The set of reflected ultrasonic signals may include at least one reflected ultrasonic signal and may be associated with a touch event between an object and the outer surface of the material layer. The touch event may include, for example, an individual touching the outer surface with their finger or other object (e.g., a stylus, etc.).

[0010] One or more data processors may be configured to determine an energy signal associated with the set of reflected ultrasonic signals and extract feature information associated with the energy signal. One or more data processors may also be configured to determine an inference associated with the object based on the extracted feature information and subsequently generate an output signal associated with the determined inference.

[0011] According to some embodiments, a computer-implemented method is provided. An emitted signal may be transmitted using an ultrasonic input device coupled to a material layer having an outer surface. A set of reflected ultrasonic signals may be received. The set of reflected ultrasonic signals may be associated with the emitted signal and may include at least one reflected ultrasonic signal. The set of reflected ultrasonic signals may be associated with a touch event between an object and the outer surface of the material layer. An energy signal associated with the set of reflected ultrasonic signals may be determined. Subsequently, feature information associated with the energy signal may be extracted. Subsequently, an inference may be determined. The inference may be associated with the object based on the extracted feature information. Subsequently, an output signal associated with the determined inference may be generated.

[0012] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch sensors and systems. The touch sensor may include an ultrasonic sensor layer and an integrated circuit layer.

[0013] The ultrasonic sensor layer may include an array of ultrasonic transducers. The array of ultrasonic transducers may include one or more ultrasonic transducers. The integrated circuit layer may be coupled to the ultrasonic sensor layer. The integrated circuit layer may include circuitry configured to drive the array of ultrasonic transducers to generate ultrasonic signals. The integrated circuit layer may also include circuitry configured to receive reflected ultrasonic signals using the array of ultrasonic transducers and generate an energy signal associated with the received reflected ultrasonic signals.

[0014] According to some embodiments, a method is provided that can be performed by a touch sensor or other suitable device. The method includes generating a drive signal in an integrated circuit coupled to a transmit ultrasonic transducer, the transmit ultrasonic transducer being located in an array of ultrasonic transducers coupled to the integrated circuit. The transmit ultrasonic transducer can be a piezoelectric micromachined ultrasonic transducer. Subsequently, an emitted ultrasonic signal can be generated by the transmit ultrasonic transducer in response to the drive signal. Generating the emitted ultrasonic signal can include transmitting the emitted ultrasonic signal through the material layer along a longitudinal direction orthogonal to an outer surface of the material layer, or in some embodiments within 20% of a normal to the outer surface of the material layer. Subsequently, a set of reflected signals can be received at a receive ultrasonic transducer in the array of ultrasonic transducers. The set of reflected signals can include one or more ultrasonic signals associated with the emitted ultrasonic signal. The receive ultrasonic transducer can be a piezoelectric micromachined ultrasonic transducer. Subsequently, an energy signal can be measured. The energy signal can be associated with the received set of reflected signals. The method can further include determining that a touch event has occurred at the outer surface of the material layer based on the measured energy signal.

[0015] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch detection and decision-making.

[0016] According to some embodiments, a method is provided that can be performed by a touch sensor device or other suitable device. The method includes receiving energy data associated with an ultrasonic input device coupled to a material layer. The energy data can include a current energy value and a past energy value associated with a reflected ultrasonic signal received at the ultrasonic input device in response to the ultrasonic input device emitting a signal through the material layer towards an outer surface of the material layer. Subsequently, the energy data can be compared with threshold data to generate a current trigger value for trigger data. In the case where the current energy value exceeds the current threshold of the threshold data, the trigger data can indicate the occurrence of a touch event. Subsequently, the threshold data can be updated based on the energy data, the trigger data, and the threshold data. Updating the threshold data can include generating a subsequent threshold.

[0017] According to some embodiments, another method is provided that can be performed by a touch sensor device or other suitable device. The method includes receiving energy data associated with an ultrasonic input device coupled to a material layer. The energy data can include a current energy value and a past energy value associated with a reflected ultrasonic signal received at the ultrasonic input device in response to the ultrasonic input device emitting a signal through the material layer towards an outer surface of the material layer. Subsequently, the energy data can be provided to a recurrent neural network to generate output data indicating that a touch event has occurred at the outer surface of the material layer.

[0018] These and other embodiments of the present invention are described in detail below. For example, other embodiments relate to systems, devices, and computer-readable media associated with the methods described herein.

[0019] A better understanding of the nature and advantages of the embodiments of the present invention can be obtained with reference to the following detailed description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram according to a particular aspect of the present disclosure, depicting the effect of touch force on reflected ultrasonic signals in an ultrasonic input system.

[0021] Figure 2 is a schematic diagram according to a particular aspect of the present disclosure, depicting an ultrasonic input system in a non-contact state and a contact state.

[0022] Figure 3 is a schematic diagram depicting an ultrasonic input device according to a particular aspect of the present disclosure.

[0023] Figure 4 is a cross-sectional view of two piezoelectric micromachined ultrasonic transducers incorporated into a CMOS wafer according to a particular aspect of the present disclosure.

[0024] Figure 5 is a set of schematic diagrams according to a particular aspect of the present disclosure, depicting an ultrasonic input device coupled to various surfaces.

[0025] Figure 6 is a schematic side view according to a particular aspect of the present disclosure, depicting an ultrasonic input system having a common board assembly.

[0026] Figure 7 is a schematic diagram according to a particular aspect of the present disclosure, depicting an exemplary ultrasonic input system.

[0027] Figure 8 is a schematic side view according to a particular aspect of the present disclosure, depicting an integrated ultrasonic input device having an ultrasonic sensor and an ASIC.

[0028] Figure 9 is a set of combined schematic diagrams according to a particular aspect of the present disclosure, depicting two processing routes for generating an integrated ultrasonic input device wafer.

[0029] Figure 10 is a set of schematic diagrams according to a particular aspect of the present disclosure, depicting a single integrated ultrasonic input device from wafer dicing, PCB mounting, and stack mounting.

[0030] Figure 11 is a schematic cross-sectional view of a consumer electronic product including an integrated ultrasonic input device according to a particular aspect of the present disclosure.

[0031] Figure 12 A set of schematic cross-sectional views comparing a non-integrated ultrasonic input device with an integrated ultrasonic input device according to a particular aspect of the present disclosure.

[0032] Figure 13A A top view of a sensor array of an ultrasonic input device according to a particular aspect of the present disclosure.

[0033] Figure 13B A top view of an alternative sensor array of an ultrasonic input device according to a particular aspect of the present disclosure.

[0034] Figure 14A According to a particular aspect of the present disclosure Figure 13A A schematic diagram of a sensor array depicting the functions of the individual transducers of the array. Figure 14B A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting eight example configurations of the individual transducers in the corner region of the sensor array. Figure 14C A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting eight example configurations of the individual transducers of the sensor array. Figure 14D A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting eight example configurations of the individual transducers of the sensor array. Figure 14E A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting two example configurations of the individual transducers of the sensor array. Figure 14F A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting example configurations of the individual transducers of sensor arrays of various sizes. Figure 14G A schematic diagram of a sensor array according to a particular aspect of the present disclosure depicting twelve example configurations of the individual transducers of a sensor array of size 8x8 transducers.

[0035] Figure 15 A set of graphs depicting energy measurements from transducers of a single sensor array operating at different frequencies according to a particular aspect of the present disclosure.

[0036] Figure 16 A graph according to a particular aspect of the present disclosure depicting the temperature behavior of an ultrasonic transducer with respect to the operating frequency.

[0037] Figure 17 A graph according to a particular aspect of the present disclosure depicting the frequency response with respect to the stack makeup.

[0038] Figure 18Is a schematic diagram depicting a circuit for receiving and transmitting signals through an ultrasonic transducer according to a particular aspect of the present disclosure, where the circuit is in a transmission state.

[0039] Figure 19 Is a schematic diagram depicting a circuit for receiving and transmitting signals through an ultrasonic transducer according to a particular aspect of the present disclosure, where the circuit is in a receiving state.

[0040] Figure 20 Is a schematic diagram according to a particular aspect of the present disclosure, which depicts an isolation circuit for receiving and transmitting signals through an ultrasonic transducer.

[0041] Figure 21 Is a set of schematic side views according to a particular aspect of the present disclosure, which depicts beamforming achieved by using an ultrasonic transducer.

[0042] Figure 22 Is a set of graphs, which depicts the operating modes of a micromachined ultrasonic transducer according to a particular aspect of the present disclosure compared to a standard bulk transducer, where the standard bulk transducer is depicted as the average displacement for different frequencies.

[0043] Figure 23 Is a set of schematic side views, which depicts the lateral signal rejection of a micromachined ultrasonic transducer according to a particular aspect of the present disclosure compared to a standard bulk transducer.

[0044] Figure 24 Is a set of schematic side views, which depicts the lateral signal rejection of a micromachined ultrasonic transducer according to a particular aspect of the present disclosure.

[0045] Figure 25 Is a schematic flowchart for digitally processing ultrasonic signals transmitted and received by an ultrasonic input device according to a particular aspect of the present disclosure.

[0046] Figure 26 Is a schematic flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to a particular aspect of the present disclosure.

[0047] Figure 27 Is a schematic example flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to a particular aspect of the present disclosure.

[0048] Figure 28 Is a schematic flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration by absolute value accumulation according to a particular aspect of the present disclosure.

[0049] Figure 29is a schematic flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration through self-mixing and integration, according to a particular aspect of the present disclosure.

[0050] Figure 30 is a schematic circuit diagram depicting an analog integrator with a negative bias current circuit, according to a particular aspect of the present disclosure.

[0051] Figure 31 is a schematic flowchart for processing ultrasonic signals, according to a particular aspect of the present disclosure, depicting the reduced impact of the time-of-flight variation of the reflected ultrasonic signal on touch input detection within the energy measurement window.

[0052] Figure 32 is a schematic diagram of a simplified process for processing ultrasonic signals, according to a particular aspect of the present disclosure, depicting the enhanced impact of the time-of-flight variation of the reflected ultrasonic signal on touch input detection outside the energy measurement window.

[0053] Figure 33 is a schematic diagram of a process for processing ultrasonic signals, according to a particular aspect of the present disclosure, depicting an ultrasonic signal with a minimal impact of the time-of-flight variation of the reflected ultrasonic signal on touch input detection outside the energy measurement window in the case of using window shaping.

[0054] Figure 34 is a schematic circuit diagram depicting a window shaping circuit, according to a particular aspect of the present disclosure.

[0055] Figure 35 is a schematic diagram depicting a process for processing ultrasonic signals to detect touch input using the amplitude of the reflected ultrasonic signal, according to a particular aspect of the present disclosure.

[0056] Figure 36 is a graph depicting a simplified example energy signal, according to a particular aspect of the present disclosure.

[0057] Figure 37 is a chart according to a particular aspect of the present disclosure, depicting reflected ultrasonic signal measurements made using an ultrasonic input device and showing techniques for improving touch input detection.

[0058] Figure 38 is a chart according to a particular aspect of the present disclosure, depicting reflected ultrasonic signal measurements made using an ultrasonic input device and showing additional techniques for improving touch input detection.

[0059] Figure 39 is a set of charts according to a particular aspect of the present disclosure, depicting the temperature dependence of the reflected ultrasonic signal.

[0060] Figure 40 A set of diagrams according to a particular aspect of the present disclosure, which depict the time-of-flight temperature dependence of two frequency methods for detecting touch input.

[0061] Figure 41 A diagram according to a particular aspect of the present disclosure, which depicts the measurement of reflected ultrasonic signals made across several frequencies using an ultrasonic input device and shows techniques for improving touch input detection.

[0062] Figure 42 A schematic plan view according to a particular aspect of the present disclosure, which depicts a dual-frequency PMUT having a concentric circle design.

[0063] Figure 43 A schematic plan view according to a particular aspect of the present disclosure, which depicts a multi-frequency ultrasonic input device having a square design.

[0064] Figure 44 A set of three diagrams according to a particular aspect of the present disclosure, which depict example signals received by an ultrasonic input system attributable to three different users.

[0065] Figure 45 A set of diagrams depicting energy measurement signals associated with a human finger, a water droplet, and placing the device on a table (e.g., placing an object on a sensor).

[0066] Figure 46 A combined schematic diagram and set of diagrams depicting how temperature can be used to further identify whether a human finger is initiating a touch event.

[0067] Figure 47 A combined schematic diagram and diagrams according to a particular aspect of the present disclosure, which depict finger touch and associated temperature information.

[0068] Figure 48 A schematic combined side view and signal diagram according to a particular aspect of the present disclosure, which depict the ridges and valleys of a fingerprint initiating a touch event on an ultrasonic input system.

[0069] Figure 49 A schematic diagram according to a particular aspect of the present disclosure, which depicts example signals received by an ultrasonic input system attributable to the same user initiating a touch event with a glove and without a glove.

[0070] Figure 50 A flowchart according to a particular aspect of the present disclosure, which depicts a process for extracting features from signals of an ultrasonic input system.

[0071] Figure 51is a diagram according to a particular aspect of the present disclosure, depicting a machine learning decision algorithm for improving touch detection.

[0072] Figure 52 is a flowchart according to a particular aspect of the present disclosure, depicting a process for detecting touch events.

[0073] Figure 53 is a schematic diagram according to a particular aspect of the present disclosure, depicting an adaptive threshold scheme for identifying touch events.

[0074] Figure 54 is an example curve graph according to a particular aspect of the present disclosure, depicting an energy signal and an adaptive threshold associated with identifying touch events.

[0075] Figure 55 is a schematic diagram according to a particular aspect of the present disclosure, depicting a general recurrent neural network.

[0076] Figure 56 is a schematic diagram according to a particular aspect of the present disclosure, depicting an example recurrent neural network for identifying trigger events.

[0077] Figure 57 is a schematic diagram according to a particular aspect of the present disclosure, depicting an example environment for touch detection and status classification using a set of recurrent neural networks.

[0078] Figure 58 is a schematic diagram according to a particular aspect of the present disclosure, depicting an electronic device with an ultrasonic input device.

[0079] Figure 59 is a schematic diagram according to a particular aspect of the present disclosure, depicting an automotive component with an ultrasonic input device.

[0080] Figure 60 is a schematic diagram according to a particular aspect of the present disclosure, depicting a keypad using an ultrasonic input device.

[0081] Figure 61 is a schematic diagram according to a particular aspect of the present disclosure, depicting a robotic arm using an ultrasonic input device.

[0082] Figure 62 is a schematic diagram according to a particular aspect of the present disclosure, depicting a piece of furniture using an ultrasonic input device.

[0083] Figure 63 is a set of diagrams according to a particular aspect of the present disclosure, depicting energy measurement signals of an ultrasonic input device showing material detection.

[0084] Figure 64Schematic diagram of a piezoelectric resonator array according to a particular aspect of the present disclosure, the piezoelectric resonator array including piezoelectric cantilevers that can be used in an ultrasonic input device.

[0085] Figure 65 Schematic diagram of a piezoelectric resonator array according to a particular aspect of the present disclosure, the piezoelectric resonator array including piezoelectric columns that can be used in an ultrasonic input device. Detailed Description

[0086] A touch input solution for improving the detection of touch input in an HMI is provided. An ultrasonic input device can utilize sensors located on the reverse side of a surface material to detect the presence of an object on any surface. The ultrasonic input device achieves a creative design without damaging the product skin or design materials (e.g., material stack). Such an ultrasonic input device can be implemented in various devices, e.g., input touch buttons, sliders, rollers, etc. The ultrasonic input device can be deployed under surfaces including various materials to simplify industrial design and appearance. Additionally, a grid of ultrasonic input device buttons can be implemented to create a keypad, mouse pad, or touch input at any location on any surface. The ultrasonic input device allows for touch input deployment on an HMI on surfaces including wood, leather, glass, plastic, metal (e.g., aluminum or steel), ceramic, plastic, a combination of one or more materials, etc.

[0087] In some cases, the ultrasonic input device can include an ultrasonic sensor coupled to a processor such as an application specific integrated circuit (ASIC) to provide a fully integrated system-on-chip (SOC) that can receive touch input via ultrasonic detection. In some cases, the ultrasonic sensor and the processor (e.g., ASIC) can be fabricated in a single wafer. The fully integrated SOC can provide many benefits such as low cost, low profile form factor, improved signal-to-noise ratio, and improved sensor array design freedom due to mass production via a wafer-level process.

[0088] In some cases, an ultrasonic input device may include an ultrasonic sensor that includes a microelectromechanical ultrasonic transducer (MUT), such as a piezoelectric microelectromechanical ultrasonic transducer (pMUT) or a capacitive microelectromechanical ultrasonic transducer (cMUT). By using MUTs in an ultrasonic input device as disclosed herein, optionally as part of a fully integrated system-on-a-chip (SOC), many benefits can be achieved. Since MUTs (due to their unique and predictable flexural mode shapes) generate signals that propagate more predominantly orthogonally to the transducer surface (longitudinal waves orthogonal to the surface) compared to other types of waves that propagate laterally, the use of MUTs can provide an improved energy transmission region. Since the predictable flexural mode shapes of MUTs are well separated from other modes (e.g., bulk modes) over a large frequency range, it is also less likely to generate or receive other types of acoustic waves, such as shear waves or surface waves that may travel laterally or orthogonally to the sensor surface. Thus, MUTs can achieve a more distinct transmission and sensing region on the surface material, such as a region directly perpendicular to the MUTs through the surface material. Additionally, the use of MUTs can reduce or minimize the amount of power required to operate the ultrasonic input device. For example, MUTs can be used with low parasitics, low drive voltages, and a small device capacitance that is approximately three orders of magnitude lower than the capacitance of traditional piezoelectric ceramic ultrasonic transducers.

[0089] The ultrasonic input device can detect a pattern associated with a touch input and distinguish between different types of touch inputs. Different types of touch inputs can vary between finger presses, palm presses, taps, touch and hold, or other such inputs. Each of the various types of touch inputs can have an identifiable and / or distinguishable pattern. In some cases, feedback from multiple sensors, such as multiple sensors arranged in an array, can be used to determine the type of touch input initiated. For example, a palm resting on the array of the ultrasonic input device can register an identifiable pattern across multiple ultrasonic input devices, and thus a processor coupled to the multiple ultrasonic input devices can determine that the touch input is a palm rest and take appropriate action (e.g., reject the palm rest as a touch input or initiate an action based on the palm rest).

[0090] An ultrasonic input device can detect a pattern associated with a touch input and distinguish between different users initiating the touch input. It has been found that different users of an ultrasonic input device will often produce recognizable and distinguishable signals when initiating a touch input. For example, the signal measured from a touch input can vary based on the user's finger, such as the moisture content of the finger, the dimensions of the ridges and valleys of the fingerprint, and other mechanical properties of the individual finger. Additionally, some users can initiate a touch input in a repeatable manner that can be used to identify the user. For example, a first user may typically tap the input device quickly, while a second user may typically place their finger on the input device and then press. As another example, different users may produce different touch pressures, which can also be detected by monitoring the amount of change in the ultrasonic signal. These factors such as the speed of the touch input and the manner of the touch input can be used to facilitate user identification.

[0091] In some cases, it can be determined whether a touch event has occurred by comparing an energy signal from an ultrasonic transducer with a threshold. In some cases, to improve the detection of touch events and the rejection of false positives, the threshold can be updated dynamically or automatically. The adaptive threshold can be updated based on any combination of the input energy signal and historical threshold data and trigger data (e.g., information about whether a touch event has occurred). These inputs can help update a threshold update function that can be used to filter the energy signal value into a new threshold. Thus, a particular change in the energy signal that does not typically indicate a touch event (e.g., a slow change) can be tracked by the adaptive threshold, while a change that indicates a touch event (e.g., a rapid change) can be not tracked by the adaptive threshold, which allows the energy signal to drop below the threshold and thus indicate a touch event.

[0092] In some cases, it can be determined whether a touch event has occurred by passing an energy signal into a recurrent neural network that has been trained based on training data. The recurrent neural network can convert the input energy signal into an output indicating whether a touch event has occurred.

[0093] In some cases, the state of the sensor (e.g., classification of the type of touch event, such as press, tap, double - tap, hold, or other such types) can be determined by analyzing trigger data. In some cases, the trigger data can be passed as an input into a recurrent neural network that has been trained based on training data for a particular state. The recurrent neural network can convert the input trigger data into an output indicating the sensor state.

[0094] In some cases, an ultrasonic input device can provide improvements to the aesthetic features and reliability of touch input detection on capacitive and mechanical devices. By defining button regions on a touch surface, buttons can be implemented on the surface. The ultrasonic input device can be embedded / placed behind the surface and thus limit environmental exposure (including dust and moisture), as well as reduce the manufacturing costs associated with special openings required to create other sensors on the surface. The ultrasonic input device can increase the flexibility of button programmability options. For example, a user can define the function of a button via a system controller, which can be embedded on a shared printed circuit board (PCB) along with the ultrasonic input device. In some embodiments, the system controller can monitor user behavior to improve machine / system preferences and performance. An ultrasonic input device that is mechanically coupled to a surface but positioned out of sight (such as below or behind an opaque surface) can be used to provide hidden input that is not observable or easily discoverable by those who are not already aware of its location. For example, the ultrasonic input device can be placed under a sign (e.g., on a laptop computer or another surface or device), behind a wall, or under the surface of a piece of furniture.

[0095] The ultrasonic input device can be low-power and / or battery-powered to operate for extended periods without the need for a direct connection to a main power source. The ultrasonic input device can be an Internet of Things (IoT) device or can be incorporated into an IoT device that is capable of providing sensor data (e.g., button presses) to other devices on a local or remote network. In some cases, the use of MUTs can allow the ultrasonic input device to operate with particularly low power requirements. In some cases, an ultrasonic input device that is a fully integrated system-on-a-chip (SOC) can operate at low power and / or can provide IoT functionality. I. Device Overview

[0096] Embodiments of the present invention relate to an ultrasonic input device for detecting touch input. Specifically, embodiments relate to an ultrasonic input device that includes a transducer coupled to a material layer that provides a surface to receive touch input signals to the system. The ultrasonic input device can be implemented using a variety of material layers, including wood, leather, glass, plastic, metal (e.g., aluminum, steel, etc.), stone, concrete, drywall, gypsum, paper, polymers, biomaterials (e.g., tissues such as skin), combinations of one or more materials, and the like. The flexibility in material selection enables the use of ultrasonic input devices in a variety of applications, including front and side buttons of mobile devices; steering wheels, infotainment units, center console controls, mirrors, seats, door handles, windows, etc. of vehicles; Internet of Things devices; medical devices such as bed controls, blood pressure measurement devices; input detection for robots, such as touch sensing for robotic fingers; and hidden input devices, such as those hidden inside furniture or behind walls. A. Detecting Touch Input Using Ultrasonic Signals

[0097] Figure 1 is a schematic diagram according to a particular aspect of the present disclosure that depicts the effect of touch on reflected ultrasonic signals in an ultrasonic input system. The ultrasonic input can include a transducer 104 coupled to a material layer 102. The material layer 102 can be referred to as a stack and can incorporate one or more sub-layers of one or more materials. For example, the stack can be a single piece of glass, a sheet of drywall, a laminated group of plastic and glass, or a plastic steering wheel wrapped in leather, etc. The material layer 102 has a first (inner) surface 106 and a second (outer) surface 108. The material layer can be characterized by the distance 110 between the first surface 106 and the second surface 108. The material layer 102 can be a covering material of a larger device that incorporates the ultrasonic input device. In some embodiments, the material layer 102 can form the body or a part of the body of the device. In these embodiments, the first surface 106 can form the inner surface of the body, and the second surface 108 can form the outer surface of the body. The second surface 108 can be considered external since it is exposed to the environment. The first surface 106 can be considered internal since it is not the surface to detect contact or since it is the surface to which the transducer 104 is acoustically coupled to the material layer 102. Figure 1 Shows an ultrasonic input device 120 without touch, an ultrasonic input device 122 with a light touch, and an ultrasonic input device 124 with a heavy touch.

[0098] The touch sensor is triggered based on the acoustic properties of the touch surface (material layer 102) and the input object 112. The detection of a light touch 122 depends on the degree of the reflected ultrasonic signal 114 in the material layer 102 relative to the absorbed ultrasonic signal 116 transmitted into the input object 112 through the second surface 108 of the material layer 102. As used herein, the reflected ultrasonic signal (e.g., the reflected ultrasonic signal 114) may refer to a signal that has reflected off the second surface 108 of the material layer 102, and the absorbed ultrasonic signal (e.g., the absorbed ultrasonic signal 116) may refer to a signal at least a portion of which has been absorbed by the input object 112 (e.g., a finger) contacting the second surface 108 of the material layer 102. The contact of the input object 112 on the touch surface (e.g., based on pressure) defines one or more contact areas 118 and the amount of reflection. The material layer 102 may be a single layer or may be composed of multiple layers of materials with different properties. For example, in some embodiments, the material layer 102 may be a uniform and isotropic material. In other embodiments, the material layer 102 may be a composite material layer composed of multiple layers of different materials. The threshold may be set based on the contact area 118 of the touch for triggering a button, the impedance difference between the input object 112 and the material layer 102, and the geometric and acoustic properties of the entire material stack of the material layer 102.

[0099] The size of the contact area 118 and the space between the contact areas 118 may indicate the size and spacing of the ridges of a finger and the size and spacing of the valleys of a finger fingerprint. A specific change in the size and / or spacing between the contact areas 118 may indicate different fingers contacting the material layer 102. For example, a younger individual may have smaller valleys (e.g., a smaller distance between the contact areas 118) than an older individual. In some cases, the detected size and / or spacing between the contact areas 118 may be used to detect or infer user contact with the material layer 102. Such inference may be used for application customization (e.g., causing different actions for different users for a touch event or having different sensing thresholds for different users), test authorization (e.g., allowing an action only when the identified user is initiating a touch event or the user touches the surface in a specific manner identical to a "password"), or performing other rule-based actions using the inference.

[0100] The heavy touch 124 can be distinguished from the light touch 122 by determining that the transducer 104 receives fewer reflected signals or fewer unattenuated signals due to an increase in the amount of the absorbed ultrasonic signal 126. If the touch pressure increases, for example, when the contact surface becomes flatter, the ultrasonic input device 100 and the input object 113 (e.g., a finger) will have a larger contact area 128. As Figure 1As shown, the larger contact area 128 increases the amount of absorbed ultrasonic signal 126 transferred through the second surface 108 of the material layer 102 into the input object 113. In the case of a user's finger, the larger contact area 128 may indicate that the ridges of the user's finger have flattened against the second surface 108 of the material layer 102. In some cases, where the input object 113 is not a finger or the finger is covered by another material, the larger contact area 128 may be the result of the texture elements of the input object 113 flattening against the second surface 108 of the material layer 102.

[0101] Figure 2 is a schematic diagram according to a particular aspect of the present disclosure, depicting an ultrasonic input system in a non-contact state and a contact state. Figure 2 An untouched ultrasonic input device 200 (e.g., non-contact state) and a touched (e.g., contact state) ultrasonic input device 250 are shown. The ultrasonic input device includes a transducer 202 coupled to a material layer 204. In this embodiment, the material layer 204 is shown as aluminum, but can be any material (e.g., glass, wood, leather, plastic, etc., or a composite material formed from a combination of materials). The transducer 202 is coupled to the first (inner) surface 206 of the material layer 204. The second (outer) surface 208 of the material layer 204 is in contact with air or some other environment having an acoustic impedance similar to that of a liquid different from that of a human finger.

[0102] For a contactless ultrasonic input device 200, the transducer 202 emits an ultrasonic signal 210A, which is directed into the material layer 204 and towards the second surface 208. Air has an acoustic impedance that is approximately zero and causes the second surface 208 to reflect the reflected ultrasonic signal 212A, which is close to 100% of the emitted ultrasonic signal (e.g., equal to or greater than 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.1%, 99.2%, 99.3%, 99.4%, 99.5%, 99.6%, 99.7%, 99.8%, 99.81%, 99.82%, 99.83%, 99.84%, 99.85%, 99.86%, 99.87%, 99.88%, 99.89%, 99.9%, 99.91%, 99.92%, 99.93%, 99.94%, 99.95%, 99.96%, 99.97%, 99.98%, and / or 99.99%). The reflected ultrasonic signal 212A itself can reflect off the first surface 206 to generate a reflection-emission signal 210B, which can reflect off the second surface 208 to cause a second reflected ultrasonic signal 212B. In the case of a composite material stack in 204, the signal reflected from 208 can reflect multiple times within the composite stack itself, and this echo chain can be sensed by the transducer 202. In the case of a composite material, analyzing the received echo chain formed by reflections between 206 and 208 and / or internal reflections within the multiple layers of 204 can be directly used to identify the material stack of 204 and / or the environment (e.g., air). This information can be used only to identify the acoustic and / or geometric properties of the stack or as additional information for sensor calibration and threshold tuning of detection algorithms. As Figure 2As depicted, four reflected ultrasonic signals 212A, 212B, 212C, 212D generate four corresponding reflection-emission signals 212B, 212C, 212D, 210E. Any number of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflection-emission signals 212B, 212C, 212D, 210E can be obtained from the initially transmitted ultrasonic signal 210A until these signals become too weak to be reflected and / or detected. Graph 214 shows a first amplitude 216 corresponding to the transmitted ultrasonic signal 210A and a set of subsequent amplitudes 218A, 218B, 218C, 218D, 218E corresponding to the reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E. Due to losses in the material layer 204, the first subsequent amplitude 218A is less than the first amplitude 216. Due to losses in the material layer 204, each of the remaining subsequent amplitudes 218B, 218C, 218D, 218E is less than the amplitude of the previous subsequent amplitudes 218A, 218B, 218C, 218D.

[0103] In some cases, one or more frequencies can be selected for use with an ultrasonic input device to achieve small or minimal attenuation in a non-contact state, thereby achieving a large or maximum number of reflected ultrasonic signals. In some cases, a set of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E resulting from a single transmitted ultrasonic signal 210A can be referred to as a string of reflected signals. For illustrative purposes, various reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflection-emission signals 210B, 210C, 210D, 210E are depicted from left to right in Figure 2 However, it will be understood that these signals are separated in time and may not necessarily be separated in space. Echo signals can be analyzed individually and / or combined or integrated with another echo signal as a detection metric analysis.

[0104] For an ultrasonic input device 250 with touch, the input object 220 (a finger in this case) contacts the second surface 208 of the material layer 204. The local reflection loss from the area contacted by the object (e.g., finger ridges) depends on how much the touch input medium differs from the input object in terms of acoustic impedance. For example, the reflection loss (dB) can be expressed as Where Z1 is the impedance of the material layer 204 and Z2 is the impedance of the input object 220. Once the input object 220 contacts the material layer 204, the transmitted ultrasonic signal 210A is split into two parts. The first part, the echo, is the reflected ultrasonic signal 213A and is reflected back to the transducer. The second part 222 is the transmitted signal that penetrates into the input object 220. The reflected ultrasonic signal 213A itself can be reflected off the first surface 206 to generate a reflection-transmission signal. The reflection-transmission signal itself can be split into two parts, one part is the second reflected ultrasonic signal 212B, and the other part is the second part 222 that penetrates into the input object 220. As Figure 2 depicted, four reflected ultrasonic signals 213A, 213B, 213C, 213D generate four corresponding reflection-transmission signals. Any number of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflection-transmission signals can be generated from the initially transmitted ultrasonic signal 210A until these signals become too weak to be reflected and / or detected.

[0105] As shown in the graph 224, the first amplitude 226 corresponds to the transmitted ultrasonic signal 210A. Compared with the touchless ultrasonic input device, since the second part 222 penetrates the input object 220, the first subsequent amplitude 228A corresponding to the reflected ultrasonic signal 213A decreases. Due to the losses in the material layer 204 and the internal multipath reflections in the case of the composite material stack 204, each of the remaining subsequent amplitudes 228B, 228C, 228D, 228E is smaller than the amplitude of the previous subsequent amplitudes 228A, 228B, 228C, 228D. For illustrative purposes, the graph 224 depicts the subsequent amplitudes 228A, 228B, 228C, 228D, 228E as solid lines, which overlap with the corresponding subsequent amplitudes 218A, 218B, 218C, 218D, 218E depicted as dashed lines. The total attenuation of the subsequent amplitudes 228A, 228B, 228C, 228D, 228E of the ultrasonic input device in the contact state can be greater than the total attenuation of the subsequent amplitudes 218A, 218B, 218C, 218D, 218E of the ultrasonic device in the non-contact state. In addition, the attenuation between each of the subsequent amplitudes 228A, 228B, 228C, 228D, 228E of the ultrasonic input device in the contact state can be greater than the attenuation of the subsequent amplitudes 218A, 218B, 218C, 218D, 218E of the ultrasonic device in the non-contact state.

[0106] Note that the subsequent amplitudes 228A, 228B, 228C, 228D, 228E associated with the touch event from the graph 224 decay faster than the corresponding subsequent amplitudes 218A, 218B, 218C, 218D, 218E associated with the no-touch event from the graph 214. In other words, the contrast between the subsequent amplitudes of the touch event and the subsequent amplitudes of the no-touch event increases with each subsequent reflection number n. In some cases, the ratio of the nth subsequent amplitude associated with the no-touch event to the nth subsequent amplitude associated with the touch event can be Γ n :(1 - Γ n ), where Γ is the percentage of the signal reflected back from the second surface 208. For example, the ratio of the subsequent amplitude 218A to the subsequent amplitude 228A can be 100:90; the ratio of the subsequent amplitude 218B to the subsequent amplitude 228B can be 100:81; the ratio of the subsequent amplitude 218C to the subsequent amplitude 228C can be 100:72; the ratio of the subsequent amplitude 218D to the subsequent amplitude 228D can be 100:63; and the ratio of the subsequent amplitude 218E to the subsequent amplitude 228E can be 100:54. B. Ultrasonic touch input device

[0107] Figure 3 An ultrasonic input device according to a particular aspect of the present disclosure is shown. The ultrasonic input device 300 can be attached to any surface to detect touch input. The ultrasonic input device 300 can include a sensor 302, such as a piezoelectric micromachined ultrasonic transducer (PMUT). The PMUT transducer is a piezoelectric ultrasonic transducer that includes a thin film coupled to a thin piezoelectric film to induce and / or sense ultrasonic signals. The sensor 302 can be integrated on a dedicated integrated circuit (ASIC), such as a CMOS (complementary metal oxide semiconductor) ASIC 304 (integrated), and formed on a substrate 306. The ASIC 304 can include circuits and / or modules that can be used to perform various processes disclosed herein, such as at least the various analog and / or digital processing referred to Figure 25 - 41 above. For example, the ASIC 304 can be used to drive the sensor 302, detect the reflected ultrasonic signal using the sensor 302, and determine the amplitude associated with the reflected ultrasonic signal (e.g., using various analog techniques such as summation and integration). In some cases, the ASIC 304 can optionally determine a threshold, and the determined amplitude can be compared with the threshold to determine whether a touch event has occurred. In such a case, the ASIC 304 can output a signal associated with the occurrence of the touch event.

[0108] In some cases, the circuitry of the ASIC 304 can perform certain processes in an analog manner, such as signal rectification, integration, mixing, modification, accumulation, etc. As used herein, an analog circuit can include any circuitry capable of performing an action (e.g., rectification, integration, etc.) on an analog signal without first digitizing the analog signal. In one example, the ASIC 304 can include an analog circuit capable of acquiring a received ultrasonic signal, rectifying the signal, and integrating at least a portion of the rectified signal to provide an integrated signal, such as described with reference to Figure 26 In another example, the ASIC 304 can include an analog circuit capable of acquiring a received ultrasonic signal, calculating the absolute value of the signal, and accumulating the absolute value to provide an accumulated signal, e.g., as described with reference to Figure 28 In another example, the ASIC 304 can include an analog circuit capable of acquiring a received ultrasonic signal, squaring the signal through self-mixing, and integrating the squared signal to provide an integrated signal, such as described with reference to Figure 29

[0109] In some cases, different forms of ultrasonic transducers can be used for the sensor 302 instead of a PMUT sensor. In some cases, the ultrasonic sensor can be formed using a deposited layer of a piezoelectric material (e.g., aluminum nitride, lead zirconate titanate (PZT), or polyvinylidene fluoride (PVDF)). In some cases, the ultrasonic sensor can be a capacitive micromachined ultrasonic transducer (CMUT). In some cases, the ultrasonic sensor can be an array of resonators of a piezoelectric device (e.g., a piezoelectric cantilever or a piezoelectric column).

[0110] The substrate 306 can be bonded 310 to a flexible printed circuit / printed circuit board 308 (FPC / PCB) of a larger integrated device such as a mobile phone. In some embodiments, the contact area 312 on the sensor 302 can be bonded to the substrate contacts 314. As shown, the size of the ultrasonic input device 300 can be equal to or less than 1.5 mm x 1.5 mm x 0.5 mm, although other sizes can also be used. In some cases, the FPC / PCB 308 to which the substrate 306 is attached can receive information associated with the amplitude of the detected reflected ultrasonic signal and perform some of the functions disclosed herein, such as determining a threshold and / or determining when a touch event has occurred. However, in some cases, the FPC / PCB 308 only receives a signal associated with the occurrence of a touch event and thus does not need to perform further analysis of the amplitude of the detected reflected ultrasonic signal based on the touch event to perform an action.

[0111] ​The integration of the ASIC 304 and the sensor 302 enables a small form factor, which results in the placement of buttons or other functions in many space-constrained applications. For example, a mechanical button on the side of a smart phone can be easily replaced by the ultrasonic input device 300 under the casing. To implement a touch interface or other suitable function of the system, the ultrasonic input device 300 can be bonded to the surface 316 using the adhesive 318.

[0112] Figure 4 is a cross-sectional view of two piezoelectric micromachined ultrasonic transducers integrated into a CMOS wafer according to a particular aspect of the present disclosure. Device 400 shows a cross-sectional view of two PMUTs bonded to a CMOS wafer 402, which can be used in an ultrasonic input device. Each PMUT can be formed on a MEMS wafer 401 bonded to the CMOS wafer 402. In this way, the PMUT can be coupled to the necessary processing electronics of the CMOS wafer 402. It should be understood that each PMUT can have an active piezoelectric layer 404 and a first electrode 403 and a second electrode 405. The first electrode 403 and the second electrode 405 can be electrically coupled to the piezoelectric layer 404.

[0113] In some embodiments, the PMUT can include a first contact 422 electrically coupled to the first electrode 403, a second contact 424 electrically coupled to the second electrode 405, and a third electrode 426 electrically coupled to the CMOS wafer 402. Applying an alternating voltage across the first electrode 403 and the second electrode 405 can cause movement (e.g., bending motion) of the piezoelectric layer 404, which can result in the generation of acoustic waves. Similarly, received acoustic waves that cause movement in the piezoelectric layer 404 can be sensed as a varying voltage across the first electrode 403 and the second electrode 405. One or more vias (vertical interconnect channels) 410 can be formed in the PMUT. Each contact can be wire-bonded to an electronic board. In some embodiments, the PMUT can include a passivation layer 428 formed on the surface 420 and the contacts. An adhesive coupling surface 430 on the surface 420 or the passivation layer 428 can be coupled to a material layer of the ultrasonic input device.

[0114] In some embodiments, the passive electrical layer 408 can include SiO 2or any other suitable passive layer. The active piezoelectric layer 404 may be aluminum nitride that is approximately 1 μm thick, and the passive elastic layer may be single-crystalline silicon that is approximately 1 μm thick, but other sizes and materials may be used. In some embodiments, the active piezoelectric layer 404 may be scandium-doped aluminum nitride. Alternatively, the active piezoelectric layer 404 may be another suitable piezoelectric ceramic, such as PZT. Both the top and bottom electrodes 406 may include molybdenum. To bond the PMUT to the top metal 412 of the CMOS wafer 402, fusion bonding via through-silicon vias (TSVs) as shown at the vias 410 may be used. This method results in significant parasitic reduction, which in turn leads to improved signal integrity and lower power consumption.

[0115] In some embodiments, the chamber 414 may be formed with a vacuum or near-vacuum to isolate the transducer from the processing electronics in the CMOS wafer 402. The sound generated by the PMUTs will not propagate through the near-vacuum of the chamber 414, minimizing reflections and interference that may be caused by the material interface with the CMOS wafer 402. The chamber 414 may cause the ultrasound 416 to travel away from the PMUT. The ultrasound 416 may travel through the adhesive coupling surface 430 and into the material layer of the ultrasound input device. The material layer may reflect the ultrasound 416, causing a return echo to reflect back to the PMUT. The return echo travels through the adhesive coupling interface and is received by the PMUT.

[0116] In some embodiments, the CMOS wafer 402 may be an application-specific integrated circuit (ASIC) that includes one or more devices required to drive the transducer. The drive voltage for the PMUT array may be less than 4 volts. In some cases, the drive voltage may be less than 1.8 volts. In some cases, the drive voltage may be or less than 4, 3.5, 3, 2.5, 2, 1.9, 1.8, 1.7, 1.6, or 1.5 volts. The ASIC may be fabricated to meet the size requirements associated with the size of the associated PMUT. In some embodiments, the ASIC may include one or more modules to receive the measured signal. The ASIC may be configured to further process the signal. For example, the ASIC may include one or more rectifiers to generate an absolute value signal by obtaining the absolute value of the received signal (which may be an alternating current). The ASIC may also include an integrator and an analog-to-digital converter (ADC) to convert the reflected ultrasound signal into a digital representation of the reflected signal. The integration of the ASIC and the PMUT further allows for the embedding of a gain amplifier and an ADC in the ASIC and eliminates a separate ADC sensor controller chip. This opens up space on the associated circuit board and reduces the implementation cost of the touch input sensor. In some embodiments, the ASIC may transmit the digital signal to at least one or more of a memory, a processor, and a remote device. In other embodiments, the ASIC may include one or more signal processing modules.

[0117] The PMUT array can be compatible with the CMOS semiconductor process. In some embodiments, the PMUT materials and dimensions can conform to the Semiconductor Equipment and Materials International (SEMI) standard specifications. Since the PMUT can conform to the SEMI specifications, the transducer array can be used with existing CMOS semiconductor manufacturing tools and methods. For example, one or more PMUTs can be formed using photolithography techniques. In contrast, current piezoelectric ultrasonic transducer arrays are formed using a wafer saw that cannot match the precision of photolithography techniques. As a result, the PMUT can be smaller, operate at lower voltages, and have lower parasitics. C. Integration with a circuit board

[0118] Figure 5 A set of schematic diagrams 502, 504, 506, 508 in accordance with a particular aspect of the present disclosure depicts ultrasonic input devices 510, 512, 514, 516 coupled to various surfaces. FIG. 502 depicts an ultrasonic input device 510 coupled to a metal surface by an adhesive. FIG. 504 depicts an ultrasonic input device 512 coupled to a glass surface by an adhesive. FIG. 506 depicts an ultrasonic input device 514 coupled to a plastic surface by an adhesive. FIG. 508 depicts an ultrasonic input device 516 coupled to a wooden surface by an adhesive. Any suitable material can be used as the sensing surface, such as a non-porous material or a semi-porous material. Porous materials can be used for the sensing surface, but better results can be obtained with smaller pores, higher density, and more consistent density.

[0119] In addition, the ultrasonic input devices 510, 512, 514, 516 can be coupled to a flexible PCB, such as on the opposite side of the ultrasonic input devices 510, 512, 514, 516 coupled to the sensing surface. The ultrasonic input devices 510, 512, 514, 516 can serve as a mechanical coupler between the sensing surface and the PCB, where the PCB is not otherwise attached to the sensing surface, although this is not always required. In some cases, a flexible PCB can be used.

[0120] The use of the PCB can allow additional components to be integrated with the ultrasonic input devices 510, 512, 514, 516 to expand the functionality of the ultrasonic input devices 510, 512, 514, 516, such as described with reference to Figure 6 as described.

[0121] Figure 6is a schematic side view according to a particular aspect of the present disclosure, depicting an ultrasonic input system 600 having a shared board assembly. The ultrasonic input system 600 may include an ultrasonic input device 602 electrically coupled to a circuit board 610, and any number of shared board assemblies 612. Each shared board assembly may be electrically coupled to the circuit board 610. In some cases, the ultrasonic input device 602 may be mechanically coupled to the circuit board 610, such as using an electrical connection (e.g., solder joints) or other mechanical supports. In some cases, one, some, or all of the shared board assemblies 612 may be mechanically coupled to the circuit board 610. In some cases, the circuit board may be a printed circuit board, such as a flexible PCB, although this need not always be the case.

[0122] The entire ultrasonic input system 600 may be contained within a single shared housing, multiple housings, or may not be contained within a housing. In some cases, two or more shared board assemblies 612 may be contained within a single housing, with or without the ultrasonic input device 602. In some cases, all of the shared board assemblies 612 may be located on the same side of the circuit board 610 as the ultrasonic input device 602, although this need not always be the case. When located on the same side as the ultrasonic input device 602, the shared board assemblies 612 may be selected or designed to have a height greater than the height of the ultrasonic input device 602.

[0123] In some cases, the ultrasonic input system 600 may include a power component 604. The power component 604 may provide power to the ultrasonic input device 602 and / or any other shared board assemblies 612. Examples of the power component 604 include a battery, a transformer (e.g., a transformer coupled to a main line), a capacitor (e.g., a supercapacitor), a solar cell, a fuel cell, and / or any other suitable power source.

[0124] In some cases, the ultrasonic input system 600 may include a processor 606. The processor 606 may enable various processing functions to be performed within the ultrasonic input system 600 based on signals received from the ultrasonic input device 602. Examples of suitable processors 606 include a microcontroller, a central processing unit, or other suitable devices. The processor 606 may also be coupled to a memory to access processing routines, access stored data, and / or store data.

[0125] In some cases, the ultrasonic input system 600 may include a communication component 608. The communication component 608 may interact with the ultrasonic input device 602 and / or the processor 606 to send signals to or receive signals from an external device. Examples of suitable communication components 608 include wireless radios (e.g., Bluetooth, WiFi, Zigbee, Z-Wave, etc.), audio devices (e.g., microphones or speakers), visual devices (e.g., cameras, lights, or displays), haptic devices (e.g., haptic feedback devices such as motors and vibrators), or other devices suitable for sending or receiving signals.

[0126] In some cases, the ultrasonic input system 600 may include a common board component 612, which includes a power component 604, a processor 606, and a communication component 608. In some cases, the ultrasonic input system 600 may include more or fewer common board components, which include different types of components. D. Example System Settings

[0127] Figure 7 is a schematic diagram according to a particular aspect of the present disclosure, which depicts an example ultrasonic input system 700. The ultrasonic input system 700 may include an ultrasonic sensor 702 and a processor 722. The ultrasonic sensor 702 may be the same as Figure 1 the transducer 104, the processor 722 may be electrically connected to the ultrasonic sensor 702, and may optionally be mechanically connected to the ultrasonic sensor 702. In some cases, the processor 722 and the ultrasonic sensor 702 may be integrated into the same package, although this need not always be the case. The processor 722 may perform specific functions as disclosed herein, such as obtaining signals from the ultrasonic sensor 702 and / or detecting touch events. In some cases, an optional computing device 724 may be connected to the processor 722 to exchange information, such as information related to touch events, information related to signals from the ultrasonic sensor 702, or information related to how the processor 722 interprets signal information. A data storage 726 may be connected to the processor 722 for storing information, such as information related to how the processor 722 interprets signal information. In some cases, the optional computing device 724 may be connected to a data storage 728 that may store information, such as information related to how to interpret signal information from the ultrasonic sensor 702 to determine touch events. The computing device 724 may be any suitable computing device, such as a desktop computer, a laptop computer, a server, a smart phone, a tablet computer, or any other suitable computing device. The computing device 724 may be connected to the processor 722 by a wired or wireless connection. The computing device 724 may be connected to the processor 722 by a local or remote connection.

[0128] In some cases, the processor 722 can be an application specific integrated circuit (ASIC). In some cases, the ultrasonic sensor 702 can be a MUT. The processor 722 can be any suitable circuitry designed to be capable of driving and receiving one or more transducers of the sensor 502. The processor 722 can drive the transducers to send and receive ultrasonic signals to implement the touch sensing capabilities described herein. In some cases, the processor 722 can output a measured energy level (e.g., an energy signal) associated with the sensor 502, which can later be used to determine whether a touch event has occurred. In some cases, the processor 722 can output a touch signal indicating the occurrence of a touch event. In such a case, the processor 722 can perform the necessary processing to determine whether a touch event has occurred. In some cases, the processor 722 can further perform the necessary processing to determine additional information associated with the touch event, such as whether the touch event was initiated by a bare finger or a gloved finger, whether the touch event was initiated by a first user or a second user, or other aspects of the touch event. Such additional information can take the form of an inference and can have different levels of confidence, although this need not always be the case. In some cases, the processor 722 can have the ability to process signals and identify the type of pattern (e.g., a click, a double click, a hold, etc.) that the user is inputting. This ability in the processor 722 can be implemented by a hardware processing block or can be written into the chip memory as part of the firmware. In some cases, the processor 722 can have the ability to self-calibrate and tune its parameters for signal identification and pattern recognition.

[0129] In some cases, the processor 722 can send the energy signal and / or the touch signal to the computing device 724. The computing device 724 can perform the necessary processing to determine whether a touch event has occurred and / or additional information associated with the touch event, such as whether the touch event was initiated by a bare finger or a gloved finger, whether the touch event was initiated by a first user or a second user, or other aspects of the touch event.

[0130] In some cases, the data store 726 can store information regarding how the processor 722 determines whether a touch event has occurred or other information associated with determining the touch event. In some cases, the data store 726 can store model information used by the processor 722 to process the energy signal and determine whether a touch event has occurred. In some cases, the model information stored in the data store 726 can be provided by and / or updated using the computing device 724. II. Fully Integrated On-Chip System for Ultrasonic Touch Input

[0131] Embodiments of the present disclosure allow for a fully integrated system-on-chip for ultrasonic touch input. For example, an integrated ultrasonic input device may include an ultrasonic sensor and an application-specific integrated circuit (ASIC). Various production techniques may be used to create the integrated ultrasonic input device wafer to allow for a low profile size as well as improved noise resistance and lower power. A. Overview of the Integrated Ultrasonic Input Device

[0132] Figure 8 is a schematic side view according to a particular aspect of the present disclosure, depicting an integrated 820 having an ultrasonic sensor 802 and an application-specific integrated circuit (ASIC) 822. The ultrasonic sensor 802 may be composed of one or more ultrasonic transducers arranged in an array. In some cases, the ultrasonic transducer is a MUT.

[0133] The ASIC 822 may be any suitable circuit designed to be able to drive and receive one or more transducers of the ultrasonic sensor 802. The ASIC 822 may drive the transducers to transmit and receive ultrasonic signals to achieve the touch sensing capabilities described herein. In some cases, the ASIC 822 may output the measured energy level associated with the ultrasonic sensor 802, which may later be used to determine whether a touch event has occurred. In some cases, the ASIC 822 may output a touch signal indicating the occurrence of a touch event. In such a case, the ASIC 822 may perform the necessary processing to determine whether a touch event has occurred. In some cases, the ASIC 822 may further perform the necessary processing to determine additional information associated with the touch event, such as whether the touch event was initiated by a bare finger or a gloved finger, whether the touch event was initiated by a first user or a second user, or other aspects of the touch event. Such additional information may take the form of an inference and may have different levels of confidence, although this need not always be the case. In some cases, the ASIC 822 may have the ability to process signals and identify the type of pattern (e.g., click, double-click, hold, etc.) that the user is inputting. This ability in the ASIC 822 may be implemented by hardware processing blocks or may be written into the chip memory as part of the firmware. In some cases, the ASIC 822 may have the ability to self-calibrate and tune its parameters for signal recognition and pattern identification.

[0134] The integrated ultrasonic input device 820 can be fully or partially encapsulated within a housing 824 to form a package. The housing 824 can take the form of any suitable material, such as a cured resin. In some cases, the housing 824 contains only the ultrasonic sensor 802 and the ASIC 822, as well as any electrical contacts necessary to couple the ASIC 822 to external components. In some cases, the housing 824 can contain additional components, such as additional sensors (e.g., thermal sensors, vibration sensors, or gyroscopes). In some cases, the material used for the housing 824 can be selected to perform well as part of a stack of the ultrasonic input system. For example, a material that has maximum energy transfer in the frequency range associated with a particular ultrasonic input device 820 can be used to maximize the signal. In some cases, additional materials can be used within the housing 824 or incorporated into the housing 824 itself to achieve a desired response for ultrasonic propagation into the stack. For example, a window can be fitted into the housing 824 adjacent to the ultrasonic sensor 802 to provide a path for transmitting ultrasonic signals to and from the ultrasonic sensor 802. The window can be made of optically transparent, translucent, or opaque material and can be selected to allow ultrasonic signals to pass through with little or no attenuation. Also, materials can be used in the stack to enhance the acoustic matching between layers to facilitate the transmission and / or reception of signals.

[0135] In some cases, the housing 824 can be applied after the ultrasonic sensor 802 and the ASIC 822 have been formed as a wafer and diced into individual chips. However, in some cases, the housing 824 can be applied while the ultrasonic sensor 802 and the ASIC 822 are still part of a wafer containing many chips. Any suitable chip packaging method can be used to package the ultrasonic sensor 802 and the ASIC 822.

[0136] In some cases, other types of processors or circuitry can be used in place of the ASIC 822. For example, a general-purpose programmable processor can be used in place of the ASIC 822 while still achieving many of the benefits associated with the integrated ultrasonic input device 820. In some cases, the ASIC 822 can receive power as an input, which can be used to power the ASIC 822 itself and drive the transducer of the ultrasonic sensor 802. In some cases, a general-purpose programmable processor can be used to communicate between multiple chips with or without internal ASICs in a master and slave configuration.

[0137] In some cases, the height of the package of the integrated ultrasonic input device 820 can be approximately 500 microns or less. In some cases, the ultrasonic sensor 802 and the ASIC 822 of the integrated ultrasonic input device 820 can have a combined height of approximately 150 microns or less. B. Manufacturing Techniques

[0138] Figure 9 is a schematic diagram of a set of combinations according to a particular aspect of the present disclosure, which depicts two processing routes 926, 928 for generating an integrated ultrasonic input device wafer 930. The first processing route 926 depicts generating the wafer 930 using a monolithic technique. The second processing route 928 depicts generating the wafer 930 using a wafer bonding technique. Any suitable process can be used to generate the wafer 930 containing the sensors and ASICs as described herein.

[0139] Below the first processing route 926, an ASIC wafer 932 is provided, and subsequently a sensor layer 934 is built on the ASIC wafer 932 to produce a monolithic wafer 930 containing both the sensor and the ASIC. This type of wafer-level manufacturing can allow for the generation of small form factors in an economical manner.

[0140] Below the second processing route 928, a sensor layer 934 is provided, and an ASIC wafer 932 is provided. Subsequently, the provided sensor layer 934 can be bonded to the ASIC wafer 932 using any suitable wafer bonding technique, with or without an intermediate layer.

[0141] The wafer 930 produced by the first processing route 926, the second processing route 928, or any other suitable processing route can include one or more instances of sensors and ASICs that can be used to create an integrated ultrasonic input device.

[0142] Figure 10 is a set of schematic diagrams depicting a single integrated ultrasonic input device 1020 cut from a wafer 1030, PCB-mounted, and stack-mounted according to a particular aspect of the present disclosure. The wafer 1030 can be Figure 10 the wafer 1030, and the wafer 1030 can be cut or diced into many pieces (e.g., chips). Each chip 1042 can contain a sensor 1002 and an ASIC 1022 for a single integrated ultrasonic input device 1020. If a housing has not been applied to the wafer 1030 previously, each chip 1042 can be encapsulated in a housing to produce the integrated ultrasonic input device 1020.

[0143] The ultrasonic input device 1020 can be mounted on a printed circuit board (PCB) 1036 or otherwise electrically connected to any other necessary electronic components. For example, in some cases, the ultrasonic input device 1020 can be electrically connected to a battery or other power source. In some cases, the ultrasonic input device 1020 can be mounted on a PCB 1036 containing other electronic components 1038 (such as a processor and a power source).

[0144] The ultrasonic input device 1020 can be mounted on a substrate 1040. The substrate 1040 can be any combination of one or more materials through which ultrasonic signals can be transmitted to the sensor 1002. The housing of the ultrasonic input device 1020 can be coupled to the substrate 1040. The combination of materials through which ultrasonic signals are transmitted from the outer surface of the substrate 1040 to the sensor 1002 can be referred to as a stack, which can include the housing of the ultrasonic input device 1020. The ultrasonic input device 1020 can be coupled to the substrate 1040 using any suitable technique, including using adhesives, mechanical coupling, active pressure, or any other suitable technique for acoustically coupling the ultrasonic input device 1020 and the substrate 1040. C. Low profile

[0145] Figure 11 FIG. is a schematic cross-sectional view of a consumer electronic product 1100 incorporating an integrated ultrasonic input device 1120 in accordance with a particular aspect of the present disclosure. The consumer electronic product 1100 can be a smart phone or any other suitable device. The integrated ultrasonic input device 1120 can be attached to a substrate composed of one or more layers of a display 1140, or to any other part of the consumer electronic device, such as a frame or a back surface (1144) that can be made of metal, plastic, or other materials. The display 1140 can include multiple layers, including a display layer, an illumination layer, a protective layer, a sensing layer, and other suitable layers. Through its coupling with the display 1140, the integrated ultrasonic input device 1120 can be used to register touch events associated with the display 1140. However, in some cases, the integrated ultrasonic input device 1120 can be coupled to any surface of the consumer electronic product to detect touch events on the opposite side of that surface, such as the back side or the side edge of the consumer electronic product.

[0146] As described herein, the integrated ultrasonic input device 1120 can be formed to have a very small height, such as equal to or less than 500 microns. Due to the low profile of the integrated ultrasonic input device 1120, one or more such integrated ultrasonic input devices can be easily positioned within the consumer electronic product 1100, leaving sufficient space for other components. For example, the low profile of the integrated ultrasonic input device 1120 can only occupy a small portion of the total height of the consumer electronic product 1100, thereby allowing more space for other components, such as a larger battery 1144 with a greater capacity, or a more open space for air flow. Additionally, due to the physical properties underlying the design and operation of the device, the integrated ultrasonic input device can be made to operate in a small local area for transmitting and receiving ultrasonic information. This local operation greatly improves the performance robustness of the device against interference sources (such as touches or holds) that occur outside the operating area. D. Improved noise resistance and lower power

[0147] Figure 12 A set of schematic cross-sectional views comparing the non-integrated ultrasonic input device 1200 with the integrated ultrasonic input device 1220 according to a particular aspect of the present disclosure. The non-integrated ultrasonic input device 1200 is more susceptible to noise at least in part due to the relatively long lengths of the wires required to connect the ASIC and the sensor. For example, the non-integrated ultrasonic input device 1200 may have exposed electrical traces. Not only is power used to transmit signals along the electrical trace, but the electrical trace may also be further susceptible to interference. Accordingly, the overall signal-to-noise ratio of the non-integrated ultrasonic input device is relatively low. If a higher signal-to-noise ratio is required, the ASIC must provide more power to drive the sensor, in which case the entire system will have a relatively higher power consumption.

[0148] In contrast, the integrated ultrasonic input device 1220 of the present disclosure is an integrated chip encapsulated in a housing. The integrated ultrasonic input device 1220 does not have large exposed traces or wires between the sensor and the ASIC. Accordingly, at least in part due to the minimal conductive traces between the sensor and the ASIC, there is little or no risk of interference and little or minimal energy depletion when transmitting signals from the sensor to the ASIC. Thus, the integrated ultrasonic input device 1220 is capable of operating with an improved signal-to-noise ratio and / or with improved power efficiency compared to a similar non-integrated ultrasonic input device 1200. III. Ultrasonic Sensor Design

[0149] The ultrasonic input device may include a plurality of transducers, which may be configured as, for example, a sensor array. In some embodiments, the plurality of transducers may allow measurements at multiple frequencies. Additionally, in other embodiments, the plurality of transducers may allow separation of transmit and receive capabilities. For example, some transducers may be configured to transmit ultrasonic signals, while other transducers of the plurality of transducers may be configured to receive ultrasonic signals. In additional embodiments, the plurality of transducers may allow beamforming. A. Transducer Array

[0150] Figure 13A A top view of a sensor array 1302 of an ultrasonic input device according to a particular aspect of the present disclosure. The sensor array 1302 may include one or more transducers 1350 (e.g., MUTs). Generally, the sensor array 1302 may have a plurality of transducers 1350. Figure 13AThe sensor array 1302 is described as having 144 different transducers 1350 that pass through a sensor array 1302 that is approximately 1.2 mm square, although other numbers of transducers 1350 and other sizes of arrays can be used. The various electrical traces in the sensor array 1302 can interconnect the different transducers 1350 with the ASIC. Each transducer 1350 can be independently addressable. In some cases, the use of the transducers 1350 for a particular purpose (e.g., as a transmitter or receiver, or having certain specific frequencies) can be set or changed by the ASIC, so that each transducer 1350 can perform any particular function performed by any other transducer 1350 of the sensor array 1302. However, in some cases, one or more of the transducers 1350 can be specifically selected or configured to perform a particular function more efficiently or effectively. For example, some of the transducers 1350 can be designed to achieve improved transmission, while other transducers 1350 can be designed to achieve improved reception.

[0151] Figure 13B is a top view of an alternative sensor array 1312 of an ultrasonic input device according to a particular aspect of the present disclosure. The sensor array 1312 can include one or more transducers 1360. The sensor array 1312 depicts a sensor array that includes 36 ultrasonic transducers 1360. The various electrical traces in the sensor array 1312 can interconnect the different transducers 1360 with the integrated circuit layer. One or more of the transducers 1360 of the sensor array 1312 can be transmitting ultrasonic transducers. One or more of the transducers 1360 of the sensor array 1312 can be receiving ultrasonic transducers. As described herein, the transducers 1360 of the sensor array 1312 can transmit and receive at any suitable frequency. The relative size of the transducer 1360 can indicate the frequency that can be transmitted / received by the transducer.

[0152] The various electrical traces (not shown) in the sensor array 1312 can interconnect the different transducers 1360 with the integrated circuit. The various electrical traces can interconnect the different transducers 1360 in any suitable manner. For example, the electrical traces can connect the transducers 1360 in a horizontal and vertical grid. As another example, the electrical traces can connect the transducers 1360 that are diagonally positioned with respect to each other.

[0153] Figure 14A is according to a particular aspect of the present disclosure Figure 13ASchematic diagram of the sensor array 1302, which depicts an example configuration of the respective transducers of the sensor array 1302. In this example configuration, in the sensor array 1402, out of 144 different transducers, 60 are set to operate as low-frequency transmitters, 8 are set to operate as low-frequency receivers, 56 are set to operate as high-frequency transmitters, and 20 are set to operate as high-frequency receivers. In Figure 14A The configuration depicted in

[0154] Figure 14A It may be particularly useful for sensing touch events using multiple ultrasonic frequencies to, for example, better identify environmental changes relative to a true touch event and / or improve the operating frequency bandwidth of the device so that the device is more responsive over a wider frequency range. Further shows a sensor array 1402 including four corner regions. The corner regions of the sensor array 1402 may include multiple transducers. For example, the sensor array 1402 includes four rotationally symmetric corner regions, which mainly include (e.g., mostly) low-frequency transmitting ultrasonic transducers that may surround low-frequency receiving ultrasonic transducers. The corner regions of the sensor array 1402 include 16 ultrasonic transducers in a 4x4 array. However, it should be understood that the corner regions of the sensor array 1402 may include up to one quarter of the total number of ultrasonic transducers included in the sensor array 1402. For example, a square sensor array including 81 transducers may include four corner regions. Each of the four corner regions may include a transducer grid of 1x1, 2x2, 3x3, or 4x4. In some cases, the sensor array 1402 may include high-frequency transmitting ultrasonic transducers surrounding low-frequency transmitting ultrasonic transducers, for example, as shown. In some implementations, the high-frequency transmitting ultrasonic transducers may not be on the diagonal of the corner region, but there may be high-frequency receiving ultrasonic transducers, for example, as Figure 14A shown. Additionally, the central region may mainly include low-frequency transmitting ultrasonic transducers. The central region may be surrounded by high-frequency transmitting ultrasonic transducers. In some cases, the central region may include transmitting ultrasonic transducers. In other cases, the central region may include receiving ultrasonic transducers. The central region of the sensor array may have any suitable size, for example, 1x1, 2x2, 3x3, 4x4, 5x5, 6x6, 7x7, etc.

[0155] In some cases, the sensor array 1302 may have any number of transducers operating at any number of different frequencies. Although Figure 13AThe example configurations may be useful in some cases, but other configurations may be used. In some cases, a single type of sensor array 1302 can be mass-produced and used with the same or different types of ASICs. For example, different types of ASICs can be configured to operate the same sensor array 1302 in different configurations (e.g., with more or fewer transmitters or receivers, different frequencies, or more or fewer different frequencies). In some cases, the same type of ASIC can also be programmed to operate in different configurations. In some cases, an integrated version of the transducer and / or ASIC can be used in combination with a non-integrated transducer and / or ASIC to achieve a specific purpose, such as increasing the transmission power.

[0156] In Figure 13A and Figure 14A The sensor arrays depicted may include one or more piezoelectric micromachined ultrasonic transducers, one or more capacitive micromachined ultrasonic transducers, one or more integrated bulk piezoelectric transducers, or one or more non-integrated bulk piezoelectric transducers. In some cases, the sensor array may include any suitable combination of the above transducers. Additionally, the sensor array may have any suitable size. For example, the sensor array may include an array of ultrasonic transducers such as 2x2, 3x3, 5x5, 9x9, 16x16, etc. For example, sensor array 1312 depicts a sensor array of 6 ultrasonic transducers by 6 ultrasonic transducers.

[0157] Figure 14B is a schematic diagram of a sensor array according to a particular aspect of the present disclosure, which depicts eight example configurations of the individual transducers in the corner region of the sensor array. In Figure 14B the depicted sensor array is 12x12 transducers in size and the corner region is 4x4 transducers in size. However, it should be understood that the embodiments may include sensor arrays and corner regions of any suitable size.

[0158] Each corner region of sensor arrays 1410-1424 may include a transmit ultrasonic transducer 1426 and a receive ultrasonic transducer 1425. The transmit ultrasonic transducer and the receive ultrasonic transducer may be arranged as in sensor arrays 1410-1424. Thus, in various combinations, the receive transducers may be diagonal to each other, where the diagonal may be in various positions and have various lengths. The receive transducers may be in blocks (e.g., 2x2) in various positions and may be other shapes, which include an odd number of receive transducers.

[0159] For example, the transmit ultrasonic transducers in the sensor array 1410 can transmit at the same frequency respectively. Similarly, the receive ultrasonic transducers in the sensor array 1410 can receive at the same frequency respectively, which can be the same frequency transmitted from the transmit ultrasonic transducers. The corner regions do not have to be all the same and can occur in various combinations. For example, the combination can have one type selected from 1410, 1412, 1414, and 1416. The inner region can have various combinations shown in Figure 14C as shown.

[0160] Figure 14C is a schematic diagram of a sensor array according to a particular aspect of the present disclosure, which depicts eight example configurations of the respective transducers of the sensor array. In Figure 14C the depicted sensor array depicts transducers inside the corner regions of the sensor array. The inner region can include transducers between at least two corner regions. The transducers shown in the sensor array can include a transmit ultrasonic transducer 1446 and a receive ultrasonic transducer 1445. The transmit ultrasonic transducer and the receive ultrasonic transducer can be arranged as in the sensor arrays 1430 - 1444. Figure 14C Any arrangement in Figure 14B can be used with any corner arrangement in

[0161] As depicted, most can be transmit transducers, but they can be in the minority. The receive transducers can be in contact with each other to form a ring, for example, as in the sensor arrays 1430 - 1438. As an alternative, the receive transducers can form non - connected groups, as in the sensor arrays 1440 - 1444. In such non - connected groups, there can be an even or odd number of receive transducers. These groups can all be the same or can vary.

[0162] Figure 14D is a schematic diagram of a sensor array according to a particular aspect of the present disclosure, which depicts two example configurations of the respective transducers of the sensor array. Figure 14D The depicted sensor array depicts sensor arrays including different numbers of ultrasonic transducers. For example, the sensor array 1450 includes 36 transducers, while the sensor array 1460 includes 64 transducers. The transmit ultrasonic transducer 1448 and the receive ultrasonic transducer 1447 can be arranged as in the sensor arrays 1450 - 1464.

[0163] Figure 14E is a schematic diagram of a sensor array according to a particular aspect of the present disclosure, which depicts two example configurations of the respective transducers of the sensor array. Figure 14ETwo example sensor arrays are shown, which are 12x12 ultrasonic transducers in size. For example, both sensor array 1465 and sensor array 1466 include 144 transducers. The transmit ultrasonic transducer 1492 and the receive ultrasonic transducer 1491 can be arranged as in sensor arrays 1465 - 1466. In some implementations, the central region of sensor arrays 1465 - 1466 may not include ultrasonic transducers. Sensor array 1465 may thus include, for example, 138 ultrasonic transducers. However, it should be understood that the central region can be larger or smaller than the size of 16 transducers in a square. In some implementations, the ultrasonic transducers can form a ring around a central region that does not include ultrasonic transducers. A sensor array that does not include ultrasonic sensors can include a central region for routing space.

[0164] In some cases, the ultrasonic transducers in a sensor array can be grouped. For example, sensor array 1465 can include 8 groups of ultrasonic transducers, where each group can include 16 ultrasonic transducers included in a square shape. The central region of sensor array 1465 does not include a group of ultrasonic transducers. Each group of ultrasonic transducers can be non - adjacent to each other. For example, there can be a gap between two or more groups of ultrasonic transducers. These groups can be non - adjacent horizontally, vertically, or diagonally.

[0165] Figure 14F is a schematic diagram of a sensor array according to a particular aspect of the present disclosure, which depicts an example configuration of individual transducers of sensor arrays of various sizes. Figure 14F Ten example sensor arrays with varying sizes are shown. For example, sensor array 1467 includes 16 ultrasonic transducers, sensor array 1468 includes 25 ultrasonic transducers, sensor array 1469 includes 36 ultrasonic transducers, sensor array 1470 includes 49 ultrasonic transducers, sensor array 1471 includes 64 ultrasonic transducers, sensor array 1472 includes 81 ultrasonic transducers, sensor array 1473 includes 100 ultrasonic transducers, sensor array 1474 includes 131 ultrasonic transducers, sensor array 1475 includes 144 ultrasonic transducers, and sensor array 1476 includes 169 ultrasonic transducers. The transmit ultrasonic transducer 1494 and the receive ultrasonic transducer 1493 can be arranged as in sensor arrays 1467 - 1476. However, it should be understood that the configuration of the transmit ultrasonic transducer 1494 and the receive ultrasonic transducer 1493 can be any suitable arrangement as described herein.

[0166] Figure 14GSchematic diagram of a sensor array according to a particular aspect of the present disclosure, depicting twelve example configurations of individual transducers of a sensor array of transducers sized 8x8. For example, sensor array 1477-1488 includes 81 transducers. The transmit ultrasonic transducer 1496 and the receive ultrasonic transducer 1495 may be arranged as in sensor array 1477-1488.

[0167] In some implementations, the sensor array may include any suitable combination of the sensor array characteristics (e.g., regions, groups, arrangements, etc.) described herein and with reference to Figure 13A - 13B and 14A-14G. For example, the sensing array may include a corner region as depicted in sensor array 1420 as Figure 14B and an inner region and a central region as depicted in sensor array 1472 as Figure 14F The arrangement of ultrasonic transducers in the sensor array may be based on the application of the sensor array, the operating frequency, size limitations, power constraints, etc.

[0168] Multiple embodiments provide many advantages. For example, different array sizes may be implemented depending on sensor area (physical size) limitations and power constraints. The total array size, the configuration of transmit and receive ultrasonic transducers (e.g., pMUTs), and the size of the ultrasonic transducers may be used to determine the transmit and receive acoustic apertures and beam shapes. The transmit and receive acoustic apertures and beam shapes may be changed using at least the above characteristics, for which different stack thicknesses and materials and the use of the sensor array may be selected to produce optimal performance given the constraints (e.g., size, power, sampling frequency, supply voltage, process breakdown voltage, etc.). B. Multi-frequency measurements

[0169] Figure 15 A set of graphs 1502, 1504, 1506 depicting energy measurements from transducers of a single sensor array operating at different frequencies according to a particular aspect of the present disclosure. Graphs 1502, 1504, 1506 show energy measurements over time for a pair of touch events. Graph 1502 depicts energy measurements of a transducer operating at 100 kHz, graph 1504 depicts energy measurements of a transducer operating at 1 MHz, and graph 1506 depicts energy measurements of a transducer operating at 10 MHz. Clearly, the measurements taken at these different frequencies have different energy traces, especially with respect to temperature drift.

[0170] Since the decrease in energy measurement associated with an ultrasonic transducer receiving a reflected ultrasonic signal is used as a factor in identifying a touch event, it may be desirable to find techniques to reduce any false touch events. As Figure 15As depicted, the energy measurements at different frequencies respond differently to temperature changes (e.g., temperature changes that occur when heat is transferred from a finger to a substrate or from a substrate to air, or other such temperature changes). Thus, an ultrasonic touch input system can use energy measurements at multiple frequencies or other types of operating procedures, such as different ultrasonic beam shapes, number of pulses, etc., to confirm or reject an inference of a touch event, rather than simply relying on identifying a drop in energy measurement to infer a touch event. For example, the perceived energy drop in Chart 1502 may not be registered as a touch event because a concurrent energy drop is not identified in Chart 1504 or 1506. However, once all three charts 1502, 1504, 1506 register a concurrent energy drop, it can be assumed that a touch event has occurred.

[0171] Figure 16 Chart 1600 is according to a particular aspect of the present disclosure and depicts the temperature behavior of an ultrasonic transducer with respect to an operating frequency. Chart 1600 includes four lines, each line associated with an air signal or a target signal at a first or second frequency. An air signal can refer to the energy measured when no touch event is present, while a target signal can refer to the energy measured when a touch event is occurring. The first and second frequencies can be any suitable different frequencies. Chart 1600 shows that for all signals, as the temperature increases, the overall signal strength decreases. Chart 1600 also shows different behaviors of each frequency with respect to temperature, which can thus be utilized to help identify whether a touch event has occurred (e.g., to identify whether a change in energy measurement is associated with a touch event or only with temperature drift).

[0172] In an example, the first and second measurements may be made by a transducer operating at a first frequency, which causes measurements at points 1610 and line 1612. At this time, it may not be clear whether the measurement at line 1612 is associated with a touch event (e.g., moving from point 1610 to point 1614) or a temperature change (e.g., moving from point 1610 to point 1616). The first and second measurements may also be made on a transducer operating at a second frequency, which causes measurements at point 1618 and either line 1620 or line 1622. If the second measurement at the second frequency falls on line 1620, it can be inferred that the energy drop is associated with a temperature change from point 1618 to point 1626 and is thus less likely to be associated with a touch event. However, if the second measurement at the second frequency falls on line 1622, it can be inferred that the energy drop is associated with a touch event because the energy drops from point 1618 to point 1624. The measurements at the first and second frequencies may be made simultaneously, sequentially, or otherwise close to each other in time (e.g., within a few milliseconds, tens of milliseconds, or hundreds of milliseconds of each other). Thus, by comparing the changes in energy measurements over a period of time at multiple frequencies, it can be determined whether a touch event has occurred.

[0173] Although graph 1600 has been described with reference to frequency-dependent energy changes due to temperature changes, this technique can be used to identify and utilize frequency-dependent energy changes due to changes in other environmental conditions such as humidity.

[0174] Figure 17 Graph 1700 is according to a particular aspect of the present disclosure and depicts the frequency response with respect to a stacked structure. Graph 1700 shows three lines, each associated with a different stack. Each different stack may be composed of different materials or different combinations of materials. Due to the inherent differences in each stack, each stack may have a unique response curve associated with the transmission frequency used by the ultrasonic input device. The response curve may be a measure of energy, the peak of the received signal, or any other quality factor. As Figure 17 depicted, the frequency that provides the highest response for stack / overlay 1 is higher than the frequency that provides the highest response for stack / overlay 2, which in turn is higher than the frequency that provides the highest response for stack / overlay 3.

[0175] Thus, specific frequencies and stack materials can be matched to provide optimal results. For example, given a set of known frequencies, the material for fabricating the housing of the integrated ultrasonic input device can be selected to maintain the highest possible energy measurement of the reflected ultrasonic signal from the initial transmission of the ultrasonic input device. As another example, given a known stack or known material (e.g., a specific display or a specific type of wood from a consumer product manufacturer), the ultrasonic input device can be set to operate at a frequency that provides the highest possible energy measurement. In some cases, the ultrasonic input device can automatically detect the optimal frequency to use based on measuring multiple frequencies that are close in time to each other. C. Separate Transmission and Reception

[0176] Figure 18 is a schematic diagram depicting circuit 1800 for transmitting and receiving signals through an ultrasonic transducer, and the circuit is in the transmission state. Circuit 1800 drives the ultrasonic transducer to transmit and receive signals, and thus requires a high-voltage switching circuit to separate the high-voltage transmitter from the low-voltage receiver. During transmission, the high-voltage switch allows the high-voltage transmitter circuit to drive the transducer while isolating the low-voltage receiver. To move to the reception state, the switch must isolate the high-voltage transmitter circuit and connect the transducer to the low-voltage receiving circuit.

[0177] Figure 19 is a depiction of Figure 18 circuit 1800 for transmitting and receiving signals through an ultrasonic transducer, and the circuit is in the reception state. When in the reception state, the high-voltage switch isolates the high-voltage transmitter circuit and connects the transducer to the low-voltage receiving circuit. However, the high-voltage switch typically has a large capacitance, which inherently attenuates the signal when the signal received at the transducer is conducted to the low-voltage receiver. Thus, for example, an input voltage of 0.37 millivolts (370 microvolts) can be attenuated to less than 2 microvolts. This parasitic effect can sharply reduce the available signal, thereby reducing the overall signal-to-noise ratio.

[0178] Figure 20 is a schematic diagram according to a specific aspect of the present disclosure, which depicts isolation circuits 2000, 2002 for transmitting and receiving signals through an ultrasonic transducer. Different from Figures 18 to 19 circuit 1800, Figure 20 circuits 2000, 2002 eliminate the need for a high-voltage switch. Thus, circuits 2000, 2002 can provide an effective drive for the transducer set to transmit while also providing an effective reception for the transducer set to receive. Circuit 2000 includes a high-voltage transmitter circuit that directly drives the transducer set to transmit the transducer. Circuit 2002 includes a low-voltage receiver circuit that directly receives signals from the transducer set to receive the transducer.

[0179] By separating the transmit and receive transducers, signal integrity can be improved, size can be reduced, and overall cost can be lowered. For example, by reducing or eliminating parasitic effects from electrical components (e.g., high-voltage switches) embedded between the transducer and its low-voltage receiver circuitry, signal integrity can be improved and power consumption can be improved. Since high-voltage devices (e.g., high-voltage switches) tend to be larger in size, the overall chip size can also be reduced. Thus, by eliminating these switches and optionally eliminating some of the high-voltage transmitter circuitry, the overall chip size and cost can be reduced. D. Beamforming

[0180] Figure 21 Are a set of schematic side views 2100, 2102, 2104, 2106 according to a particular aspect of the present disclosure, which depict beamforming achieved by using ultrasonic transducers.

[0181] Figure 2100 depicts the beam pattern of a single ultrasonic transducer such as a standard piezoelectric transducer. The beam is wide and is fixed by the sensor size and sensor topology. There is no ability to adjust the beam of the transducer for Figure 2100.

[0182] Figure 2102 depicts a focused beam achieved by activating a particular group of transducers. Using beamforming techniques, the activated transducers can focus the beam to a specific distance, which can improve the pressure sensitivity and accuracy of the ultrasonic sensor. For example, the focused beam can be used to provide fine point accuracy for touch events and for detecting other information associated with the touch event, such as the ridges and valleys of a user's fingerprint.

[0183] Figure 2104 depicts a wide beam achieved by activating a particular group of transducers. Using beamforming techniques, the activated transducers can focus the beam to a certain close distance to allow the beam to reach a point and spread out again before reaching the target distance. Such a wide beam can improve the overall coverage of the sensor and can be used to obtain more average measurements over a larger area. Such a wide beam can be used to reduce the target position sensitivity, which is advantageous in situations where a certain degree of variability is expected or desired, such as providing a large touch-sensitive area and / or additional touch-sensitive area on or around a button.

[0184] As depicted in Figures 2104 and 2106, the beam can be adjusted as needed and a trade-off can be made between a more focused transmit pressure on the target and a larger effective area with lower target sensitivity.

[0185] Figure 2106 depicts a multi-receiver configuration of activated transducers. In this configuration, a set of transmit transducers can emit an ultrasonic signal that can be reflected and received at two or more sets of receive transducers. For example, a first set of receive transducers (e.g., one or more transducers) can be positioned to receive the ultrasonic signal that has been reflected within a first zone, and a second set of receive transducers can be positioned to receive the ultrasonic signal that has been reflected within a second zone. As depicted in Figure 2106, the first zone can be smaller and enclosed within the second zone.

[0186] By performing beamforming using an array of ultrasonic transducers, energy can be confined to a specific region of interest, and thus the ultrasonic transducers can be less sensitive to regions outside the region of interest. IV. Microelectromechanical Ultrasonic Transducers for Touch Input

[0187] In some cases, an ultrasonic input device can include an ultrasonic sensor that includes a microelectromechanical ultrasonic transducer (MUT), such as a piezoelectric microelectromechanical ultrasonic transducer (pMUT) or a capacitive microelectromechanical ultrasonic transducer (cMUT). Other types of transducers in addition to pMUTs and cMUTs can include bulk piezoelectric transducers that are integrated (i.e., directly fabricated on CMOS) and non-integrated (i.e., fabricated separately and subsequently assembled with a CMOS chip on a board or directly communicate with any hardware having internal integrated circuit (I2C) or serial peripheral interface (SPI) communication capabilities). As described herein, microelectromechanical ultrasonic transducers for touch input can allow for an improved energy sensing region. Additionally, MUTs can also reduce the overall power consumption of the ultrasonic input device. A. Improved Energy Sensing Region

[0188] Figure 22 is a set of graphs 2202, 2204 that depict the operating modes of microelectromechanical ultrasonic transducers according to specific aspects of the present disclosure compared to a standard bulk transducer, which is depicted as the average displacement for different frequencies. Graphs 2202, 2204 contain lines depicting the average displacement over a frequency range from 0.5 MHz to 5 MHz, and axisymmetric cross-sectional visual illustrations of the transducer modes.

[0189] Diagram 2202 depicts the operating modes of a standard bulk transducer (e.g., a standard piezoelectric transducer) operating from 0.5 MHz to 5 MHz. During this relatively small frequency range, the number of peaks in the average displacement and the overall range of each of these peaks significantly depend on the various combinations of bulk modes, shear modes, flexural modes, surface acoustic modes, and other modes that the bulk transducer experiences. As a result, in addition to the normal longitudinal waves of interest, shear waves and surface acoustic waves can be generated in different directions. Therefore, sensors using such bulk transducers may have uncontrollable beam patterns, harmful crosstalk, more multipath reflections from different angles from different modes, parasitic modes and notches in the spectrum, less clean received signals, more energy wasted on unwanted modes, and other such problems.

[0190] In contrast, Diagram 2204 depicts a uniform and predictable flexural mode shape present in a MUT (e.g., a pMUT) over the same frequency span, and this flexural mode shape is used to emit longitudinal acoustic waves along the normal direction towards the outer surface of the stack. As a result, the MUT is capable of implementing much improved performance compared to a standard bulk transducer.

[0191] Due to the nature of the ultrasonic input device, it is desirable to detect ultrasonic reflections based on longitudinal acoustic waves (e.g., propagating along a direction orthogonal to the sensor). In the case where a MUT is used as an ultrasonic transducer for touch input, due to their inherent ability to perform flexural mode displacements to generate such longitudinal acoustic waves without inadvertently generating many (if any) transverse waves or other unwanted waves, MUTs perform particularly well. Therefore, MUTs can be used for beamforming operations, such as those described herein, can be tightly packaged into sensor arrays, can be used with less filtering equipment, and can obtain a higher signal-to-noise ratio using the same or less power as in the case of using standard bulk piezoelectric transducers.

[0192] Figure 23 Are a set of schematic side views 2302, 2304, 2306, which depict the operating modes of a standard bulk transducer for ultrasonic touch detection. When a standard bulk transducer is used for ultrasonic touch detection, driving the transducer to transmit a signal may cause the transducer to displace in multiple operating modes, which can cause false signals to be transmitted into the receiving medium (e.g., the stack).

[0193] Figure 2302 depicts a longitudinal operation mode, in which the driving of the transducer initiates a longitudinal signal along a direction orthogonal to the sensor. However, the same or similar driving of the transducer in Figure 2302 can cause a lateral displacement as depicted in Figure 2304. This lateral displacement (e.g., due to a lateral operation mode) can initiate a lateral signal that is carried into the receiving medium along a direction different from the normal of the sensor, or can result in an undesired shear wave traveling normally. As a result, the driving body transducer can generate a signal as depicted in Figure 2306, in which both normal and abnormal signals propagate from the body transducer. Since the sensing region (e.g., the region where sensing is desired) is typically located directly above the stack, the abnormal signal can cause interference with the signal received from the sensing region. In addition, the body transducer may be vulnerable to the physical topology of the region of the stack near the sensing region, because different topologies may initiate different reflections of the abnormal signal, which may lead to false positives or false negatives.

[0194] Figure 24 are a set of schematic side views 2402, 2404 that depict the lateral signal rejection of a micromachined ultrasonic transducer according to a particular aspect of the present disclosure. Figure 2402 is a close-up view of a single transducer of a MUT array. The transducer can be composed of multiple layers, including a piezoelectric layer that, when excited, can initiate a bending displacement to emit a longitudinal wave along a direction orthogonal to the sensor (e.g., a direction orthogonal to the surface of the MUT).

[0195] Figure 2404 depicts an ultrasonic input device using a sensor with a MUT. The ultrasonic input device is depicted as being coupled to an aluminum layer and a glass layer, but any other stack configuration can be used. The nature of the MUT can allow ultrasonic signals to be emitted along a direction orthogonal to the sensor, while minimizing or eliminating any signals that would otherwise propagate along a direction not orthogonal or substantially not orthogonal to the sensor in the case of using a body transducer. Thus, using a MUT as a transducer in an ultrasonic input device can help focus energy into the desired sensing region and reduce the sensitivity to false positives or false negatives due to incorrect reflections. B. Driving Convenience

[0196] When used with an ultrasonic input device, in addition to the above benefits of the MUT, the MUT can also reduce the overall power consumption of the ultrasonic input device. Since the power required to drive a transducer is proportional to its capacitance multiplied by the square of its voltage, the low capacitance level (e.g., on the order of picofarads) of a MUT array results in much lower power consumption than the relatively high capacitance level (e.g., on the order of nanofarads, three orders of magnitude larger than picofarads) of an equivalent standard body transducer. V. Ultrasonic Signal Processing

[0197] It can process the reflected ultrasonic signals to generate an image and determine the range to the object. The embodiments described herein can process the reflected ultrasonic signals to determine whether the object is in contact with the surface. A. Detecting Touch Input by Digitizing Reflected Signals

[0198] Figure 25 FIG. 2500 is a schematic flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device according to a particular aspect of the present disclosure. Flowchart 2500 includes transmitting and receiving ultrasonic signals as shown in a first graph 2502. The first graph 2502 shows an analog measurement of a first signal 2503 for the transmitted ultrasonic signal associated with the ultrasonic input device and a set of subsequent signals 2504A, 2504B, 2504C, 2504D, 2504E of a set of reflected ultrasonic signals associated with the ultrasonic input device. The first signal 2503 and the subsequent signals 2504 can be measured using a high-speed ADC 2506 to digitize the signals.

[0199] The output of the high-speed ADC 2506 is shown in a second graph 2508. The second graph 2508 includes a first digital representation 2510 of the transmitted ultrasonic signal and subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E of the reflected ultrasonic signals associated with the ultrasonic input device. The first digital representation 2510 and the subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E can be processed by a digital processing module 2514 embedded in the ultrasonic input device and / or a system coupled to the ultrasonic input device. The digital processing module 2514 can demodulate the digital representations of the data to extract touch input information. For example, the digital processing module can process one or more of the subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E to determine that the amplitude of the second digital representation is below a threshold associated with an object in contact with the surface of the ultrasonic input device. B. Detecting Touch Input Using Energy Integration

[0200] Figure 26FIG. 2600 is a schematic flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to a particular aspect of the present disclosure. The flow chart 2600 includes transmitting and receiving ultrasonic signals as shown in the first graph 2602. The first graph 2602 shows an analog measurement of an ultrasonic signal 2603 for transmitting a set of reflected ultrasonic signals associated with the ultrasonic input device and a set of subsequent signals 2604A, 2604B, 2604C, 2604D, 2604E. The flow chart 2600 may include an ultrasonic input device having an analog circuit that includes a rectifier 2606 to rectify the subsequent signals 2604A, 2604B, 2604C, 2604D, 2604E.

[0201] A second graph 2608 shows the first signal 2603 and a set of rectified signals 2610A, 2610B, 2610C, 2610D, 2610E, each rectified signal corresponding to a respective reflected ultrasonic signal in the set of reflected ultrasonic signals. The rectified signals 2610A, 2610B, 2610C, 2610D, 2610E may be processed by an analog integrator 2612 to output a direct current (DC) signal 2613 as shown in the third graph 2614, which is proportional to the amplitude of the reflected ultrasonic signal. The DC signal 2613 may be determined using an energy measurement window 2616. The DC signal 2613 may represent an energy value associated with the energy of the received signal measured during the energy measurement window 2616. The DC signal 2613 may be processed by a low-speed ADC 2618. The DC signal 2613 is output by the rectifier 2606 and the integrator 2612 eliminates the need to generate a high-frequency digital output, and as a result, the low-speed ADC can use less power and can be fabricated on a smaller chip area.

[0202] Figure 27 FIG. 2700 is a schematic example flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to a particular aspect of the present disclosure. The flow chart 2700 includes transmitting and receiving ultrasonic signals as shown in the first graph 2702. The first graph 2702 shows an analog measurement of a first signal 2703 for transmitting an ultrasonic signal and an analog measurement of a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E for a set of reflected ultrasonic signals associated with the ultrasonic input device. The flow chart 2700 may include an ultrasonic input device having an analog summing or integrating circuit 2720 and a summed voltage output 2722.

[0203] The second graph 2708 shows a first signal 2703 and a set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E, which respectively correspond to the energy of the corresponding reflected ultrasonic signals in the set of reflected ultrasonic signals. For illustrative purposes, the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E are depicted as solid lines, which overlap with the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E from the first graph 2702 shown as dashed lines.

[0204] The summing or integrating circuit 2720 can receive the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E within the energy measurement window 2716. The summing or integrating circuit 2720 can generate a voltage output 2722, which is an analog value representing the summed / integrated energy within the energy measurement window 2716.

[0205] In some cases, an optional negative DC charging circuit 2724 can be applied to the summing or integrating circuit 2720 to cancel information not associated with a touch event. Since a touch event is identified based on the difference between the signals received during a non-contact state and the signals received during a contact state, there is some information (e.g., a baseline signal) within the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E that is not associated with these differences. Removing such a baseline signal can result in a more efficient sampling range during analog-to-digital conversion. Since removing such an analog baseline signal in the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E would require precise phase alignment, it is difficult to apply such a correction. However, as Figure 27 depicted, the optional negative DC charging circuit 2724 applied to the summing or integrating circuit 2720 can cancel a specific amount of energy associated with the baseline signal or a portion of it, thereby increasing the amount of the effective range available for analog-to-digital conversion. In this case, the voltage output 2722 can be proportional to the energy of the signal minus the energy of the negative DC charging circuit 2724.

[0206] The voltage output 2722 can be processed by the low-speed ADC 2718. The voltage output 2722 of the summed / integrated energy within the energy measurement window 2716 can eliminate the need to generate a high-frequency digital output, and as a result, the low-speed ADC can use less power and can be fabricated on a smaller chip area.

[0207] Figure 28FIG. 2800 is a schematic flowchart for processing ultrasonic signals transmitted and received by an ultrasonic input device that uses energy integration by absolute value summation, according to a particular aspect of the present disclosure. Flowchart 2800 may be a technique for implementing Figure 27 Flowchart 2700. Flowchart 2800 includes transmitting and receiving ultrasonic signals, as shown in a first graph 2802. The first graph 2802 shows an analog measurement of a first signal of the transmitted ultrasonic signal associated with the ultrasonic input device and subsequent signals of a set of reflected ultrasonic signals. The first graph 2802 may depict voltage (e.g., V(t)) as a function of time. The first graph 2802 may be Figure 27 The first graph 2702 of. Flowchart 2800 may include an ultrasonic input device having an analog sampling circuit 2806, an absolute value circuit 2814, an analog accumulator 2824, and a summed voltage output 2828.

[0208] The set of subsequent signals from the first graph 2802 may pass through the analog sampling circuit 2806 to produce a sampled first signal 2810 and a set of sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E as depicted in a second graph 2808. The first signal may correspond to the initially transmitted ultrasonic wave. The second graph 2808 may depict voltage as a function of samples (e.g., V(n), where n is the number of samples). The sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E may be passed to the absolute value circuit 2814, which may generate a set of energy signals 2820A, 2820B, 2820C, 2820D, 2820E as depicted in a third graph 2816. The third graph 2816 may depict the absolute value of voltage as a function of samples (e.g., |V(n)|). The absolute value circuit 2814 may pass all zero or positive values of the set of sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E and invert the polarity of all negative values. The sampled first signal 2818 is also shown in the third graph 2816, and the sampled first signal 2818 may be similar to the sampled first signal 2810.

[0209] A switched capacitor analog accumulator 2824 may be used to sum the set of energy signals 2820A, 2820B, 2820C, 2820D, 2820E within an energy measurement window 2822. The switched capacitor analog accumulator may generate a voltage output 2828, which is an analog value representing the sum of the energy within the energy measurement window 2822. In some cases, an analog integrator may be used instead of an accumulator.

[0210] In some cases, an optional negative-timed DC charging circuit 2826 can be applied to the switched capacitor analog accumulator 2824 to cancel information not associated with a touch event. Since the sampling circuit 2806 is timed according to the sampling rate, the optional negative-timed DC charging circuit 2826 can be timed at the same rate to ensure that a bias voltage is applied at appropriate intervals corresponding to the samples of the subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E of the samples. When the optional negative-timed DC charging circuit 2826 is used, the voltage output 2828 can be proportional to the energy of the signal minus the energy of the negative-timed DC charging circuit 2826.

[0211] The voltage output 2828 can be processed by the low-speed ADC 2830. The voltage output 2828 of the summed energy within the energy measurement window 2822 can eliminate the need to generate a high-frequency digital output, and as a result, the low-speed ADC can use lower power and can be fabricated on a smaller chip area.

[0212] Figure 29 FIG. 2900 is a schematic flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device that uses energy integration by self-mixing and integration according to a particular aspect of the present disclosure. The flow chart 2900 can be a technique for implementing Figure 27 the flow chart 2700. The flow chart 2900 includes transmitting and receiving ultrasonic signals, as shown in the first graph 2902. The first graph 2902 shows an analog measurement of a first signal associated with the ultrasonic input device for transmitting an ultrasonic signal and a set of subsequent signals for a set of reflected ultrasonic signals. The first graph 2802 can depict the voltage as a function of time (e.g., V(t)). The first graph 2902 can be Figure 27 the first graph 2702. The flow chart 2900 can include an ultrasonic input device having a self-mixing circuit 2906, an analog integrator circuit 2920, and an integrated voltage output 2922.

[0213] The set of subsequent signals from the first graph 2902 can be passed through the self-mixing circuit 2906 to generate a set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E, as depicted in the second graph 2908. The self-mixing circuit 2906 can effectively self-multiply each analog value over time. As a result, the second graph 2908 can depict the squared voltage as a function of time (e.g., V 2 (t)). Due to the nature of squaring and thus the nature of the self-mixing circuit 2906, the set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E will always be positive.

[0214] The set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E can be passed to an analog integrator circuit 2920. The analog integrator circuit 2920 can integrate the set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E within the energy measurement window 2916 to generate an integrated voltage output 2922. The integrated voltage output 2922 can be an analog representation of the total energy over time within the energy measurement window 2916. In some cases, an accumulator can be used instead of the analog integrator circuit 2920.

[0215] In some cases, an optional negative bias current circuit 2924 can be applied to the analog integrator circuit 2920 to cancel information not associated with a touch event. The negative bias current circuit 2924 can constantly drain charge from the analog integrator circuit 2920 during integration. When the optional negative bias current circuit 2924 is used, the voltage output 2922 can be proportional to the energy of the signal minus the energy of the negative bias current circuit 2924.

[0216] The voltage output 2922 can be processed by a low-speed ADC 2926. The voltage output 2922 of the integrated energy within the energy measurement window 2916 can eliminate the need to generate a high-frequency digital output, and as a result, the low-speed ADC can use lower power and can be fabricated on a smaller chip area.

[0217] Figure 30 is a schematic circuit diagram depicting an analog integrator 3000 with a negative bias current according to a particular aspect of the present disclosure. The analog integrator 3000 negative bias can be Figure 29 of the analog integrator circuit 2920 and the optional negative bias current circuit 2924.

[0218] The analog integrator 3000 can receive an input voltage (V in ) through a resistor (R in ) to obtain an input current (I in ). A capacitor (C) can be charged by a charging current (I f ) to generate an integrated signal, which can feed a voltage output (V out ). Item (A) is an operational amplifier. A negative bias current (I bias ) can be applied at point X to drain charge from the analog integrator 3000, thereby causing a reduced charging current (I f ). Thus, the charging current can be calculated as I f = I in - I bias . C. Energy measurement windowing

[0219] Figure 31 is a schematic flowchart for processing ultrasonic signals according to a particular aspect of the present disclosure, depicting the reduced impact of time-of-flight variations on touch input detection within an energy measurement window. In an ultrasonic imaging system or a proximity detection system, precise time-of-flight is crucial for determining the distance of an object in the field of view from an ultrasonic transducer. In contrast to imaging and proximity systems, distances to the first and second surfaces of a material layer in an ultrasonic input device can be provided, and touch input can be detected without considering time-of-flight variations. Figure 31 Shows a first graph 3102 in which a first set of reflected ultrasonic signals 3104 starts being received at a first time 3106, and shows a second graph 3108 in which a second set of reflected ultrasonic signals 3110 is received at a second time 3112. A first signal 3103 may be associated with the transmitted ultrasonic signal, which occurs prior to the first time 3106 and the second time 3112 in the first graph 3102 and the second graph 3108, respectively. The first set of reflected ultrasonic signals 3104 passes through an energy accumulator or integrator circuit 3120 to generate an output voltage 3122 (e.g., V sum1 ), which can be fed into a low-speed ADC 3124 and processed to obtain an output value 3118 (e.g., 3000 LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3110 passes through an energy accumulator or integrator circuit 3120 to generate an output voltage 3123 (e.g., V sum2 ), which can be fed into the low-speed ADC 3124 and processed to obtain an output value 3119 (e.g., 3000 LSB, where LSB represents the least significant bit). The output values 3118, 3119 may represent the pulse echo energy during the energy measurement window 3116 of the graphs 3102, 3108. Although the start times of the first set of reflected ultrasonic signals 3104 and the second set of reflected ultrasonic signals 3110 are different (e.g., the first time 3106 and the second time 3112), the output values 3118, 3119 may be the same or substantially the same because the entire first set of reflected ultrasonic signals 3104 and the entire second set of reflected ultrasonic signals 3110 each fit within the energy measurement window 3116.

[0220] Accordingly, the ultrasonic input device can be insensitive to the time of flight at least to some extent (e.g., within an energy measurement window). In some cases, advanced windowing techniques such as those disclosed herein can further improve the insensitivity of the ultrasonic input device to the time of flight. As a result, the surface of the ultrasonic input device (e.g., the material layer) does not need to be completely flat and / or the alignment of the ultrasonic input device relative to the material (e.g., the material layer) does not need to be precisely 90° (e.g., the angle between the propagation direction of the ultrasonic transducer and the surface of the material layer). Additionally, the insensitivity to the time of flight can allow for a certain degree of insensitivity to the varying refractive index through which the ultrasonic signal passes (e.g., a material layer having an overall slightly inconsistent refractive index).

[0221] For example, as Figures 26 to 29 and Figure 31 shown, the energy of the reflected ultrasonic signals (e.g., reflected echoes and standing waves) is summed or integrated over the energy measurement window. This energy is related to the conditions of the touch input and can thus be used for input touch detection. The size of the energy measurement window 3116 can be determined to include the pulse time of the ultrasonic signal and taking into account variations in the time of flight due to temperature, stack variations (e.g., variations in the materials making up the ultrasonic input device), etc. The energy measurement window 3116 can reduce errors caused by variations in the time of flight. The ultrasonic touch device can determine an input touch contact based on a specific threshold.

[0222] Figure 32 is a schematic simplified flowchart for processing ultrasonic signals according to a particular aspect of the present disclosure, which depicts the enhanced effect of variations in the time of flight of reflected ultrasonic signals on touch input detection outside the energy measurement window. Figure 32 Shows a first graph 3202 where a first set of reflected ultrasonic signals 3204 starts to be received at a first time 3206, and shows a second graph 3208 where a second set of reflected ultrasonic signals 3210 is received at a second time 3212. The first signal 3203 can be associated with the transmitted ultrasonic signal, which occurs before the first time 3206 and the second time 3212 in the first graph 3202 and the second graph 3208 respectively. The first set of reflected ultrasonic signals 3204 can be processed as disclosed herein to obtain an output value 3218 (e.g., 3000 LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3210 can be processed as disclosed herein to obtain an output value 3219 (e.g., 2500 LSB, where LSB represents the least significant bit). The output values 3218, 3219 can represent the pulse reflection energy during the energy measurement window 3216 of the graphs 3202, 3208.

[0223] As Figure 32As shown, since almost all of the first set of reflected ultrasonic signals 3204 fit within the energy measurement window 3216, but a smaller portion of the second set of reflected ultrasonic signals 3210 fit within the energy measurement window 3216, the output value 3218 is greater than the output value 3219. As depicted in Figure 32 , the output values 3218, 3219 differ by 500 LSB. If the reflected ultrasonic signal falls outside the energy measurement window 3216, some of the measured pulses may be cut off and not measured, and thus the ultrasonic input device may be vulnerable to variations in the time of flight (e.g., variations that would cause a difference between the first time 3206 and the second time 3212).

[0224] Figure 33 is a schematic flowchart for processing ultrasonic signals according to a particular aspect of the present disclosure, which depicts the minimal impact of variations in the time of flight of reflected ultrasonic signals on touch input detection outside the energy measurement window in the case of using window shaping. Figure 33 shows a first graph 3302 in which a first set of reflected ultrasonic signals 3304 starts being received at a first time 3306, and shows a second graph 3308 in which a second set of reflected ultrasonic signals 3310 is received at a second time 3312. The first signal 3303 may be associated with the transmitted ultrasonic signal, which occurs prior to the first time 3306 and the second time 3312 in the first graph 3302 and the second graph 3308, respectively. The first set of reflected ultrasonic signals 3304 may be processed as disclosed herein to obtain an output value 3318 (e.g., 2500 LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3310 may be processed as disclosed herein to obtain an output value 3319 (e.g., 2450 LSB, where LSB represents the least significant bit). The output values 3318, 3319 may represent the pulse reflection energy during the energy measurement window 3316 of the graphs 3302, 3308.

[0225] Unlike Figure 32 , the energy measurement window envelope 3320 is used in combination with the energy measurement window 3316. The energy measurement window envelope 3320 scales multiple portions of the signals within the energy measurement window 3316 such that portions near the edges of the energy measurement window 3316 have less weight than portions near the center of the energy measurement window 3316. Thus, although there are small variations near the ends of the energy measurement window 3316, the resulting output value will be primarily based on the signals measured within the center of the energy measurement window 3316. The energy measurement window envelope 3320 is depicted in Figure 33 as having a particular horn bell shape, although any suitable shape may be used, including symmetric and asymmetric shapes. As in Figure 33The vertical extent of the energy measurement window envelope 3320 depicted may represent any suitable scaling, such as 0% to 100%. In some cases, the energy measurement window envelope 3320 may include amplifying signals near the center of the energy measurement window 3316 to values greater than 100% of the original signal at that time.

[0226] As depicted in Figure 33 , due to the use of the energy measurement window envelope 3320, signals (e.g., the first set of reflected ultrasound signals 3304 and the second set of reflected ultrasound signals 3310) are weighted such that multiple portions of the signal closest to the center of the energy measurement window 3316 are given greater weight than portions closest to the edges of the energy measurement window 3316, thereby de-emphasizing any portions clipped by the start or end of the energy measurement window 3316. As a result, the output values 3318 and 3319 are closer than Figure 32 the output values 3218 and 3219. As depicted in Figure 33 , the output values 3318, 3319 differ by only 50 LSB. Thus, as a result of the energy measurement window envelope 3320, the ultrasound input device can become less susceptible to variations in time of flight.

[0227] Figure 34 is a schematic circuit diagram depicting a window shaping circuit 3400 in accordance with a particular aspect of the present disclosure. The window shaping circuit 3400 may generate an energy measurement window having an energy measurement window envelope (e.g., the energy measurement window 3316 having the Figure 33 energy measurement window envelope 3320). With the addition of the tunable capacitor 3402, the window shaping circuit 3400 may operate as a conventional analog accumulator circuit. The tunable capacitor 3402 may take any suitable form, such as a switched ladder of capacitors of different sizes. Selection of the capacitor sizes for the tunable capacitor 3402 can cause the gain on the analog accumulator circuit to be adjusted over time. In some cases, the tunable capacitor 3402 may be driven by a clock 3404 or other source to determine when to change the capacitance. In some cases, the tunable capacitor 3402 may be used in conjunction with an analog sampling circuit (such as Figure 28 the analog sampling circuit 2806) and the tunable capacitor 3402 may change with different numbers of samples (e.g., n of V(n)).

[0228] Figure 35 is a schematic diagram depicting a process 3500 for processing ultrasound signals to detect touch inputs using the amplitude of reflected ultrasound signals in accordance with a particular aspect of the present disclosure. Figure 35An ultrasonic input device 3502 without a touch input 3504 and with a touch input 3506 is shown. A first graph 3508 associated with the ultrasonic input device 3502 without a touch input 3504 shows a transmission signal 3510 and a first set of reflected signals 3512. The first set of reflected signals 3512 can be processed to generate an output voltage 3530 (e.g., V sum1 ) associated with the first set of reflected signals 3512, which can be provided to a low-speed ADC 3534 and further processed to generate a first output 3536. The first output 3536 can represent the energy of the first set of reflected signals 3512 within an energy measurement window envelope 3516. A second graph 3520 shows a transmission signal 3522 and a second set of reflected signals 3524. As disclosed herein, the second set of reflected signals 3524 can be processed to generate an output voltage 3532 (e.g., V sum2 ), which can be provided to a low-speed ADC 3534 and further processed to generate a second output 3538. The second output 3538 can represent the energy of the second set of reflected signals 3524 within the energy measurement window envelope 3516.

[0229] The energy measurement window envelope 3516 (e.g., an envelope similar to the energy measurement window envelope 3320 of Figure 33 ) can be applied to the first set of reflected signals 3512 and the second set of reflected signals 3524. In some embodiments, the energy measurement window envelope 3516 can be applied to the first set of reflected signals 3512 and the second set of reflected signals 3524 to attenuate signals at the edges of the energy measurement window envelope 3516.

[0230] The first output 3536 and the second output 3538 can be compared to determine whether a touch input (e.g., a touch event) has occurred. For example, if the second output 3538 is lower than the first output 3536 by a predetermined amount and / or if the second output 3538 is below a threshold, the ultrasonic input device 3502 can generate a signal indicating the presence of a touch input on the surface. Since the output voltages 3530, 3532 indicate the first output 3536 and the second output 3520 respectively, the output voltages 3530, 3532 can be used to determine whether a touch input has occurred. In some embodiments, only a single output, such as the first output 3518, can be compared with a reference value. The reference value can be established at manufacturing time and / or updated based on background characteristics (such as temperature) measured by or transmitted to the device.

[0231] Referring to Figure 35The described techniques can be used to generate an output signal from the ultrasonic input device 3502, although other techniques can also be used. Any technique that can convert a signal associated with the first set of reflected signals 3512 or the second set of reflected signals 3524 into a measurement of the total energy can be used.

[0232] Figure 36 FIG. 3600 is a graph depicting a simplified example energy signal 3614 in accordance with a particular aspect of the present disclosure. When the ultrasonic input system processes an input ultrasonic signal received by an ultrasonic transducer, the ultrasonic input system can convert the ultrasonic signal into an energy signal 3614. The energy signal 3614 can represent the overall energy associated with the input ultrasonic signal. For example, as depicted in Figure 35 , the signals depicted in FIGS. 3508 and 3520 can be converted into outputs 3536 and 3538. These outputs can be recorded, graphed, or output over time as the energy signal 3614. The output 3536 associated with a non-touch event can generally be regarded as an area where the overall energy of the energy signal is higher, while the output 3538 associated with a touch event can generally be regarded as an area where the overall energy of the energy signal is lower. It should be understood that the continuous output of the processed ultrasonic signal can be used to generate an energy signal that can subsequently be used to determine whether a touch event has occurred at a particular point in time. Figure 36 The energy signal 3614 of FIG. is simplified for illustrative purposes only. D. Touch Input Error Prevention

[0233] Figure 37 FIG. 3700 is a diagram in accordance with a particular aspect of the present disclosure that depicts ultrasonic signal measurements of reflections made using an ultrasonic input device and shows techniques for improving touch input detection. The sensor readout (e.g., a DC signal or other sensor data) determined by the ultrasonic input device can be measured continuously or at a particular frequency according to the present application. In some embodiments, the sensor readout can be measured at a frequency of 100 Hz. A single measurement 3702 can correspond to an energy measurement within an energy measurement window. One or more single measurements can be used to determine a current state 3706. The current state can be defined by a current individual measurement 3702 or by a best fit line based on two or more single measurements. In some embodiments, the least squares method can be used to calculate the best fit line. Multiple single measurements can be used to determine a moving average threshold 3704.

[0234] The current state 3706 and the moving average threshold 3704 can be used to detect touch events. The moving average threshold 3704 can be used to determine a sudden signal drop that can trigger a touch input event. For example, the system can detect a "hand touch" effect only when a "rapid signal change" 3708 from the current state 3706 is detected. The rapid signal change 3708 can be associated with a sudden signal drop on all or many channels and can be considered a touch input event. The threshold for detecting the rapid signal change 3708 can be the moving average threshold 3704 (dynamic threshold) when no hand touch event is detected. In some embodiments, the rapid signal change 3708 can be a pre-programmed static threshold. The rapid signal change 3708 event can trigger a touch input event and cause the ultrasonic input device to generate a signal indicating a touch input on the surface of the device. For the rapid signal change 3708 event, multiple measurements 3710 are made to ensure that the signal actually does drop and does not jump back up, such as back to its original value. For example, a hard press by the user may result in a dropping sensor reading but will still provide a continuous signal. During the multiple measurements 3710, if the signal quickly returns to a higher value (such as the value previously seen before the suspected touch event), the ultrasonic input device may identify the temporary signal drop as a false touch event and not classify it as a touch event. The multiple measurements 3710 can occur within a very short time frame (e.g., on the order of tens or hundreds of milliseconds). In some embodiments, a "gradual signal change" can be considered a temperature change rather than a hand touch event because the moving average will be adjusted with each individual measurement 3702 at a rate based on the number of measurements used to determine the moving average.

[0235] In some cases, the threshold 3704 can be based on calculations other than moving average calculations. In some cases, the threshold 3704 is simply some function of past history (e.g., historical measurements), such as a function of the past x measurements. In some cases, the past measurements can be weighted, such that more recent measurements have a greater weight than measurements made longer ago. In such cases, the response time of the ultrasonic input device can be adjusted based on the weights of the past x measurements. For example, the threshold can be calculated as a function of historical values according to Threshold = f(X[n - 1], X[n - 2], …, X[n - m]) (where X[n] is the nth sensor reading (or the current sensor reading)). In another example, the threshold can be according to Threshold = w 1 X[n - 1] + w 2 X[n - 2], …, w m X[n - m] (where w n(where is the weight parameter read by the nth sensor) is calculated as a function of the weighted history value,. In some cases, the weight parameter can be trained using machine learning, such as described in more detail herein.

[0236] In some cases, in addition to or instead of determining the fast signal change 3708 based on the measurement itself, the determination can be made using the slope of a set of measurements (such as the slope of the current measurement and a certain number of past measurements).

[0237] Figure 38 is a diagram 3800 according to a particular aspect of the present disclosure, which depicts the measurement of reflected ultrasonic signals using an ultrasonic input device and shows additional techniques for improving touch input detection. A portion of the diagram 3800 is depicted as Figure 37 the diagram 3700 of. The diagram 3800 shows that the signal may change over time due to various factors (such as temperature changes), however, the ultrasonic input device may be able to distinguish that these changes are not touch events. However, a sudden signal drop between consecutive measurements can indicate a touch event. The current state 3806 may be similar to Figure 37 the current state 3706 of. The moving average threshold 3804 is similar to Figure 37 the threshold 3704 of. This threshold 3804 is partially based on the moving average of previous measurements of the current state 3806, such as the moving average of previous measurements offset by a given amount. This type of threshold 3804 can be referred to as a dynamic threshold, although other threshold techniques can be used.

[0238] At region 3816, a touch event occurs. When the touch event occurs, the current state 3806 drops rapidly. As depicted in the marked portion of the diagram 3800, various measurements 3802 are shown. Each measurement 3802 can be separated in time based on the measurement frequency. For example, each measurement 3802 can be separated by 0.01 seconds (e.g., at 100 Hz), although other frequencies can be used. A sudden drop can be detected between two or more consecutive measurements 3802. When the sudden drop in the current state 3806 drops below the threshold 3804, a touch event can be considered to have occurred. Region 3817 depicts another touch event.

[0239] At regions 3818 and 3820, a gradual change in the temperature of the ultrasonic sensor and the temperature of the surface to which the sensor is coupled can cause a gradual change in the current state 3806. Due to the relatively slow change in the current state 3806, the threshold 3804 based on the moving average of the current state 3806 will also change. Since the threshold 3804 can compensate for the slow changes in the current state 3806, such as those occurring due to temperature changes, these slow changes in the current state 3806 will not exceed the threshold 3804 and thus will not trigger a touch event. Additionally, since the threshold 3804 is dynamically updated, the threshold 3804 can operate properly at different temperatures. In some cases, the change in the current state 3806 due to temperature changes can even be greater than the contrast caused by an actual hand touch, but since these temperature changes are much slower than the change in the current state 3806 due to a touch event, they are not detected as a touch event. VI. Multi - frequency Touch Detection

[0240] Figure 39 is a diagram according to a particular aspect of the present disclosure, which depicts the temperature dependence of the reflected ultrasonic signal. The reflected ultrasonic signal received by the ultrasonic input device may include a main signal 3902 and any unwanted signal 3904. The main signal travels through the material layer along a first path and is associated with a first time - of - flight (TOF), and any unwanted signal 3904 travels through the material layer along a second path and is associated with a second TOF. The speed of sound in the material layer depends on the temperature of the material layer. As a result of the change in the speed of sound due to the temperature change, the main signal 3902 and the unwanted signal 3904 travel through different acoustic paths, and the associated first TOF and second TOF change by different amounts accordingly. This creates a net TOF difference Δt(T)3906 between the main signal 3902 and the unwanted signal 3904, which changes with temperature T. It is then converted into a phase - delay difference Δφ(T) between the main signal 3902 and the unwanted signal 3904. And thus a different integrated signal intensity difference Dout(T) is produced, as depicted by line 3910.

[0241] Figure 40 is a set of diagrams according to a particular aspect of the present disclosure, which depicts the TOF temperature dependence of two - frequency methods for detecting touch input. These diagrams can be similar to Figure 39In the graph of, in a multi - frequency ultrasonic input device, different frequencies will have different temperature effects, thus causing different TOFs for each signal. When a signal drop is detected in a threshold number of frequency channels, the multi - frequency ultrasonic input device can process a "finger touch" (e.g., a touch event). For example, two different methods can detect whether a finger touches the ultrasonic touch input device, and when both methods agree that a finger touch has been detected, the device can process the touch event only.

[0242] In a multi - frequency ultrasonic touch input device, the first signal 4002 at the first frequency and the second signal 4004 at the second frequency have different background and temperature drift characteristics. For example, when the temperature changes, the first signal 4002 and the second signal 4004 experience the same Δt(T). Due to different temperature drift characteristics, the same Δt(T) will translate into different phase delays for each frequency. For example, the first signal 4002 will have a first phase delay Δφ↓1(T) 4006, and the second signal 4004 will have a second phase delay Δφ↓2(T) 4008. The resulting phase delay differences can cause two different ADC output value patterns, which are respectively at temperatures Dout↓1(T) and Dout↓2(T), as depicted by lines 4010, 4012.

[0243] Therefore, signal drops can be measured at multiple frequencies to improve touch detection reliability and reduce false - trigger detection. If sudden signal drops are detected in all frequency channels, the touch input event can be processed. Multiple measurements can occur very quickly (<1ms) to ensure that the sudden signal drop is not due to temperature effects.

[0244] The multi - frequency ultrasonic touch input device can avoid false triggers by reducing the noise associated with environmental conditions. The touch input device can immediately perform a fast pulse - echo test to ensure that the touch event is real and not a false trigger caused by noise. In some embodiments, multiple tests can occur within 1ms.

[0245] Figure 41 is a multi - part graph 4100 according to a particular aspect of the present disclosure, which depicts the measurement of reflected ultrasonic signals across several frequencies using an ultrasonic input device and shows techniques for improving touch input detection. Different frequencies of ultrasonic signals can exhibit different variations due to temperature changes. Therefore, by using multiple ultrasonic frequencies for sensing, the ultrasonic input device can compare a suspected touch event with data from one or more other frequencies to ensure that the suspected touch event is confirmed by one or more other frequencies. The use of multiple frequencies can reduce the error rate.

[0246] Line 4106 may represent an energy signal associated with a 100 kHz frequency, line 4105 may represent an energy signal associated with a 1 MHz frequency, and line 4107 may represent an energy signal associated with a 10 MHz frequency. Line 4104 may represent a moving average threshold, e.g., Figure 37 threshold 3704, and for illustrative purposes, the moving average threshold is depicted only with respect to the 100 kHz frequency, but there may be corresponding thresholds for each frequency used (e.g., 1 MHz and 10 MHz). Although Figure 41 frequencies of 100 kHz, 1 MHz, and 10 MHz are used, any other suitable frequencies may be used. Although Figure 41 three different frequencies are used, any number of different frequencies may be used, such as two or more than three. A touch event may be registered only when the touch event is detected at all, most, or at least a threshold percentage of the different frequencies used for detection.

[0247] In some cases, instead of or in addition to driving the ultrasonic input device at different frequencies, the ultrasonic input device may drive an ultrasonic array with different phase delays to generate different beam patterns. Since different beam patterns may have different temperature characteristics, different beam patterns may be used similar to different frequencies to reduce errors and confirm suspected touch events.

[0248] Figure 42 is a schematic plan view according to a particular aspect of the present disclosure, which depicts a dual-frequency PMUT 4200. In some embodiments, a circular PMUT design may be fabricated to implement a multi-frequency transducer. The circular PMUT design may consist of multiple individual channels for transmission and reception for each frequency. In some cases, the multiple channels or transducers may be arranged concentrically. For example, the dual-frequency PMUT 4200 includes a first transmit / receive pair 4202 associated with a low frequency. The first transmit / receive pair 4202 may include a low-frequency transmit ring 4204 and a low-frequency receive ring 4206. The dual-frequency PMUT 4200 also includes a second transmit / receive pair 4208 associated with a high frequency. The second transmit / receive pair 4208 may include a high-frequency transmit ring 4210 and a low-frequency receive ring 4212. In various embodiments, the circular PMUT design may include a range of multiple frequencies from 2 to 10, and the frequency range may be from 1 MHz to 10 MHz. In some embodiments, frequencies less than 1 MHz may be used depending on the material layer and the specific application. A second PMUT array may be added for TOF measurement in the 1 MHz - 3 MHz range. In some cases, the frequency range for any array may be from 30 kHz to 50 MHz.

[0249] Figure 43is a schematic plan view according to a particular aspect of the present disclosure, which depicts a multi-frequency ultrasonic input device 4300 having a square design. The square sensor design may consist of a square grid of multiple individual channels for transmission and reception at each frequency. In some cases, one or more receiving channels may be located between multiple transmission channels. In such cases, the location of the receiving channels between the multiple transmission channels may facilitate the reception and detection of reflected signals. In an example, the multi-frequency ultrasonic input device 4300 may include various low-frequency transmitters 4302, low-frequency receivers 4304, high-frequency transmitters 4306, and high-frequency receivers 4308. The square design may include a nested style, such as the cross-shaped nested style depicted in Figure 43 and any other suitable style may be used. The various transmitters and receivers may be any suitable frequency, such as between 30 kHz and 50 MHz, between 1 MHz and 10 MHz, or any other suitable range. It should be understood that the frequencies described with reference to Figure 43 may be applicable to any suitable sensor array, for example, as described with reference to Figures 14A to 14G . VII. Feature Extraction

[0250] The systems and methods according to embodiments may allow for the extraction of features from signals received, for example, by an ultrasonic input device. The ultrasonic input device is capable of extracting features such as energy signals as well as physical characteristics. A. Discernible Energy Signals

[0251] Figure 44 is a set of three graphs 4402, 4404, 4406 according to a particular aspect of the present disclosure, which depict example signals 4412, 4414, 4416 attributable to three different users received by an ultrasonic input system. Each of the graphs 4402, 4404, 4406 depicts an energy measurement over time associated with a reflected signal detected by the ultrasonic input device.

[0252] The signal 4412 of graph 4402 is an example of a dry finger pressed quickly with a relatively small force. The dryness of the finger and the relatively small force show a relatively small dip in the energy measurement during the touch event. The speed of the press is seen in the relatively short dip duration in the energy measurement.

[0253] The signal 4414 of Chart 4404 is an example of a wet finger moderately pressed with a relatively heavy force. Both the moisture of the finger and the intensity of the press can result in a greater damping effect on the reflected signal and thus a deeper dip in the energy measurement. The speed of the press can be seen in a moderately wide dip in the energy measurement. Additionally, when the energy measurement first dips, the appearance of the initial and subsequent dips is more pronounced, indicating that little time is spent in contact with the surface before the full pressing force is initiated.

[0254] The signal 4416 of Chart 4406 is an example of a touch event pattern where the user touches the surface lightly before pressing and initiating a full touch event. The initial dip in the energy measurement and the relatively long delay until the subsequent full dip indicate that the user places the finger on the surface and waits for a short time before pressing the finger.

[0255] Although the signals 4412, 4414, 4416 can be used respectively to indicate a desired touch event due to the presence of sufficient dips in the energy measurement, each of the signals 4412, 4414, 4416 contains various distinguishable characteristics. Examples of distinguishable characteristics include the depth of the dip in the energy measurement, the width of the dip in the energy measurement, the presence of an initial dip before a subsequent and deeper dip in the energy measurement, the delay between the initial dip and the subsequent and deeper dip in the energy measurement, the rate of decrease and / or increase of the energy measurement as it enters and exits the dip (e.g., the rate of change of the energy signal at the edge of the dip), or any other characteristic of the energy measurement.

[0256] By extracting various characteristics from the energy measurement signal, different users can be distinguished and even recognized to enable additional user-based advanced functions. For example, after a training session, due to the specific way a user interacts with the ultrasonic input device, such as the way of touching (e.g., quick tap or place and press), the duration of the touch, the characteristics of the skin (e.g., the natural moisture or dryness of the finger), the intensity of the touch (e.g., light press or heavy press), or other characteristics distinguishable from the energy measurement signal, the ultrasonic input system can distinguish between a first user and a second user. Although the characteristics can be distinguished from the energy measurement signal, since the energy measurement signal can be acquired at high speed, they may not be easily perceivable by the user. Thus, the difference between a quick tap and a place and press can be easily distinguished from the energy measurement signal, but may be indistinguishable or not easily distinguishable from a visual inspection of the touch action.

[0257] Figure 45A set of graphs depicting energy measurement signals associated with a human finger, a water droplet, and placing the device on a table (e.g., placing an object on the sensor). For a human finger, the energy measurement signal inevitably has slight movement or variation, even during the duration of a touch event, which can be detected and recognized to confirm that the human finger is initiating a touch event. For a droplet or water droplet, if any, the energy measurement signal has specific characteristics, such as a sharp drop followed by an overall stable signal with not much change. Detection of these characteristics can be used to distinguish an actual intentional touch event from an accidental contact with other objects (such as falling water). Placing the device or other object on the sensor (e.g., a sensor mounted on a table) can have an energy measurement signal with specific characteristics, such as a relatively shallow drop followed by an overall stable signal with not much change, if any.

[0258] Thus, a system as described herein can determine an energy signal associated with a set of reflected ultrasonic signals. The system can then extract feature information associated with the energy signal and then determine an inference associated with the object based on the extracted feature information. Determining the inference can include using the feature information to determine whether the touch event is associated with a human finger or with dripping water. For example, as Figure 27 shown, dripping water (i.e., a water droplet) can cause a greater drop in the energy signal determined by the system than a human finger (i.e., a finger). A finger can have peaks and valleys (i.e., fingerprints) that reduce the amount of surface area placed on the sensor and thus reduce the amount of ultrasonic signal absorbed by the object.

[0259] Therefore, a criterion for the magnitude of the energy signal (e.g., corresponding to a sharp drop) can be used to distinguish between a finger touch and a water droplet. In addition, the energy signal is more consistent over time than a human finger. Thus, a criterion that the energy signal is within a specified range over a specified amount of time can be used to distinguish between a water droplet and a human finger. This measurement can be performed using the variation of the energy signal over time (e.g., standard deviation). Thus, the feature information can include the magnitude of the energy signal and / or the variation of the energy signal. Determining the inference can include comparing the magnitude and / or variation with corresponding thresholds to determine whether the touch event is associated with a human finger or with dripping water.

[0260] Figure 46 is a combined schematic diagram and a set of graphs depicting how temperature can be utilized to further identify whether a human finger is initiating a touch event. The energy measurement signal output by the sensor (e.g., a sensor chip and / or substrate) depends to some extent on the temperature of the sensor. As the temperature increases, the energy measurement signal tends to decrease.

[0261] Typically, the chip will be at room temperature (e.g., at or about 20 or 21 °C), while a human finger will be at body temperature (e.g., at or about 30 °C). When a living tissue (e.g., a human finger) initiates a touch event, heat will transfer between the tissue (e.g., the finger) and the chip. When the finger is hotter, it can cause a slight increase in the chip temperature. Since the energy measurement signal depends in part on the temperature of the chip and / or the substrate as a whole, fluctuations in the temperature of the chip and / or the substrate can be detected as a potentially steady increase or decrease in the energy measurement signal over time. As depicted in the graph in the lower left corner of Figure 46 when a warm finger is placed on a cooler sensor, heat transfer will cause the energy measurement signal to exhibit an overall downward slope. As depicted in the graph in the lower middle of Figure 46 when a cold finger is placed on a warmer sensor, heat transfer will cause the energy measurement signal to exhibit an overall upward slope (i.e., an upward trend). However, as depicted in the graph in the lower right corner of Figure 46 when something other than a living tissue (e.g., a finger) is placed on the sensor and that other object has a temperature at or near the same temperature as the sensor (e.g., both are at room temperature), the lack of heat transfer will cause the energy measurement signal to exhibit an overall flat slope. In summary, this temperature effect on the energy measurement signal can be used to identify when something in contact with the sensor is at or near body temperature, or at or near some other temperature. In some cases, the approximate temperature of the object initiating the touch event can be discerned by analyzing the general slope of the energy measurement signal.

[0262] In some cases, one or more temperature sensors can be used to measure the temperature of the chip and / or the substrate. Knowledge of the temperature of the chip and / or the substrate can help inform the determination of whether the object initiating the touch event is a human finger.

[0263] Figure 47 is a combined schematic diagram and graph according to a particular aspect of the present disclosure, which depicts finger touch and associated temperature information. In some cases, the ultrasonic input system can include a temperature sensor, such as within the chip, on the chip, or near the chip. The temperature sensor can provide a temperature signal associated with the temperature of the ultrasonic input system (e.g., a temperature sensor readout). Typically, when no touch event is initiated, the temperature signal will have minimal or no change, as the ultrasonic input system will remain at or near ambient temperature, such as room temperature. However, if a touch event is initiated using a human finger, an expected temperature change towards body temperature (e.g., a temperature increase from room temperature to body temperature) may occur. As depicted in the lower left graph of Figure 47 a human finger touch can be detected or confirmed by identifying a change in the temperature signal towards body temperature (e.g., at or near 30 °C). As depicted in Figure 47As depicted in the lower right chart, touch events initiated by an object other than a human finger (e.g., a room temperature object) will not cause the temperature of the ultrasonic input system to change towards body temperature. B. Discernible Physical Characteristics

[0264] Figure 48 FIG. 4802 is a schematic combined side view and FIG. 4804 is a signal diagram according to a particular aspect of the present disclosure, depicting the ridges 4806 and valleys 4808 of a fingerprint that initiate a touch event on an ultrasonic input system. When a user places a finger on a surface associated with the ultrasonic input device 4810, the ultrasonic input device 4810 can detect a portion of the user's fingerprint. Generally, the ultrasonic input device 4810 can sense an area less than the entire fingerprint of the user, although this is not always required.

[0265] The ultrasonic input device 4810 can identify the ridges 4806 and valleys 4808 of the user's fingerprint (e.g., a portion of the user's fingerprint). At the ridges 4806, the ultrasonic input device 4810 will detect a decrease in the energy measurement of the reflected signal due to the damping effect of the flesh of the ridges 4806. However, at the valleys 4808, there is no such damping effect.

[0266] Thus, the ultrasonic input device 4810 that measures the finger as depicted in the schematic side view 4802 can generate a signal diagram 4804 showing the ridges 4806 and valleys 4808. As visible in the signal diagram 4804, darker regions represent dips in the energy measurement of the reflected signal, while brighter regions represent signals closer to the baseline energy measurement. Although the entire fingerprint cannot be discerned from the field of view of the ultrasonic input device 4810, many ridges 4806 and valleys 4808 can be discerned. By measuring the widths of the ridges 4806 and valleys 4808 and the distances between valleys and between ridges (e.g., the ridge-to-ridge distance 4812), the ultrasonic input device 4810 can discern one finger from another. In an example case, an adult's finger may exhibit wider ridges 4806 and valleys 4808 than a young person's finger. Thus, in a household with adults and children, the ultrasonic input device 4810 can discern between two users based on the discernible physical characteristics of the user's finger, such as fingerprint characteristics. In some cases, the presence of a repeating line pattern (e.g., the pattern of ridges 4806 and valleys 4808) can be used to confirm or determine whether the object initiating the touch event is a human finger.

[0267] In some cases, discernible physical characteristics, such as fingerprints, can be used in conjunction with discernible energy signals to further identify a user.

[0268] Figure 49is a schematic diagram according to a particular aspect of the present disclosure, which depicts example reflected signals 4924, 4925 attributable to the same user initiating touch events with glove 4908 and without glove 4906 received by ultrasonic input system 4902. A first curve 4910 associated with ultrasonic input device 3502 having a touch input from a user without a glove 4906 shows transmitted signal 4922 and a first set of reflected signals 4924. The first set of reflected signals 4924 shows the characteristic attenuation of the reflected signals associated with the touch event. A second curve 4920 associated with ultrasonic input device 3502 having a touch input from a user with a glove 4908 on shows transmitted signal 4922 and a second set of reflected signals 4925. The second set of reflected signals 4925 shows the characteristic attenuation of the reflected signals associated with the touch event, which is somewhat similar to the first set of reflected signals 4925, but may have additional attenuation due to the presence of glove 4912. The first set of reflected signals 4924 can be processed to generate a first output voltage 4932. Similarly, the second set of reflected signals 4925 can be processed to generate a second output voltage 4933.

[0269] Accordingly, ultrasonic input system 4902 can distinguish between a gloved hand and an ungloved hand. In some cases, depending on whether the user is wearing a glove, certain actions may be available or unavailable. For example, in a medical facility, certain functions associated with the ultrasonic input system may be unavailable unless the user is wearing a glove to ensure proper protection is in place. C. Feature extraction and use

[0270] Figure 50 is a flowchart according to a particular aspect of the present disclosure, which depicts 5000 for extracting features from signals of an ultrasonic input system. Figure 50 The method shown in will be described in the context of a system including an ultrasonic input device and one or more data processors that determine an energy signal from a touch event. However, it should be understood that the present invention can be applied to other situations.

[0271] At optional block 5002, a baseline signal can be received by the ultrasonic input system. The baseline signal can be an energy measurement associated with no touch event (e.g., when no user touches the surface coupled to the ultrasonic input device). Removing such a baseline signal can result in a more efficient sampling range during analog-to-digital conversion, e.g., as described herein at least with reference to Figure 27As described above. For example, an ultrasonic input system may transmit a first signal. Subsequently, any suitable number of reflected ultrasonic signals and reflection-emission signals may be measured by the ultrasonic input system. Based on the characteristics of the received signal, it may be determined that the signal is not associated with a touch event (e.g., a finger touching the outer surface). For example, the received signal may indicate a baseline signal associated with an airborne signal. Further example details of the baseline signal are described herein.

[0272] At block 5004, the system may use an ultrasonic input device to transmit the transmitted signal. The ultrasonic input device may be coupled to a material layer having an outer surface that is positioned opposite the material layer of the ultrasonic input device. The transmitted signal may pass through the material layer toward the outer surface. As described in detail herein, any number of reflected ultrasonic signals and reflection-emission signals may be generated from the initially transmitted ultrasonic signal until the signal becomes too weak to be reflected and / or detected.

[0273] At block 5006, a signal associated with a touch event is received. For example, the system may receive a set of reflected ultrasonic signals associated with the transmitted signal. The received signal may be a measurement of the energy associated with the reflected ultrasound. The signal received at block 5004 may depend on how the touch event is initiated (e.g., the timing of the touch, the manner of the touch, the amount of force of the touch, the physical characteristics of the object initiating the touch).

[0274] At block 5008, one or more data processors of the system may determine an energy signal associated with a set of reflected ultrasonic signals that is associated with a touch event between an object and the outer surface of a material layer coupled to the ultrasonic input device.

[0275] As an example, referring to Figure 27 , flowchart 2700 includes transmitting and receiving ultrasonic signals, as shown in first graph 2702. First graph 2702 shows an analog measurement of a first signal 2703 for the transmitted ultrasonic signal and analog measurements of a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E of a set of reflected ultrasonic signals associated with the ultrasonic input device. Flowchart 2700 may include an ultrasonic input device having an analog summing or integrating circuit 2720 and a summed voltage output 2722.

[0276] The second graph 2708 shows a first signal 2703 and a set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E, which respectively correspond to the energy of a corresponding one of the set of reflected ultrasonic signals. For illustrative purposes, the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E are depicted as solid lines, which overlap with a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E from a first graph 2702 shown as a dashed line.

[0277] At block 5010, after determining the energy signals associated with the set of reflected ultrasonic signals, features can be extracted from the signals associated with the touch event. The extracted features can be any suitable characteristic of the signal, which can be distinguishable and / or capable of informing an inference. One or more data processors can be configured to extract feature information associated with the energy signals in any suitable manner.

[0278] In some embodiments, extracting feature information can include identifying a pattern in the energy signals associated with a dip in the energy measurement (associated with a touch event). For example, when an individual places their finger on the system, particularly on the outer surface, the individual's finger can absorb at least a portion of the emitted ultrasonic signal, thereby causing a dip in the energy measurement.

[0279] The pattern can be identified in any suitable manner described herein. For example, in some embodiments, identifying a pattern in the energy signals can include identifying the depth of the dip, the duration of the dip, the presence of subsequent dips after the dip, the delay between one dip and another, and / or the rate of change of the energy signal at the edge of the dip (e.g., during finger land or removal). In other embodiments, identifying the pattern can include identifying a change in the energy signal attributable to temperature drift in a material layer, as described in detail herein.

[0280] In some cases, extracting features (i.e., feature information) at block 5010 can include comparing the signal with a stored historical signal(s), such as to determine whether the signal received at block 5010 matches the stored signal associated with a particular user. In some cases, extracting features at block 5010 can include identifying a pattern in the received signal, such as to identify that the received signal is attributable to a distinct tap or a place-and-press action. In some cases, extracting features at block 5010 can include measuring a characteristic of the received signal. Any distinguishable characteristic of the received signal can be measured and used to make a determination or inference about the source of the touch event.

[0281] At block 5012, an inference can be determined based on the extracted feature information. One or more data processors can be configured to determine an inference associated with an object in any suitable manner based on the extracted feature information.

[0282] For example, in some embodiments, determining the inference can include estimating a relative temperature of an object based on an identified change in an energy signal attributable to a temperature drift in a material layer. For example, an individual touching the outer surface of the material layer can have a body temperature that is higher than the ambient temperature and / or the temperature of the material layer. As described herein, the determined energy signal can be affected by temperature and thus allows one or more data processors to determine an inference of a temperature measurement and / or a temperature drift (e.g., as measured by a temperature sensor as described below).

[0283] In other embodiments, one or more data processors can determine an inference by comparing the identified pattern with stored data. The stored data can be associated with a previous touch event on the outer surface. For example, a previous touch event on the outer surface may have been performed by an individual. As described herein, the current touch event can be compared with the previous touch event to determine whether the current touch event was also performed by an individual.

[0284] In other embodiments, one or more data processors can determine an inference by using the feature information to determine that a touch event is associated with a human finger, a bare human finger, a wet human finger, a dry human finger, and / or a gloved human finger. For example, as described herein, the determined energy signal can be affected by one or more characteristics of the (multiple) individual fingers placed on the outer surface of the material layer. One or more data processors can also determine an inference by using the feature information to determine the manner of touch of the touch event (e.g., tap, double - tap, place, press, etc.), the touch intensity associated with the touch event, and / or the physical characteristics of the object.

[0285] In some embodiments, determining the inference can include identifying that the object is associated with one of a plurality of users based on associating the touch event with the manner of touch of the touch event, the touch intensity associated with the touch event, and / or the physical characteristics of the object. The physical characteristics of the object can include measurements associated with a portion of a fingerprint that touches the outer surface.

[0286] In some embodiments, one or more data processors can determine an additional signal associated with an additional sensor associated with the ultrasonic input device (e.g., Figure 29 a temperature sensor). Subsequently, one or more data processors can further use the additional signal to determine an inference. The additional sensor can include any suitable additional sensor associated with the ultrasonic input device. For example, the additional sensor can include a temperature sensor, a pressure sensor, a charge - coupled device, etc.

[0287] For example, the system may include a temperature sensor. The temperature sensor may record, for example, the temperature of the outer surface of the system over time. Since a human fingertip has a specific physical size and temperature range, when a human touches the outer surface, one or more data processors may determine that the touch event is caused by a human finger. As an illustrative example, the temperature sensor may record the temperature of at least a portion of the outer surface at a predetermined interval (e.g., 1 ms, 0.1 s, 1 s, etc.). The temperature sensor may record the ambient temperature (e.g., 70°F). When a user touches the outer surface during a touch event, the system may record an energy signal, which may include, for example, a sink of energy. During the touch event, the temperature sensor may continue to measure the temperature of the outer surface. A human finger in contact with the outer surface may increase the temperature of the outer surface, causing the temperature sensor to record an increase in temperature. For example, a human finger may be approximately 98°F. The temperature sensor may record the temperature between the ambient temperature of 70°F and the human finger temperature of 98°F because the finger will heat the outer surface and the temperature sensor.

[0288] The temperature measured by the temperature sensor may be an additional signal associated with an additional sensor (e.g., a temperature sensor) associated with the ultrasonic input device. One or more data processors may use the additional signal together with the energy signal to make a determination. For example, one or more data processors may determine that the sink of the energy signal and the temperature rise from the ambient temperature to a higher temperature between the ambient temperature and the average human body temperature indicate that the touch event indicates a human finger touching the outer surface. In some cases, one or more data processors may use the temperature data from the temperature sensor to determine whether the signal change is the result of a human touch or from another object in contact with the outer surface (e.g., a table, pocket fabric, pen / stylus, etc.). For example, when in contact with a table, pocket fabric, pen / stylus, etc., the temperature sensor may not measure as large an increase in temperature as when in contact with a human finger.

[0289] In some cases, the temperature sensor may be a known (i.e., predetermined) distance from the finger. For example, the temperature sensor may be located on the side of the outer surface opposite the finger. In such a case, during the processing of the additional signal associated with the additional sensor (e.g., a temperature sensor), a heat transfer problem with known boundary conditions and initial values may be solved to determine what the temperature is at the outer surface.

[0290] In some embodiments, the additional sensors can include a pressure sensor and / or a strain gauge. For example, a typical touch from a human finger can apply a specific force and strain on the outer surface that can be propagated to the additional sensors. The pressure sensor and / or the strain gauge can measure the force and / or strain applied by the finger to the system. One or more data processors can determine that the force and / or strain measured by the pressure sensor and / or the strain gauge indicate the typical force and / or strain of a finger touch. One or more data processors can also determine whether the energy signal indicates a finger touch. If both the additional signal from the pressure sensor and / or the strain gauge and the energy signal indicate a finger touch, then one or more data processors can determine that the touch event is a finger touch.

[0291] In some cases, the additional sensors can include a strain gauge. The strain gauge can detect the deflection of the surface associated with the touch event and can output an electrical signal. The stronger the touch event (e.g., the greater the force applied by an object such as a finger on the outer surface), the greater the deflection applied to the strain gauge. Thus, the strain gauge can output a larger electrical signal.

[0292] At block 5014, one or more data processors can generate an output signal associated with the determined inference. The output signal can include any suitable output generated based on the determined inference. In some embodiments, the output signal can indicate a specific action that can be performed by one or more data processors and / or an external device.

[0293] In some embodiments, one or more data processors can perform an action based on the extracted feature(s). The action can include any suitable process that can occur based on the output signal. In an example, if the extracted feature is used to identify a specific use, the action performed can be to authenticate or authorize user access to a resource. In another example, if multiple users have preset customizations for a specific ultrasonic input system, the extracted feature information can be used to determine which user is interacting with the ultrasonic input system and thus perform a customized action for that specific user. In some cases, performing the action can include allowing or denying access to a resource, such as denying access to a room or a tool when the extracted feature indicates that the user is not wearing gloves (when gloves are required). VIII. Machine Learning Decision Algorithm

[0294] Figure 51 is Chart 5100 according to a particular aspect of the present disclosure, which depicts a machine learning decision algorithm for improving touch detection. As referred to Figure 37As described above, the weight parameters can be used to drive various decisions regarding when a touch event is detected or not detected. In some cases, machine learning methods can consider the sensor output values as well as the slope between the sensor values and the previous sensor values to generate an inference as to whether a touch event has occurred or not. The machine learning method can use a decision function (f), such as: f = w 0 X[n] + w 1 X[n - 1] + w 2 X[n - 2] + … + w m X[n - m] + w s0 S[n] + W s1 S[n - 1] + … + w sm S[n - m] where w n and w sn are weight parameters, X[n] is the current sensor output, X[n - 1] is the previous sensor output, X[n - m] is the m-th previous sensor output, S[n] is the slope of the current sensor output (e.g., compared to the immediately previous sensor output), S[n - 1] is the slope of the previous sensor output, and S[n - m] is the slope of the m-th previous sensor output. In some cases, other parameters can be used in the decision function.

[0295] The weight parameters of the decision function can be trained on a corpus of data to generate a decision boundary between inputs that are considered touch events and inputs that are not considered touch events, as depicted in Chart 5100. Thus, for any given sensor output and slope of the sensor output, a point on Chart 5100 can be identified, and if that point falls above the decision boundary, those sensor outputs and the slope of the sensor output can be considered to indicate a touch event. IX. Intelligent Touch Event Detection

[0296] The systems and methods according to embodiments can allow for a touch event detection framework. Embodiments allow for an adaptive threshold for touch event detection. The adaptive threshold scheme can involve using a continuously adaptive threshold to identify touch events from the energy signal of the sensor. Embodiments also allow for a recurrent neural network for touch event detection and / or a recurrent neural network for state classification. A. General Touch Event Detection Framework

[0297] Figure 52 is a flowchart according to a particular aspect of the present disclosure, which depicts a process 5200 for detecting touch events. The process 5200 can be performed by any suitable device, including Figure 7processor 722 and / or computing device 724. In some cases, data from multiple sensors can be used in any of blocks 5202, 5204, 5206.

[0298] At block 5202, energy signal data is accessed. The energy signal data is signal data from an ultrasonic sensor that indicates the amount of energy sensed by the ultrasonic sensor over a period of time, such as the energy signal 3614 depicted and described with reference to Figure 36 depicted and described. Any suitable period of time can be used.

[0299] At block 5204, a touch event can be identified based on the energy signal. Identifying the touch event can include determining whether a touch event has occurred based on the energy signal. In some cases, identifying the touch event at block 5204 can include outputting a touch signal. The touch signal can indicate whether the associated energy signal is inferred to be associated with a touch event.

[0300] At optional block 5206, a state classification can be identified from the touch event data (e.g., the touch signal from block 5204). In some cases, the state classification can be identified from the touch event data and the associated energy signal data. The state classification can be a classification associated with the touch event. Any suitable classification can be determined, such as the type of touch event that has occurred. Examples of suitable state classifications related to the type of touch event that has occurred include single tap, double tap, triple tap, n taps, hold (e.g., touch and hold), tap and hold (e.g., tap followed by touch and hold), press (e.g., longer than a tap), double press, press and hold (e.g., press followed by touch and hold), hold and press (e.g., touch and hold for a duration followed by press), and grasp (e.g., hold with more surface area or other characteristics). The state classification can be determined and can be associated with the touch event based on a trigger value. Examples of suitable state classifications related to other information associated with the touch event can include whether the user is wearing gloves, whether the user appears older or younger (e.g., based on the distance between fingerprint ridges), whether the user appears to be a pre-identified user, or other such classifications.

[0301] As an example, another classification can include hydration and / or sweating of a user's finger and / or body. The system can detect a user's hydration and / or sweating, for example, by determining an ultrasound signal absorption that is lower than a user's typical ultrasound signal absorption. When a user's finger is drier, the finger will absorb less ultrasound signal. Thus, different levels of thresholds for the amplitude and its change over time can be used. For example, a wet finger can cause a more uniform decrease in the energy signal than a dry finger. Thus, a criterion for the magnitude of the energy signal (e.g., corresponding to a sharp decrease) can be used to distinguish between a dry finger and a wet finger. Additionally, since there is additional water in a wet finger, over time, the energy signal will be more consistent in the case of a wet finger than in the case of a dry finger. Thus, a criterion that the energy signal is within a specified range over a specified amount of time can be used to distinguish between a wet finger and a dry finger. The change of the energy signal over time (e.g., standard deviation) can be used to perform this measurement. Thus, the feature information can include the magnitude of the energy signal and / or the change of the energy signal. The inferential determination can include comparing the magnitude and / or the change with corresponding thresholds to determine whether the touch event is associated with a wet finger or a dry finger.

[0302] In some cases, depending on the orientation and placement of the sensor, any number of classifications can be used. In some cases, the state classification can be trained such that the recognition of the state classification at block 5206 can refer to training data or a model generated using the training data. B. Adaptive Thresholds for Touch Event Detection

[0303] Figure 53 is a schematic diagram according to a particular aspect of the present disclosure, which depicts an adaptive threshold scheme 5300 for identifying touch events. The adaptive threshold scheme 5300 can be executed, in part or in whole, on a processor coupled to an ultrasound sensor, such as Figure 7 processor 722. Figure 53 The method described in can be executed by an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other suitable device and / or controller described herein. The scheme 5300 does not require machine training; however, one-time tuning for each new overlay material to which the sensor is coupled can improve detection. The adaptive threshold scheme 5300 can operate outside of discrete time periods or frames such that at any time, the scheme 5300 can analyze current data from the current observation and historical data from any number of past observations.

[0304] Adaptive threshold scheme 5300 involves identifying touch events from an energy signal of a sensor using a continuously adaptive threshold. The threshold is a continuously tracked version of the energy signal that has been filtered by an adaptive threshold update function, and the filter parameters of the adaptive threshold update function are updated adaptively based on the energy signal, a historical threshold, and optional trigger data (e.g., whether a touch event is occurring or has recently occurred). Whenever the energy signal exceeds the adaptive threshold (i.e., is less than, greater than, less than or equal to, or greater than or equal to the adaptive threshold), a sensor trigger (e.g., a touch event) can be identified. Optionally, scheme 5300 can analyze the current and past trigger history (e.g., based on a specific number of recently observed touch event signals) to identify a current state classification, such as determining whether a touch event is a tap, a press, a hold, or some other action. For example, analyzing the touch event signal (e.g., the trigger history) can show how many times the energy signal drops below the threshold within a particular time frame, how far the energy signal drops, how long the energy signal is below the threshold, and other such characteristics that can be used by the scheme to make a determination about the current state of the sensor.

[0305] Sensor data 5302 can include current and historical sensor data, which is an energy signal (e.g., energy data) from a sensor such as Figure 7 the ultrasonic sensor 702 or is based on an energy signal (e.g., energy data) from a sensor such as Figure 7 the ultrasonic sensor 702. The sensor data 5302 can be provided to an adaptive threshold update function 5304, which can use this data to generate threshold data 5306. The adaptive threshold update function 5304 can use only the current sensor data or both the current sensor data and the historical sensor data, along with any additional parameters, as appropriate. The adaptive threshold update function 5304 acts as a kind of low-pass filter, which allows slow changes in the energy signal to have a significant impact on the adaptive threshold, while fast changes in the energy signal have a minimal or negligible impact on the adaptive threshold. Thus, factors that cause slow changes in the energy signal, such as temperature changes in a room, can be automatically compensated for in the adaptive threshold, while factors that cause fast changes in the energy signal, such as a finger press, will be correctly detected as exceeding the threshold.

[0306] Threshold data 5306 can represent a threshold such that an energy signal should be considered to have triggered a touch event if it exceeds (e.g., is below) the threshold. A condition analyzer 5308 can compare sensor data 5302 (e.g., energy data) and threshold data 5306 to determine whether a triggering event (e.g., a touch event) has occurred. For example, when a current energy signal from sensor data 5302 drops below the current threshold in threshold data 5306, it can be assumed that a touch event has occurred. In some cases, the condition analyzer 5308 can also provide feedback to an adaptive threshold update function 5304 to update the parameters of the adaptive threshold update function 5304. For example, the speed, degree, or number of times the energy signal drops below the threshold can be used to manipulate how the adaptive threshold update function 5304 generates threshold data 5306 from sensor data 5302. If a triggering event (e.g., a touch event) is detected, the condition analyzer 5308 can output one or more trigger values in trigger data 5310. Trigger data 5310 can include current trigger data and historical trigger data indicating whether a triggering event (e.g., a touch event) has occurred. For example, the threshold data can be updated based on energy data, trigger data, and threshold data, where updating the threshold data includes generating subsequent thresholds.

[0307] In some cases, trigger data 5310 can be used by the condition analyzer 5308 to further inform the decision of the condition analyzer 5308 as to whether a triggering event has occurred. For example, historical trigger data can be used by the condition analyzer 5308 to confirm or refute a possible triggering event. However, in some cases, trigger data 5310 can be used by the condition analyzer 5308 to provide updated parameters to the adaptive threshold update function 5304 such that the adaptive threshold update function 5304 is further updated based on current and / or historical trigger data.

[0308] In some cases, trigger data 5310 can be passed to a trigger analyzer 5312 to determine a sensor state 5314. The trigger analyzer 5312 can obtain information from the trigger data 5310 to determine whether the most recent triggering event is a touch, a tap, or some other such classification of a triggering event. Subsequently, the trigger analyzer 5312 can output its determination as the sensor state 5314. The sensor state 5314 can indicate not only the triggering event but also the classification of the state associated with the triggering event. For example, while trigger data 5310 can take the form of indicating whether a trigger has occurred (e.g., a binary signal), the sensor state 5314 can also take the form of indicating what state the sensor is in. Example sensor states include hold, tap, press, double tap, etc. The classification of the sensor state can be selected from a predetermined list, each list having a different pattern of energy signals.

[0309] As used herein, various signals may be considered to include a plurality of data points, which include a current data point (e.g., the most recent data point) and any number of previous data points. As used herein, the term historical data may include the current data point and past data points. The data points may be represented as analog or digital signals.

[0310] Figure 54 is an example graph 5400 according to a particular aspect of the present disclosure, which depicts an energy signal 5402 associated with identifying a touch event and an adaptive threshold 5404. As referenced Figure 53 above, the energy signal 5402 may be from any suitable sensor data 5302, such as Figure 7 of the ultrasonic sensor 702. Such as referenced Figure 53 above, the adaptive threshold 5404 may be threshold data 5306 generated from the energy signal 5402.

[0311] As depicted in Figure 54 the gradual change in the intensity of the energy signal 5402 throughout the graph 5400 is reflected in the adaptive threshold 5404. Specifically, as visible in the relatively constant segments when no rapid spikes occur, the average intensity of the energy signal 5402 steadily decreases over time, which subsequently reflects itself as steadily decreasing over time in the adaptive threshold 5404. However, the rapid change associated with the touch event depicted as a rapid negative spike in the energy signal 5402 is not fully reflected in the adaptive threshold 5404, which allows the energy signal 5402 to drop below the adaptive threshold 5404. Whenever the energy signal 5402 drops below the adaptive threshold 5404, the system may register a touch event. Based on various attributes of the detected touch event (such as frequency, intensity, duration, and other such attributes as disclosed herein), a determination may be made regarding the classification of the touch event (e.g., the state of the sensor), such as whether the sensor is being tapped, pressed, held, double-tapped, or otherwise manipulated. C. Recurrent Neural Network for Touch Event Detection

[0312] Figure 55 is a schematic diagram according to a particular aspect of the present disclosure, which depicts a general recurrent neural network 5500. The recurrent neural network 5500 is a data analysis technique that operates to convert input data 5502 into output data 5508. The recurrent neural network 5500 can be used to identify touch events from the energy signal (e.g., as visible in Figure 52 box 5204) and to identify state classifications from the touch event and / or the energy signal (e.g., as in Figure 52one or both of which are visible in the frame 5206. For example, the input data 5502 can be an energy signal or energy data from any suitable sensor, such as Figure 7 the ultrasonic sensor 702, and the output data can be a trigger signal or a status classification signal. For example, the ultrasonic sensor can provide energy data to a recurrent neural network to generate output data indicating the occurrence of a touch event.

[0313] The recurrent neural network 5500 can pass the input data 5502 through any number of nodes across any number of layers until the output data 5508 is generated. In some cases, the output data (e.g., the output data 5508) may include status classification information associated with a touch event. One or more hidden layers 5504, 5506 can be located between the input data 5502 and the output data 5508. Within each hidden layer 5504, 5506, the nodes 5510 can process the input data into outgoing data. At the node 5510, any number of inputs can be received and processed (e.g., summed and passed through a function) to generate an output. In the example node 5514, three inputs are received (e.g., weighted versions of other layers, such as w i k,1 ; w i k,2 ; w i k,m((i-1) ) and they are summed, and passed through a function to generate a single output (e.g., a i k ). In other words, in some cases, the output of a node can be a decision function that is a linear combination of the outputs of the previous layer, optionally with additional feedback as described with reference to the tapped delay line. In other cases, the inputs to the node 5514 can be linearly combined and then passed to another function f. The function f can be an activation function, which can be linear or non-linear. For example, the activation function can include a sigmoid function, a hyperbolic tangent function, a rectified linear unit (ReLU) function, an identity function, and / or any suitable activation function. The activation function can constrain the output to a probability form between any suitable bounds (e.g., 0 to 1, -1 to 1, -0.5 to 0.5, etc.). Subsequently, the output from the node 5510 can be passed to one, some, or all of the nodes in the subsequent layer, or, in the case of being in the final layer, can be passed to the output and used to generate the output data 5508 together with other outputs from the same layer (e.g., by summing or other functions).

[0314] As in Figure 55As depicted, the recurrent neural network 5500 may also utilize tapped delay lines 5512. Each tapped delay line 5512 may be used to provide inputs to one, some, or all of the nodes 5510 of a particular layer (e.g., layer 5504), which may receive the current or delayed output from that layer or a subsequent layer (e.g., layer 5506 or output 5508) via the tapped delay line 5512. In some cases, the tapped delay line 5512 may also provide a delayed version of the input data 5502 as an input to each node 5510 of a particular layer. An example tapped delay line 5516 depicts a vector of inputs from layer j 5506 (e.g., a j ), which is delayed and output as an input to various layers of layer i 5504 (e.g., wd i,j ). In this way, the output from a subsequent layer (e.g., historical output) can inform earlier layers in the recurrent neural network 5500. The tapped delay line 5512 may include data from any suitable length of time. For example, the tapped delay line 5512 may provide data from a single past frame or from multiple frames.

[0315] Figure 55 The recurrent neural network 5500 in

[0316] is depicted as having a single input (e.g., input data 5502), a hidden layer i 5504 containing m(i) different nodes, a hidden layer j 5506 containing m(j) different nodes, and a single output (e.g., output data 5508). In some cases, the recurrent neural network 5500 used in accordance with particular aspects of the present disclosure may include any suitable number of inputs, layers, nodes, and outputs. By providing labeled sensor data to the system, the recurrent neural network 5500 may be pre-trained via supervised machine learning to allow the function of each node (e.g., the weight value of each node) to be updated until the recurrent neural network 5500 performs as desired.

[0317] In some embodiments, the set of state classifications can be selected from a plurality of available state classifications via user input. The plurality of available state classifications can include, for example, a list of state classifications available for the user to select. For example, in some cases, the plurality of available state classifications can include tap, double - tap, press, and hold. In other cases, the plurality of available state classifications can include tap, press, double - press, and grip. The plurality of available state classifications can include any suitable combination of state classifications.

[0318] In other embodiments, the plurality of historical touch events can also include a plurality of non - touch events. The non - touch events can facilitate training an additional recurrent neural network to reject false - positive events. The non - touch events can include, for example, touch events indicating no touch. For example, a non - touch event can be associated with a sink in the energy signal, but can be associated with an event where a drop touches the outer surface rather than through a finger, as described herein. In some cases, the user can be prompted to touch the outer surface against other objects (e.g., a pen, fabric, etc.) that the user wishes to classify as non - touch events. In this way, the user can provide non - touch event data associated with situations where the user does not want the device to determine a touch event. In some cases, the device can provide output data from a recurrent neural network to an additional recurrent neural network to generate state classification information associated with the touch event.

[0319] Figure 56 is a schematic diagram according to a particular aspect of the present disclosure, which depicts an example recurrent neural network 5600 for identifying trigger events. The recurrent neural network 5600 can be particularly useful for effectively and accurately detecting trigger events from an ultrasonic energy signal.

[0320] At input 5602, the sensor data can be from such as Figure 7The energy signal of the ultrasonic sensor of ultrasonic sensor 702 is provided to recurrent neural network 5600 in the form of. The energy signal is passed on two nodes 5610, 5612 of hidden layer 1 5604 and on tapped delay line 5618, which provides one or more delayed signals to nodes 5610, 5612 of hidden layer 1 5604 based on the energy signal from input 5602. For example, tapped delay line 5618 can be set to provide the energy signal of the last three or four frames to hidden layer 1 5604. In addition, nodes 5610, 5612 of hidden layer 1 5604 can take the output of tapped delay line 5620 as an additional input, which can be configured to output a set number of past frames of output 5608 of recurrent neural network 5600. For example, tapped delay line 5620 can be configured to provide the immediately previous frame of the data output via output 5608 of recurrent neural network 5600 as an input to hidden layer 15604. The outputs from nodes 5610, 5612 of hidden layer 15604 can then be passed as inputs to nodes 5614, 5616 of hidden layer 2 5606. Subsequently, the outputs of nodes 5614, 5616 of hidden layer 2 5606 can be passed as trigger data to output 5608 (e.g., combined and output).

[0321] In some cases, a strong combination of efficiency and accuracy for identifying trigger data from ultrasonic energy signals can be to use a recurrent neural network 5600 having a first layer and a second layer, where the first layer receives certain combinations of sensor data, past sensor data, and past trigger data, and the second layer receives the output of the first layer. The output from the second layer can be used to generate a trigger data output.

[0322] Recurrent neural network 5600 can be pre-trained and / or trained by the user. The training of recurrent neural network 5600 can include providing energy signals that are appropriately labeled as touch events or not touch events. The training data can be provided by a supplementary input device (e.g., a physical button or electrical contact) that simultaneously records the touch event when the ultrasonic sensor detects an energy signal associated with the touch event; or by associating the recorded energy signal with the touch event, such as by instructing the user to initiate touch events at specific times or in a specific rhythm. Once the training data has been obtained, the recurrent neural network can be programmed or trained by supervised machine learning, which allows the function of each node (e.g., the weight value of each node) to be updated until recurrent neural network 5600 performs as desired (e.g., accurately identifies trigger events). In some cases, recurrent neural network 5600 can be re-trained whenever the ultrasonic sensor is coupled to a new material stack.

[0323] Due to the nature of typical recurrent neural networks, the output 5608 can take the form of a number. Inferring appropriate trigger data from this number can include applying a threshold to the actual output 5608 of the recurrent neural network 5600. For example, if the recurrent neural network 5600 outputs a number between 0 and 1.0, a threshold can be set between these two numbers, and if above this threshold, the output can be considered a trigger event (e.g., a touch event), and if at or below this threshold, the output can be considered not a trigger event (e.g., no touch event), or vice versa. In this example, the threshold can be set to 0.5, so an output of 0.55 can be considered a touch event. In some cases, rather than simply retraining the entire neural network by adjusting the threshold, the recurrent neural network 5600 can adjust its sensitivity. Thus, to reduce the likelihood of false triggers (e.g., reduce sensitivity), the threshold can be moved from 0.5 to 0.6. Thus, the same output of 0.55 will not be considered not a touch event.

[0324] In some cases, an unsupervised machine learning model can analyze data that has not yet been labeled. The unsupervised machine learning model can include any suitable type of unsupervised machine learning model, e.g., clustering (e.g., k-means, hierarchical clustering, etc.), anomaly detection, etc. The plurality of trigger values can be measured by the device while the user is using it. For example, the user can perform any appropriate number of touch events that can be recorded. At this point, the plurality of trigger values from the touch events may not yet be labeled as, e.g., taps, holds, presses, etc. The plurality of trigger values can at least include 0s and 1s, which indicate touches detected at a particular time. Data items including consecutive trigger values can be used to determine a state.

[0325] The unsupervised machine learning model can group the plurality of trigger values or data items created therefrom (e.g., using a clustering method). As an illustrative example, the unsupervised machine learning model can group data items similar to (0, 0, 0, 1, 1, 1, 0, 0, 0) into a first cluster. The unsupervised machine learning model can group data items similar to (0, 0, 1, 1, 0, 0, 1, 1, 0, 0) into a second cluster. The unsupervised machine learning model can create any suitable number of clusters based on the plurality of trigger data.

[0326] The user may be prompted to provide supervised data (e.g., provide a desired touch event). In some embodiments, the recurrent neural network 5600 may further determine the classification of clusters determined from an unsupervised machine learning model based on the supervised data as part of training the recurrent neural network 5600. For example, a cluster having data items similar to (0, 0, 0, 1, 1, 1, 0, 0, 0) may be labeled as "tap", while a cluster having data items similar to (0, 0, 1, 1, 0, 0, 1, 1, 0, 0) may be labeled as "double tap". D. Recurrent Neural Network for State Classification

[0327] Figure 57 is a schematic diagram according to a particular aspect of the present disclosure, which depicts an example environment 5700 for touch detection and state classification using a set of recurrent neural networks 5706, 5708. The environment 5700 shows a user interface 5702 that may be presented on a computing device (such as Figure 7 computing device 724) for generating information related to how an ultrasonic sensor (e.g., Figure 7 ultrasonic sensor 702) can interpret the energy signal. The user interface 5702 allows the user to select those states to be detected and identified. Subsequently, the recurrent neural networks 5706, 5708 can be trained to recognize the selected states. However, it should be understood that the recurrent neural networks 5706, 5708 are not limited to recognizing the selected states. For example, the recurrent neural networks 5706, 5708 can be trained to recognize a trigger output based on trigger data.

[0328] In some cases, a single recurrent neural network can be used to generate an output indicating a state based on a received energy signal as input. However, as depicted in Figure 57 a first recurrent neural network 5706 can receive the energy signal as input 5704 and output a trigger signal, which can then be passed as input to a second recurrent neural network 5708, which can then output a state signal as output 5710. In Figure 57 the output 5710 is depicted as a graph in a hypothetical feature space. In this hypothetical feature space, different possible states are distinguishable based on their positions in the hypothetical feature space. The hypothetical feature space can be depicted as two-dimensional, although it can actually be based on any number of dimensions, including one dimension or more than two dimensions. The output 5710 of the environment 5700 can indicate a particular state of the ultrasonic sensor, such as being tapped, touched, pressed, double tapped, or any other suitable state.

[0329] As in Figure 57The environment 5700 depicted herein shows a single input and depicts four possible energy patterns that can be placed into the input. However, in some cases, a single environment 5700 can utilize multiple sensors to provide multiple energy signals to a (one or more) recurrent neural network.

[0330] When training the (one or more) recurrent neural network of the environment 5700, model information can be stored. In some cases, the model information can be stored locally at the sensor (e.g., stored on a data storage associated with the processor driving the ultrasonic transducer), although this need not always be the case. In some cases, the model information can be stored remotely (e.g., on a computing device separate from the sensor) or can be split, such as storing the model information for determining whether a trigger event has occurred locally at the sensor and storing the model information for determining the state of the sensor based on the trigger signal remotely. The model information can be any information that can be used to generate, and optionally interpret, an output from the input energy signal. For example, the model information can include information about the structure and weights found in any (one or more) recurrent neural network of the environment 5700.

[0331] During an example training session, a user can select a set of states to train into the model information. As depicted in Figure 57 the selected states include single click, double click, and hold. When prompted to do so, the user can engage in each action associated with each state, thereby generating energy signals as input data. Since the user is prompted to engage in a particular action, the environment 5700 can associate the detected energy signal as being associated with a particular state (e.g., single click or double click). As depicted in Figure 57 information associated with a single click is shown in green, double click in blue, hold in yellow, and error events in red. For example, an error event can be generated by prompting the user to touch around the sensor rather than directly above the sensor, making the algorithm less susceptible to such unwanted error inputs and thus more sensitive to local inputs closely located above the sensor. Training data can be collected from one or more users once or repeatedly until the (one or more) recurrent neural network is sufficiently trained.

[0332] In some cases, the training can be offline (e.g., performed on a set of test sensors and the best network parameters written for all sensors for that particular application), or the user can be prompted to perform a training session at system initialization (e.g., this can be similar to fingerprint login on a phone). The method can also be a combination of the two methods. For example, the method can include offline training and some optimization during user use. In other cases, data (e.g., energy data, status data, trigger data, etc.) can be shared on the cloud or through other suitable communication channels to strengthen the training data database for improving the network model, training, and optimization.

[0333] In some cases, recurrent neural networks can be particularly useful for time series data and can be more easily optimized for different material stacks and different environmental conditions. In some cases, an environment with multiple recurrent neural networks can allow different types of useful information to be output from sensors (e.g., from a processor driving an ultrasonic transducer). For example, the sensors can output energy signals, trigger signals, and status information from different points in the environment 5700, respectively. Thus, the same sensors can be mass-produced and quickly used in various different ways. While some customers may prefer to utilize the trigger signal, other customers may wish to utilize the status information. Thus, the same mass-produced sensors can meet the needs of different customers. Additionally, if multiple sensors are installed in a unit, the host / customer can also decide how to combine the information from the sensor network into bundled events when an action is triggered and / or even during training. Examples of this can be a slider bar or a mouse pad. Generally, in the case where multiple sensors are installed in a unit, the information from multiple sensors can be used to enhance the performance of the algorithm and improve its robustness. X. Applications

[0334] Figure 58 is a schematic diagram according to a particular aspect of the present disclosure, depicting an electronic device having an ultrasonic input device. The electronic device 5800 can include a housing 5802, a screen 5804, one or more front buttons 5806, a pair of ultrasonic input devices 5808, and a separate ultrasonic input device 5810. The electronic device 5800 can include a processor, a memory, and a network interface. In some embodiments, the ultrasonic input device can be coupled to the processor of the electronic device 5800.

[0335] In some embodiments, the pair of ultrasonic input devices 5808 may define an input touch area 5812 to detect user input. For example, a user may contact the input touch area 5812 to adjust the volume, brightness, etc. of the electronic device. In some embodiments, an array of ultrasonic input devices may be positioned below the screen or at other locations such as the side or back of the electronic device to detect touch input and replace or enhance the capacitive touch or force touch capabilities or mechanical buttons of the electronic device. A single ultrasonic input device 5810 may define an input touch area 5814 to detect user input. The input touch area 5814 may be configured to control device power, screen on / off, etc.

[0336] In some embodiments, ultrasonic input devices may be used to detect touch input at each of one or more front buttons 5806. The ultrasonic input devices may replace capacitive sensing used to detect touch input on a fingerprint sensor. The ultrasonic input devices provide a low-power solution to detect touch input on a fingerprint sensor. In some embodiments, one or more ultrasonic input devices may be positioned below a logo 5822 on the back 5820 of the housing 5802 to detect user input. They may also be placed below the side of the electronic device to replace the side mechanical buttons typically used, such as for power or volume.

[0337] Figure 59 FIG. is a schematic diagram of a steering wheel 5902 having an ultrasonic input device 5904 according to a particular aspect of the present disclosure. The ultrasonic input device 5904 may be used to form a touch input area on the steering wheel 5902 to detect touch input. The flexibility of the ultrasonic input device 5904 facilitates the detection of touch input through various materials (such as plastic, leather, wood, etc.) used to manufacture the steering wheel. A cross-section 5906 of the steering wheel 5902 shows the ultrasonic input device coupled to a surface 5908 to form a touch input area 5910. The touch input area may be combined with multiple touch input areas for applications such as cruise control, infotainment input control, cellular communication control, volume, and driver detection systems. For example, the ultrasonic input device 5904 may be used in a driver detection system to determine whether the driver's hand is in contact with the steering wheel.

[0338] Figure 60 FIG. is a schematic diagram of a keypad 6000 using an ultrasonic input device according to a particular aspect of the present disclosure. The shape and material of the touch area where the ultrasonic input device is located can be designed only limited by the creativity of the designer. For example, in Figure 60 a 12-key standard telephone keypad is shown. The keypad 6000 may include 12 ultrasonic input devices 6002 to form a touch area 6004 for each key. As another example, the keypad 6000 may include 23 or fewer ultrasonic input devices 6002.

[0339] Figure 61 Schematic diagram according to a particular aspect of the present disclosure, which depicts a robotic arm using an ultrasonic input device. The robotic arm 6100 may include a first finger 6102 and a second finger 6104. The ultrasonic input device may be implemented as a robotic finger input device. The first finger 6102 and the second finger 6104 may include a first ultrasonic input device 6106 and a second ultrasonic input device 6108, respectively. The first ultrasonic input device 6106 may form a contact area 6110 on the surface of the first finger 6102, and the second ultrasonic input device 6108 may form a second contact area 6112 on the second finger. The ultrasonic input device improves the detection ability of the robotic arm because they can be integrated into fingers made of any material. In addition, the ultrasonic input device can detect touch inputs without the need for incisions and / or different materials integrated into the fingers.

[0340] In some cases, the ultrasonic input device can identify the type of material being touched by analyzing the energy measurement signal. In some cases, the ultrasonic input device can identify the elasticity of the object being grasped. For example, an object with less elasticity will generally absorb less ultrasound than an object with greater elasticity, thus causing a generally higher energy measurement signal. In some cases, the determination of the elasticity of the object can be used to adjust the behavior of the robotic arm, such as adjusting the force with which the robotic arm grasps the object. In some cases, the ultrasonic input device is capable of detecting the texture or other mechanical properties of the object based on analyzing the energy measurement signal associated with the object. In some cases, the analysis of the energy measurement signal from the ultrasonic input device can be combined with other inputs, such as machine vision, to confirm or make a determination about the object with which the robotic arm will interact or is interacting.

[0341] Figure 62FIG. 0 is a schematic diagram according to a particular aspect of the present disclosure depicting a piece of furniture 6202 with an ultrasonic input device 6204. The ultrasonic input device 6204 can be coupled to the furniture 6202 in any suitable manner. A user touching the furniture 6202 at or near the location of the ultrasonic input device 6204 can be detected by the ultrasonic input device 6204 (e.g., via an ultrasonic touch sensor 6212). Upon detecting the touch, the ultrasonic input device 6204 can perform any pre-programmed function. For example, a communication module 6214 of the ultrasonic input device 6204 can send a signal (e.g., a wireless signal) to a control module 6206 spaced apart from the ultrasonic input device 6204 and / or the furniture 6202. The control module 6206 can control another device, such as a power switch 6208 coupled to a light bulb 6210. Thus, when pressing on a location on the furniture 6202 that is at or adjacent to the ultrasonic input device 6204, the light bulb 6210 can be turned on, off, or otherwise controlled. The device being controlled (e.g., the light bulb 6210) can be in the same environment as the ultrasonic input device 6204, although this need not always be the case. In some instances, the device being controlled can be in a neighboring environment or even in a remote environment.

[0342] In some embodiments, a piece of furniture or an IoT can be equipped with one or more of these ultrasonic input devices, which can operate individually or in the form of a sensor network that communicates with each other to perform multiple tasks. The sensors can also communicate with other sensors on other devices via the IoT device itself or via a general purpose programmable processor of the sensor to exchange information.

[0343] The ultrasonic input device 6204 according to a particular aspect of the present disclosure can operate at very low power, such as from an internal battery 6216. This battery-powered low-power operation can allow the ultrasonic input device 6204 to be used in locations that are otherwise difficult to access or inconvenient. For example, a light switch can be incorporated into a table or desk, or a TV remote control can be incorporated into an armrest of a chair.

[0344] In some cases, the ultrasonic input device 6204 can be located on a hidden surface 6218 so as to be hidden from view during normal operation. The hidden surface 6218 can be the underside of a table (e.g., the furniture 6202), the inside of a piece of furniture, the inside of a wall, or any other suitable location that is hidden from view. Thus, the hidden ultrasonic input device can only be actuated by those who know its location and will be hidden from view to others.

[0345] Figure 63A set of diagrams according to a particular aspect of the present disclosure depicts energy measurement signals of an ultrasonic input device demonstrating material detection. Characteristics of the energy measurement signals, such as shape, duration, slope, or other characteristics, can be used to determine the material interacting with the ultrasonic input device. For example, a bare human finger can cause a different energy measurement signal than a human finger wearing a plastic glove. The top diagram depicts an example of an energy measurement signal from a bare human finger contacting the ultrasonic input device, where when the finger is removed, the characteristics rapidly decline and relatively rapidly increase back to the baseline signal. However, the bottom diagram depicts an example of an energy measurement signal from a human finger wearing a plastic glove. When wearing a plastic glove, the energy measurement signal has different characteristics than when not wearing a plastic glove. For example, when wearing a plastic glove, after the finger is removed, the energy measurement signal discriminably increases back to the baseline signal more slowly. This relatively slow increase is due in part to the slight adhesion between the plastic glove and the substrate of the ultrasonic input device. While the human finger can be pulled away more quickly, the plastic glove around the finger will tend to remain on the substrate for a short duration, which is discriminable in the resulting energy measurement signal. This type of comparison can be used to determine the material initiating a touch event. XI. Additional Piezoelectric Array Designs

[0346] Figure 64 A schematic diagram of a piezoelectric resonator array 6400 according to a particular aspect of the present disclosure includes piezoelectric cantilevers 6402 that can be used in an ultrasonic input device. The piezoelectric resonator array 6400 can include a set of piezoelectric cantilevers 6402 on a substrate 6404. The piezoelectric resonator array 6400 can operate at a particular acoustic resonance when acoustically coupled to a material layer (e.g., Figure 1 material layer 102). When a touch event is occurring, the touch event can cause the piezoelectric resonator array 6400 to resonate differently. This change in the acoustic resonance caused by the touch event can be detected and used as a sensor signal in the ultrasonic input device, such as instead of a PMUT. Additionally, the piezoelectric cantilevers 6402 can be driven to bend and thus cause an emitted signal.

[0347] Figure 65 A schematic diagram of a piezoelectric resonator array 6500 according to a particular aspect of the present disclosure includes piezoelectric posts 6502 that can be used in an ultrasonic input device. The piezoelectric resonator array 6500 can include a set of piezoelectric posts 6502 on a substrate 6504. When acoustically coupled to a material layer (e.g., Figure 1When the piezoelectric resonator array 6500 is in contact with the material layer 102), it can operate at a specific acoustic resonance. When a touch event is occurring, the touch event can cause the piezoelectric resonator array 6500 to resonate differently. This change in the acoustic resonance caused by the touch event can be detected and used as a sensor signal in an ultrasonic input device, such as instead of a PMUT. Additionally, the piezoelectric pillars 6502 can be driven to bend and thus cause an emitted signal. The piezoelectric pillars 6502 can be arranged in any suitable pattern, such as a hexagonal grid.

[0348] Multiple aspects of the embodiments can be implemented in a modular or integrated manner using hardware circuits (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or using computer software with a generally programmable processor in the form of control logic. As used herein, a processor can include a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked, as well as dedicated hardware. Based on the disclosure and teachings provided herein, those of ordinary skill in the art will know and understand other ways and / or methods of implementing the embodiments of the present invention using hardware as well as combinations of hardware and software.

[0349] Any software component or function described in this application can be implemented as software code, which can be executed by a processor using any suitable computer language (e.g., Java, C, C++, C#, Objective-C, Swift, or a scripting language, such as Perl or Python using, for example, conventional or object-oriented techniques). The software code can be stored on a computer-readable medium as a series of instructions or commands for storage and / or transmission. Suitable non-transitory computer-readable media can include random access memory (RAM), read-only memory (ROM), magnetic media (such as a hard disk drive or a floppy disk), or optical media (such as a compact disc (CD) or a digital versatile disc (DVD)), flash memory, etc. The computer-readable medium can be any combination of such storage or transmission devices.

[0350] Such a program can also be encoded and transmitted using a carrier signal suitable for transmission over wired, optical, and / or wireless networks (including the Internet) compliant with various protocols. Thus, a computer-readable medium can be established using a data signal encoded with such a program. The computer-readable medium encoded with the program code can be packaged with a compatible device or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable medium can exist on or within a single computer product (e.g., a hard disk drive, a CD, or an entire computer system) and can exist on or within different computer products in a system or network. A computer system can include a monitor, a printer, or other suitable display for providing any of the results mentioned herein to a user.

[0351] Any method described herein may be performed, in whole or in part, using a computer system that includes one or more processors configurable to perform a plurality of steps. Accordingly, embodiments may relate to a computer system configured to perform the steps of any method described herein, potentially using different components to perform corresponding steps or corresponding groups of steps. The steps of the methods herein, although presented as numbered steps, may be performed simultaneously or at different times or in a different order. Additionally, multiple portions of these steps may be used in combination with multiple portions of other steps of other methods. Moreover, all or multiple portions of the steps may be optional. Further, any step of any method may be performed using modules, units, circuits, or other means for performing those steps.

[0352] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the present invention. However, other embodiments of the present invention may relate to particular embodiments related to each individual aspect or to a particular combination of these individual aspects.

[0353] The above description of example embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the exact form disclosed, and many modifications and variations are possible in light of the above teachings.

[0354] Unless specifically indicated to the contrary, the recitation of "a," "an," or "the" means "one or more." Unless specifically indicated to the contrary, the use of "or" means "inclusive or" rather than "exclusive or." The recitation of a "first" component does not necessarily require the provision of a second component. Additionally, unless explicitly stated, the recitation of a "first" or "second" component does not limit the recited component to a particular location. The term "based on" is intended to mean "at least partially based on."

[0355] All patents, patent applications, publications, and descriptions mentioned herein are hereby incorporated by reference in their entirety for all purposes. It is not admitted that they are prior art.

Claims

1. A system, comprising: an ultrasonic input device coupled to a material layer having an outer surface located at a position of the material layer opposite to the ultrasonic input device, wherein the ultrasonic input device is coupled to the material layer to transmit an emitted signal through the material layer toward the outer surface and receive a set of reflected ultrasonic signals associated with the emitted signal, wherein the set of reflected ultrasonic signals includes at least one reflected ultrasonic signal, and wherein the set of reflected ultrasonic signals is associated with a touch event between an object and the outer surface of the material layer; and one or more data processors configured to: determine an energy signal, the energy signal including energy measurements over time, each energy measurement corresponding to a sum value obtained from a portion of the set of reflected ultrasonic signals; extract feature information associated with the energy signal; determine an inference associated with the object based on the extracted feature information; and generate an output signal associated with the determined inference.

2. The system according to claim 1, wherein, extracting the feature information includes identifying a pattern in the energy signal associated with a sink in the energy measurements, the sink in the energy measurements being associated with the touch event.

3. The system according to claim 2, wherein, identifying the pattern includes identifying one or more of: the depth of the sink, the duration of the sink, the presence of subsequent sinks after the sink, the delay between the sink and another sink, and the rate of change of the energy signal at the edge of the sink.

4. The system according to claim 2, wherein, identifying the pattern includes identifying a change in the energy signal attributable to a temperature drift in the material layer, and wherein determining the inference includes estimating a relative temperature of the object based on the identified change in the energy signal attributable to the temperature drift in the material layer.

5. The system according to claim 2, wherein, determining the inference includes comparing the pattern with stored data, wherein the stored data is associated with a previous touch event of the outer surface.

6. The system according to claim 1, wherein, determining the inference includes using the feature information to determine that the touch event is associated with one selected from the group consisting of a bare human finger, a wet human finger, a dry human finger, and a gloved human finger.

7. The system according to claim 1, wherein, the feature information includes the magnitude of the energy signal and / or the change in the energy signal, and wherein determining the inference includes: comparing at least one selected from (a) the magnitude and (b) the change with a corresponding threshold to determine whether the touch event is associated with a human finger or a water droplet.

8. The system according to claim 1, wherein, determining the inference includes using the feature information to determine one or more of a touch manner of the touch event, a touch intensity associated with the touch event, and a physical characteristic of the object.

9. The system according to claim 8, wherein, determining the inference further includes: identifying that the object is associated with one of a plurality of users based on associating the touch event with one or more of a touch manner of the touch event, a touch intensity associated with the touch event, or a physical characteristic of the object.

10. The system according to claim 8, wherein, the physical characteristic of the object is a measurement associated with a portion of a fingerprint that contacts the outer surface.

11. The system according to claim 1, wherein, the one or more data processors are further configured to: determine an additional signal associated with an additional sensor associated with the ultrasonic input device, wherein determining the inference further includes using the additional signal.

12. The system according to claim 1, wherein, the energy signal represents the energy of the set of reflected ultrasonic signals that occur within an energy measurement window, and wherein determining the energy signal includes determining the energy signal by integrating the reflected ultrasonic signals in the set of reflected ultrasonic signals that occur within the energy measurement window.

13. The system according to claim 12, wherein, at least one of the one or more data processors includes a summing circuit or an integrating circuit, wherein the summing circuit or the integrating circuit is configured to: generate the energy signal based on the set of reflected ultrasonic signals, the energy signal including the energy measurement over time, wherein each energy measurement is a sum value of the set of reflected ultrasonic signals within the energy measurement window.

14. A computer-implemented method, comprising: transmitting a transmitted signal using an ultrasonic input device coupled to a material layer having an outer surface located at a position of the material layer opposite to the ultrasonic input device, wherein the transmitted signal travels through the material layer towards the outer surface; receiving a set of reflected ultrasonic signals associated with the transmitted signal, wherein the set of reflected ultrasonic signals includes at least one reflected ultrasonic signal, and wherein the set of reflected ultrasonic signals is associated with a touch event between an object and the outer surface of the material layer; determining an energy signal, the energy signal including an energy measurement over time, each energy measurement corresponding to a sum value obtained from a portion of the set of reflected ultrasonic signals; extracting feature information associated with the energy signal; determining an inference associated with the object based on the extracted feature information; and generating an output signal associated with the determined inference.

15. The method according to claim 14, wherein, extracting the feature information includes identifying a pattern in the energy signal associated with a dip in the energy measurement, the dip in the energy measurement being associated with the touch event.

16. The method according to claim 15, wherein, Identifying the pattern includes identifying one or more of the following: the depth of the sink, the duration of the sink, the presence of subsequent sinks after the sink, the delay between the sink and another sink, and the rate of change of the energy signal at the edge of the sink.

17. The method according to claim 15, wherein, identifying the pattern includes identifying a change in the energy signal attributable to temperature drift in the material layer, and wherein determining the inference includes estimating a relative temperature of the object based on the identified change in the energy signal attributable to temperature drift in the material layer.

18. The method according to claim 15, wherein, determining the inference includes comparing the pattern with stored data, wherein the stored data is associated with a previous touch event on the outer surface.

19. The method according to claim 14, wherein, determining the inference includes using the feature information to determine that the touch event is associated with one selected from the group consisting of a bare human finger, a wet human finger, a dry human finger, and a gloved human finger.

20. The method according to claim 14, wherein, determining the inference includes using the feature information to determine one or more of a touch manner of the touch event, a touch intensity associated with the touch event, or a physical characteristic of the object.

21. The method according to claim 20, wherein, determining the inference further includes: identifying that the object is associated with one of a plurality of users based on associating the touch event with one or more of a touch manner of the touch event, a touch intensity associated with the touch event, or a physical characteristic of the object.

22. The method according to claim 14, further comprising: determining an additional signal associated with an additional sensor associated with the ultrasonic input device, wherein determining the inference further includes using the additional signal.

23. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product including instructions configured to enable a data processing device to perform operations, the operations including: transmitting an emitted signal using an ultrasonic input device coupled to a material layer having an outer surface located at a position of the material layer opposite to the ultrasonic input device, wherein the emitted signal travels through the material layer towards the outer surface; receiving a set of reflected ultrasonic signals associated with the emitted signal, wherein the set of reflected ultrasonic signals includes at least one reflected ultrasonic signal, and wherein the set of reflected ultrasonic signals is associated with a touch event between an object and the outer surface of the material layer; determining an energy signal, the energy signal including energy measurements over time, each energy measurement corresponding to a sum value obtained from a portion of the set of reflected ultrasonic signals; extracting feature information associated with the energy signal; determining an inference associated with the object based on the extracted feature information; and Generate an output signal associated with the determined inference.

24. The computer program product according to claim 23, wherein, extracting the feature information includes identifying a pattern in an energy signal associated with a sink in the energy measurement, the sink in the energy measurement being associated with the touch event.

25. The computer program product according to claim 24, wherein, identifying the pattern includes identifying one or more of: the depth of the sink, the duration of the sink, the presence of a subsequent sink after the sink, the delay between the sink and another sink, and the rate of change of the energy signal at the edge of the sink.

26. The computer program product according to claim 24, wherein, identifying the pattern includes identifying a change in the energy signal attributable to a temperature drift in the material layer, and wherein determining the inference includes estimating a relative temperature of the object based on the identified change in the energy signal attributable to the temperature drift in the material layer.

27. The computer program product according to claim 24, wherein, determining the inference includes comparing the pattern with stored data, wherein the stored data is associated with a previous touch event on the outer surface.

28. The computer program product according to claim 23, wherein, determining the inference includes using the feature information to determine that the touch event is associated with one selected from the group consisting of a bare human finger, a wet human finger, a dry human finger, and a gloved human finger.

29. The computer program product according to claim 24, wherein, determining the inference includes using the feature information to determine one or more of a touch manner of the touch event, a touch intensity associated with the touch event, or a physical characteristic of the object.

30. The computer program product according to claim 29, wherein, determining the inference further includes: identifying that the object is associated with one of a plurality of users based on associating the touch event with one or more of a touch manner of the touch event, a touch intensity associated with the touch event, or a physical characteristic of the object.

Images