Ultrasonic touch and force input detection

Abstract

A system includes an ultrasonic input device coupled to a material layer having an outer surface and one or more data processors. The ultrasonic input device can transmit a signal through the material layer toward the outer surface and receive a set of reflected ultrasonic signals associated with a touch event between an object and the outer surface. The system can determine an energy signal associated with the set of reflected ultrasonic signals and then extract feature information associated with the energy signal. The system can then determine an inference associated with the object based on the extracted feature information and generate an output signal.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] No. 62 / 751,053 filed on October 26, 2018, entitled “ULTRASONIC TOUCHFEATURE EXTRACTION”; and U.S. Provisional Application Serial No. 62 / 784,615 filed on December 24, 2018, entitled “ULTRASONIC TOUCH SENSOR AND SYSTEM”. SYSTEM”, U.S. Provisional Application Serial No. 62 / 810,786, filed on February 26, 2019, entitled “ULTRASONIC TOUCH DETECTION AND DECISION”, U.S. Patent Application Serial No. 16 / 396,597, filed on April 26, 2019, entitled “ULTRA SONIC TOUCH AND FORCE INPUT DETECTION”, and U.S. Patent Application Serial No. 16 / 417,184, filed on May 20, 2019, entitled “ULTRASONIC TOUCH FEATURE EXTRACTION”, the entire contents of which are incorporated herein by reference. Background Art

[0003] Capacitive, resistive, and inductive sensing are used in industrial, automotive, medical, and consumer applications to detect touch inputs. The use of capacitive technology to detect touch inputs has grown rapidly in human interface devices (HIDs) such as trackpads and touch screens. Consumer and industrial applications are beginning to adopt touch buttons and sliders using capacitive technology in devices such as mobile phones, TV controls, car dashboards, remote controls, or industrial controls. Capacitive sensing has proven to be more attractive than mechanical switches and rotary encoders in terms of appearance and reliability.

[0004] However, due to the challenges of touch input layout and system stacking, the use of capacitive, resistive or inductive sensing limits creative industrial design. The conflicting priorities between design and robustness further complicate the design. It should also be noted that current input touch sensing methods cannot be implemented on metal surfaces. In addition, current sensing technologies have inherent properties that limit waterproof applications. Pressure sensing technology using strain gauges has emerged as an alternative sensing technology for metal surface touch input. However, the measurement of deflection and strain is generally unreliable, especially in metal. Such sensors are very susceptible to unwanted interference that causes surface deflection, and their sensitivity and performance are very dependent on the overall boundary conditions of the surface to which they are attached. In addition, the surface to which the sensor is attached must be sufficiently conformal so that it deflects sufficiently when a person touches it so that the sensor can detect it. An additional sensing layer (e.g., capacitive) is required to detect the xy position of the input touch detected using a strain gauge. The increased complexity of the touch input interface material, the implication of complex interfaces in industrial design, waterproofing and cost have become key challenges that limit the use of touch input in any environment and in any material. There exists a need for improved systems and methods for detecting touch input to a human machine interface (HMI).

[0005] Embodiments of the invention address these and other problems individually and collectively. Summary of the invention

[0006] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch and force input detection.

[0007] According to some embodiments, a method is provided. A transducer coupled to a first surface of a material layer may transmit an ultrasonic signal directed toward a second surface, the material layer having a distance between the first surface and the second surface. The transducer may then detect the reflected ultrasonic signal and then determine the amplitude of the reflected ultrasonic signal. The transducer may then determine that the amplitude exceeds a threshold associated with a portion of the ultrasonic signal that penetrates the second surface. In the event that the amplitude exceeds the threshold, the transducer may generate a signal indicating a touch input on the second surface.

[0008] According to other embodiments, a method is provided. A transducer coupled to a first surface of a material layer may transmit an ultrasonic signal directed toward a second surface, the material layer having a distance between the first surface and the second surface. The transducer may then detect the reflected ultrasonic signal. The method then includes determining an energy value associated with the reflected ultrasonic signal. The method may also include determining that the energy value exceeds a threshold associated with a portion of the ultrasonic signal penetrating the second surface. In the event that the energy value exceeds the threshold, the method may include generating a signal indicating a touch input on the second surface.

[0009] Some embodiments of the present disclosure relate to systems, methods and devices related to ultrasonic touch feature extraction.The system may include an ultrasonic input device and one or more data processors.

[0010] The ultrasonic input device may be coupled to a material layer that may have an outer surface. The outer surface may be located at a position of the material layer opposite to the ultrasonic input device. The ultrasonic input device may be coupled to the material layer to transmit a transmitted signal through the material layer toward the outer surface and to receive a set of reflected ultrasonic signals associated with the transmitted signal. The set of reflected ultrasonic signals may include at least one reflected ultrasonic signal and may be associated with a touch event between an object and the outer surface of the material layer. The touch event may include, for example, an individual touching the outer surface with their finger or other object (e.g., a stylus, etc.).

[0011] The one or more data processors may be configured to determine an energy signal associated with the set of reflected ultrasound signals and extract characteristic information associated with the energy signal. The one or more data processors may also be configured to determine an inference associated with the object based on the extracted characteristic information, and then generate an output signal associated with the determined inference.

[0012] According to some embodiments, a computer-implemented method is provided. An ultrasonic input device coupled to a material layer having an outer surface may be used to transmit a transmitted signal. A set of reflected ultrasonic signals may be received. The set of reflected ultrasonic signals may be associated with the transmitted signal and may include at least one reflected ultrasonic signal. The set of reflected ultrasonic signals may be associated with a touch event between an object and the outer surface of the material layer. An energy signal associated with the set of reflected ultrasonic signals may be determined. Subsequently, feature information associated with the energy signal may be extracted. Subsequently, an inference may be determined. The inference may be associated with an object based on the extracted feature information. Subsequently, an output signal associated with the determined inference may be generated.

[0013] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch sensors and systems.The touch sensor may include an ultrasonic sensor layer and an integrated circuit layer.

[0014] The ultrasonic sensor layer may include an array of ultrasonic transducers. The array of ultrasonic transducers may include one or more ultrasonic transducers. The integrated circuit layer may be coupled to the ultrasonic sensor layer. The integrated circuit layer may include circuitry configured to drive the array of ultrasonic transducers to generate ultrasonic signals. The integrated circuit layer may also include circuitry configured to receive reflected ultrasonic signals using the array of ultrasonic transducers and to generate energy signals associated with the received reflected ultrasonic signals.

[0015] According to some embodiments, a method that can be performed by a touch sensor or other suitable device is provided. The method includes generating a drive signal in an integrated circuit coupled to a transmission ultrasonic transducer, the transmission ultrasonic transducer being located in an array of ultrasonic transducers coupled to the integrated circuit. The transmission ultrasonic transducer may be a piezoelectric micromechanical ultrasonic transducer. Subsequently, a transmitted ultrasonic signal may be generated by the transmission ultrasonic transducer in response to the drive signal. Generating the transmitted ultrasonic signal may include transmitting the transmitted ultrasonic signal through the material layer in a longitudinal direction orthogonal to the outer surface of the material layer, or in some embodiments within 20% of the normal to the outer surface of the material layer. Subsequently, a set of reflected signals may be received at a receiving ultrasonic transducer in the array of ultrasonic transducers. The set of reflected signals may include one or more ultrasonic signals associated with the transmitted ultrasonic signal. The receiving ultrasonic transducer may be a piezoelectric micromechanical ultrasonic transducer. Subsequently, an energy signal may be measured. The energy signal may be associated with the received set of reflected signals. The method may also include determining that a touch event has occurred at the outer surface of the material layer based on the measured energy signal.

[0016] Some embodiments of the present disclosure relate to systems, methods, and devices related to ultrasonic touch detection and decision making.

[0017] According to some embodiments, a method that can be performed by a touch sensor device or other suitable device is provided. The method includes receiving energy data associated with an ultrasonic input device connected to a material layer. The energy data may include a current energy value and a past energy value associated with a reflected ultrasonic signal received at the ultrasonic input device in response to the ultrasonic input device transmitting a signal through the material layer toward the outer surface of the material layer. Subsequently, the energy data can be compared with threshold data to generate a current trigger value for trigger data. In the event that the current energy value exceeds a current threshold of the threshold data, the trigger data can indicate the occurrence of a touch event. Subsequently, the threshold data can be updated based on the energy data, the trigger data, and the threshold data. Updating the threshold data may include generating subsequent thresholds.

[0018] According to some embodiments, another method is provided that can be performed by a touch sensor device or other suitable device. The method includes receiving energy data associated with an ultrasonic input device coupled to a material layer. The energy data may include current energy values ​​and past energy values ​​associated with reflected ultrasonic signals received at the ultrasonic input device in response to the ultrasonic input device transmitting signals through the material layer toward an outer surface of the material layer. The energy data may then be provided to a recurrent neural network to generate output data indicating that a touch event has occurred at the outer surface of the material layer.

[0019] These and other embodiments of the present invention are described in detail below.For example, other embodiments relate to systems, devices, and computer-readable media associated with the methods described herein.

[0020] A better understanding of the nature and advantages of embodiments of the present invention may be obtained by reference to the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic diagram illustrating the effect of touch force on reflected ultrasound signals in an ultrasound input system according to certain aspects of the present disclosure.

[0022] Figure 2 is a schematic diagram according to certain aspects of the present disclosure, which depicts an ultrasound input system in a non-contact state and a contact state.

[0023] Figure 3 is a schematic diagram depicting an ultrasound input device according to certain aspects of the present disclosure.

[0024] Figure 4 is a cross-sectional view of two piezoelectric micromachined ultrasound transducers bonded to a CMOS wafer in accordance with certain aspects of the present disclosure.

[0025] Figure 5 is a set of schematic diagrams depicting ultrasound input devices coupled to various surfaces in accordance with certain aspects of the present disclosure.

[0026] Figure 6 is a schematic side view of an ultrasound input system having a common plate assembly according to certain aspects of the present disclosure.

[0027] Figure 7 is a schematic diagram depicting an example ultrasound input system in accordance with certain aspects of the present disclosure.

[0028] Figure 8 is a schematic side view diagram depicting an integrated ultrasound input device with an ultrasound sensor and an ASIC according to certain aspects of the present disclosure.

[0029] Fig. 9 is a combined set of schematic diagrams depicting two processing routes for producing an integrated ultrasound input device wafer in accordance with certain aspects of the present disclosure.

[0030] Fig.10 is a set of schematic diagrams depicting a single integrated ultrasound input device cut from a wafer, mounted on a PCB, and stacked according to certain aspects of the present disclosure.

[0031] Fig.11 is a schematic cross-sectional view of a consumer electronic product including an integrated ultrasound input device according to certain aspects of the present disclosure.

[0032] Fig.12 is a set of schematic cross-sectional views comparing a non-integrated ultrasound input device to an integrated ultrasound input device according to certain aspects of the present disclosure.

[0033] Fig.13A is a top view of a sensor array of an ultrasound input device according to certain aspects of the present disclosure.

[0034] Fig. 13B is a top view of an alternative sensor array of an ultrasound input device according to certain aspects of the present disclosure.

[0035] Fig.14A According to certain aspects of the present disclosure Fig.13A Schematic diagram of a sensor array, which depicts the function of the individual transducers of the array. Fig. 14B is a schematic diagram of a sensor array depicting eight example configurations of individual transducers in a corner region of the sensor array in accordance with certain aspects of the present disclosure. Fig. 14C is a schematic diagram of a sensor array depicting eight example configurations of individual transducers of the sensor array in accordance with certain aspects of the present disclosure. Fig.14D is a schematic diagram of a sensor array depicting eight example configurations of individual transducers of the sensor array in accordance with certain aspects of the present disclosure. Fig.14E is a schematic diagram of a sensor array depicting two example configurations of individual transducers of the sensor array in accordance with certain aspects of the present disclosure. Fig.14F is a schematic diagram of a sensor array according to certain aspects of the present disclosure, depicting example configurations of individual transducers of sensor arrays of various sizes. Figure 14G is a schematic diagram of a sensor array according to certain aspects of the present disclosure, depicting twelve example configurations of individual transducers of a sensor array having dimensions of 8x8 transducers.

[0036] Fig.15 is a set of graphs depicting energy measurements from transducers of a single sensor array operating at different frequencies in accordance with certain aspects of the present disclosure.

[0037] Fig.16 is a graph depicting the temperature behavior of an ultrasonic transducer with respect to operating frequency in accordance with certain aspects of the present disclosure.

[0038] Fig.17 is a graph depicting frequency response for a stack makeup according to certain aspects of the present disclosure.

[0039] Fig.18is a schematic diagram illustrating a circuit for receiving and transmitting signals via an ultrasound transducer, wherein the circuit is in a transmitting state, according to certain aspects of the present disclosure.

[0040] Fig.19 is a schematic diagram illustrating a circuit for receiving and transmitting signals through an ultrasound transducer, wherein the circuit is in a receiving state, according to certain aspects of the present disclosure.

[0041] Fig. 20 is a schematic diagram depicting an isolation circuit for receiving and transmitting signals through an ultrasound transducer, in accordance with certain aspects of the present disclosure.

[0042] Fig.21 is a set of schematic side views illustrating beamforming achieved using an ultrasound transducer according to certain aspects of the present disclosure.

[0043] Fig. 22 is a set of graphs depicting modes of operation of a micromachined ultrasonic transducer according to certain aspects of the present disclosure compared to a standard bulk transducer depicted as average displacement for different frequencies.

[0044] Fig.23 is a set of schematic side views depicting lateral signal rejection of a micromachined ultrasound transducer according to certain aspects of the present disclosure compared to a standard bulk transducer.

[0045] Fig.24 is a set of schematic side views depicting lateral signal rejection of a micromachined ultrasonic transducer according to certain aspects of the present disclosure.

[0046] Fig.25 is a schematic flow chart for digitally processing ultrasound signals transmitted and received by an ultrasound input device according to certain aspects of the present disclosure.

[0047] Fig.26 is a schematic flow chart for processing ultrasound signals transmitted and received by an ultrasound input device using energy integration according to certain aspects of the present disclosure.

[0048] Fig. 27 is a schematic example flow chart for processing ultrasound signals transmitted and received by an ultrasound input device using energy integration according to certain aspects of the present disclosure.

[0049] Fig.28 is a schematic flow chart for processing ultrasound signals transmitted and received by an ultrasound input device using energy integration by absolute value accumulation according to certain aspects of the present disclosure.

[0050] Fig.29is a schematic flow chart of processing ultrasound signals transmitted and received by an ultrasound input device using energy integration through self-mixing and integration according to certain aspects of the present disclosure.

[0051] Fig.30 is a schematic circuit diagram depicting an analog integrator with a negative bias current circuit according to certain aspects of the present disclosure.

[0052] Fig.31 is a schematic flow chart for processing an ultrasound signal according to certain aspects of the present disclosure, which illustrates the impact of reflected ultrasound signal flight time variation on the reduction of touch input detection within an energy measurement window.

[0053] Fig.32 is a schematic diagram of a simplified process for processing an ultrasound signal according to certain aspects of the present disclosure, which illustrates the effect of reflected ultrasound signal flight time variation on the enhancement of touch input detection outside the energy measurement window.

[0054] Fig.33 is a schematic diagram of a process for processing an ultrasound signal according to certain aspects of the present disclosure, which depicts an ultrasound signal with minimal impact of reflected ultrasound signal flight time variation on touch input detection outside an energy measurement window when using window shaping.

[0055] Fig.34 is a schematic circuit diagram depicting a window shaping circuit according to certain aspects of the present disclosure.

[0056] Fig.35 is a schematic diagram illustrating a process for processing an ultrasonic signal to detect a touch input using the amplitude of a reflected ultrasonic signal according to certain aspects of the present disclosure.

[0057] Fig.36 is a graph depicting a simplified example energy signal according to certain aspects of the present disclosure.

[0058] Fig.37 is a diagram depicting reflected ultrasonic signal measurements using an ultrasonic input device and showing techniques for improving touch input detection in accordance with certain aspects of the present disclosure.

[0059] Fig.38 is a diagram depicting reflected ultrasonic signal measurements using an ultrasonic input device and showing additional techniques for improving touch input detection in accordance with certain aspects of the present disclosure.

[0060] Fig.39 is a set of graphs depicting the temperature dependence of reflected ultrasound signals in accordance with certain aspects of the present disclosure.

[0061] Fig.40 is a set of graphs depicting time-of-flight temperature dependencies for two frequency methods of detecting touch input in accordance with certain aspects of the present disclosure.

[0062] Fig.41 is a graph depicting reflected ultrasonic signal measurements across several frequencies using an ultrasonic input device and showing techniques for improving touch input detection in accordance with certain aspects of the present disclosure.

[0063] Fig.42 is a schematic plan view depicting a dual-frequency PMUT having a concentric circle design according to certain aspects of the present disclosure.

[0064] Fig.43 is a schematic plan view depicting a multi-frequency ultrasound input device having a square design according to certain aspects of the present disclosure.

[0065] Fig.44 is a set of three graphs depicting example signals received by an ultrasound input system attributable to three different users in accordance with certain aspects of the present disclosure.

[0066] Fig.45 is a set of graphs depicting energy measurement signals associated with a human finger, a water droplet, and placing the device on a table (eg, placing an object on the sensor).

[0067] Fig.46 is a combined schematic diagram and set of graphs depicting how temperature can be used to further identify whether a human finger is initiating a touch event.

[0068] Fig.47 is a combined schematic diagram and graph depicting finger touches and associated temperature information in accordance with certain aspects of the present disclosure.

[0069] Fig.48 is a schematic combined side view and signal diagram depicting ridges and valleys of a fingerprint initiating a touch event on an ultrasonic input system in accordance with certain aspects of the present disclosure.

[0070] Fig.49 is a schematic diagram depicting example signals received by an ultrasonic input system attributable to the same user initiating touch events with gloves and without gloves, in accordance with certain aspects of the present disclosure.

[0071] Fig.50 is a flow chart depicting a process for extracting features from a signal input to an ultrasound system, according to certain aspects of the present disclosure.

[0072] Fig.51is a diagram depicting a machine learning decision algorithm for improving touch detection according to certain aspects of the present disclosure.

[0073] Fig.52 is a flow chart depicting a process for detecting touch events according to certain aspects of the present disclosure.

[0074] Fig.53 is a schematic diagram depicting an adaptive threshold scheme for identifying touch events according to certain aspects of the present disclosure.

[0075] Fig.54 is an example graph depicting energy signals and adaptive thresholds associated with identifying touch events according to certain aspects of the present disclosure.

[0076] Fig.55 is a schematic diagram depicting a general recurrent neural network according to certain aspects of the present disclosure.

[0077] Fig.56 is a schematic diagram depicting an example recurrent neural network for identifying triggering events in accordance with certain aspects of the present disclosure.

[0078] Fig.57 is a schematic diagram depicting an example environment for touch detection and state classification using a set of recurrent neural networks according to certain aspects of the present disclosure.

[0079] Fig.58 is a schematic diagram depicting an electronic device having an ultrasound input device according to certain aspects of the present disclosure.

[0080] Fig.59 is a schematic diagram depicting an automotive assembly having an ultrasonic input device according to certain aspects of the present disclosure.

[0081] Fig.60 is a schematic diagram depicting a keypad using an ultrasonic input device in accordance with certain aspects of the present disclosure.

[0082] Fig.61 is a schematic diagram depicting a robotic arm using an ultrasonic input device according to certain aspects of the present disclosure.

[0083] Fig.62 is a schematic diagram depicting a piece of furniture using an ultrasonic input device according to certain aspects of the present disclosure.

[0084] Fig.63 is a set of graphs depicting energy measurement signals of an ultrasonic input device demonstrating material detection in accordance with certain aspects of the present disclosure.

[0085] Fig.64is a schematic diagram of a piezoelectric resonator array including piezoelectric cantilevers that can be used in an ultrasound input device according to certain aspects of the present disclosure.

[0086] Fig.65 is a schematic diagram of a piezoelectric resonator array including piezoelectric posts that may be used in an ultrasound input device according to certain aspects of the present disclosure. DETAILED DESCRIPTION

[0087] A touch input solution for improving the detection of touch input in an HMI is provided. An ultrasonic input device can detect the presence of an object on any surface using a sensor located on the reverse side of the surface material. The ultrasonic input device enables creative design without damaging the product skin or design materials (e.g., material stacking). Such an ultrasonic input device can be implemented in a variety of devices, such as input touch buttons, sliders, scroll wheels, etc. The ultrasonic input device can be deployed under surfaces including various materials to simplify industrial design and appearance. In addition, a grid of ultrasonic input device buttons can be implemented to create a keypad, mouse pad, or touch input at any location on any surface. The ultrasonic input device allows the HMI to be deployed for touch input on surfaces including wood, leather, glass, plastic, metal (e.g., aluminum or steel), ceramics, plastic, a combination of one or more materials, and the like.

[0088] In some cases, the ultrasonic input device may include an ultrasonic sensor coupled to a processor such as an application specific integrated circuit (ASIC) to provide a fully integrated system on a chip (SOC) that can receive touch input through ultrasonic detection. In some cases, the ultrasonic sensor and the processor (e.g., ASIC) can be produced in a single wafer. A fully integrated SOC can provide many benefits, such as low cost due to mass production via wafer-level processes, low profile form factors, improved signal-to-noise ratio, and improved sensor array design freedom.

[0089] In some cases, the ultrasonic input device may include an ultrasonic sensor that includes a micromachined ultrasonic transducer (MUT), such as a piezoelectric micromachined ultrasonic transducer (pMUT) or a capacitive micromachined ultrasonic transducer (cMUT). By using a MUT in an ultrasonic input device as disclosed herein, optionally as part of a fully integrated SOC, many benefits can be achieved. Since the MUT (due to its unique and predictable bending mode shape) produces a signal that propagates more orthogonally to the transducer surface (longitudinal waves orthogonal to the surface) than other types of waves that propagate laterally, the use of the MUT can provide an improved energy transmission area. Since the predictable bending mode shape of the MUT is separated far from other modes (e.g., bulk modes) over a large frequency range, it is also less likely to generate or receive other types of acoustic waves, such as shear waves or surface waves that may travel laterally or orthogonally to the sensor surface. Therefore, the MUT can achieve a more significant transmission and sensing area on the surface material, such as an area directly perpendicular to the MUT through the surface material. In addition, the use of the MUT can reduce or minimize the amount of power required to operate the ultrasonic input device. For example, the MUT can be used with low parasitics, low drive voltage, and with a small device capacitance that is approximately three orders of magnitude lower than the capacitance of a conventional piezoelectric ceramic ultrasonic transducer.

[0090] The ultrasonic input device can detect patterns associated with touch inputs and distinguish between different types of touch inputs. Different types of touch inputs can vary between finger presses, palm presses, taps, touch and hold, or other such inputs. Each of the various types of touch inputs may have a recognizable and / or distinguishable pattern. In some cases, feedback from multiple sensors (such as multiple sensors arranged in an array) can be used to determine the type of touch input initiated. For example, a palm resting on an array of ultrasonic input devices can register a recognizable pattern across multiple ultrasonic input devices, and thus a processor connected to multiple ultrasonic input devices can determine that the touch input is a palm rest and take appropriate action (e.g., rejecting a palm rest as a touch input or initiating an action based on a palm rest).

[0091] The ultrasonic input device can detect patterns associated with touch input and distinguish between different users who initiate touch input. It has been found that different users of the ultrasonic input device will often generate recognizable and distinguishable signals when initiating touch input. For example, the signal measured from the touch input can change based on the user's finger, such as the moisture content of the finger, the size of the ridges and valleys of the fingerprint, and other mechanical properties of the individual finger. In addition, some users can initiate touch input in a repeatable manner, which can be used to identify the user. For example, the first user can usually tap the input device quickly, while the second user can usually place their finger on the input device and then press. As another example, different users may generate different touch pressures, which can also be detected by monitoring the amount of change in the ultrasonic signal. These factors such as the speed of the touch input and the way the touch input is used can be used to facilitate the identification of users.

[0092] In some cases, it can be determined whether a touch event has occurred by comparing the energy signal from the ultrasonic transducer to a threshold. In some cases, in order to improve the detection of touch events and the rejection of false positives, the threshold can be updated dynamically or automatically. The adaptive threshold can be updated based on any combination of the input energy signal and historical threshold data and trigger data (e.g., information about whether a touch event has occurred). These inputs can help update the threshold update function, which can be used to filter the energy signal value into a new threshold. Therefore, specific changes in the energy signal that do not generally indicate a touch event (e.g., slow changes) can be tracked by the adaptive threshold, while changes that indicate a touch event (e.g., fast changes) may not be tracked by the adaptive threshold, which allows the energy signal to drop below the threshold and thus indicate a touch event.

[0093] In some cases, it may be possible to determine whether a touch event has occurred by passing the energy signal into a recurrent neural network that has been trained based on training data. The recurrent neural network may convert the input energy signal into an output that indicates whether a touch event has occurred.

[0094] In some cases, the state of the sensor (e.g., classification of the type of touch event, such as press, tap, double tap, hold, or other such type) can be determined by analyzing the trigger data. In some cases, the trigger data can be passed as input to a recurrent neural network that has been trained based on training data for a particular state. The recurrent neural network can convert the input trigger data into an output indicating the state of the sensor.

[0095] In some cases, ultrasonic input devices can provide improvements in aesthetic features and reliability of touch input detection on capacitive and mechanical devices. By defining a button area on the touch surface, a button can be implemented on the surface. The ultrasonic input device can be embedded / placed behind the surface, and thus limit environmental exposure (including dust and moisture), and reduce the manufacturing costs associated with special openings required to create other sensors on the surface. The ultrasonic input device can increase the flexibility of button programmability options. For example, a user can define the function of a button through a system controller, which can be embedded on a shared printed circuit board (PCB) with the ultrasonic input device. In some embodiments, the system controller can monitor user behavior to improve machine / system preferences and performance. Ultrasonic input devices that are mechanically coupled to a surface but positioned away from the field of view (such as below or behind an opaque surface) can be used to provide hidden input that is not observable or difficult to find for those who do not know its location. For example, the ultrasonic input device can be placed under a sign (e.g., on a laptop or another surface or device), behind a wall, or under the surface of a piece of furniture.

[0096] The ultrasonic input device may be low power and / or battery powered to operate for extended periods of time without requiring direct connection to a mains power source. The ultrasonic input device may be or may be incorporated into an Internet of Things (IoT) device that can provide sensor data (e.g., button presses) to other devices on a local or remote network. In some cases, the use of a MUT can allow the ultrasonic input device to operate with particularly low power requirements. In some cases, the ultrasonic input device as a fully integrated SOC can operate at low power and / or can provide IoT functionality.

[0097] I. Device Overview

[0098] Embodiments of the present invention relate to ultrasonic input devices for detecting touch input. Specifically, embodiments relate to ultrasonic input devices, which include a transducer coupled to a material layer that provides a surface to receive touch input signals to a system. The ultrasonic input device can be implemented using various material layers, including wood, leather, glass, plastic, metal (e.g., aluminum, steel, etc.), stone, concrete, drywall, plaster, paper, polymers, biomaterials (e.g., tissue such as skin), combinations of one or more materials, and the like. The flexibility of material selection enables the use of ultrasonic input devices in a variety of applications, including front and side buttons of mobile devices; steering wheels, infotainment units, center console controls, mirrors, seats, door handles, windows, etc. of vehicles; Internet of Things devices; medical devices, such as bed controls, blood pressure measurement devices; input detection for robots, such as touch sensing for robot-like fingers; and hidden input devices, such as hidden in furniture or behind walls.

[0099] A. Using ultrasonic signals to detect touch input

[0100] Figure 1 is a schematic diagram according to certain aspects of the present disclosure, which depicts the effect of touch on reflected ultrasonic signals in an ultrasonic input system. The ultrasonic input may include a transducer 104 coupled to a material layer 102. The material layer 102 may be referred to as a stack, and may include one or more sublayers of one or more materials. For example, the stack may be a single sheet of glass, a piece of drywall, a laminated group of plastic and glass, or a plastic steering wheel wrapped in leather, etc. The material layer 102 has a first (inner) surface 106 and a second (outer) surface 108. The material layer may be characterized by a distance 110 between the first surface 106 and the second surface 108. The material layer 102 may be a covering material for a larger device with an integrated ultrasonic input device. In some embodiments, the material layer 102 may form a body or part of a body of the device. In these embodiments, the first surface 106 may form an inner surface of the body, and the second surface 108 may form an outer surface of the body. The second surface 108 may be considered external because it is exposed to the environment. The first surface 106 may be considered internal because it is not the surface on which contact is to be detected, or because it is the surface on which the transducer 104 is acoustically coupled to the material layer 102 . Figure 1 An ultrasound input device 120 with no touch, an ultrasound input device 122 with a light touch, and an ultrasound input device 124 with a heavy touch are shown.

[0101] The touch sensor is triggered based on the material acoustic properties of the touch surface (material layer 102) and the input object 112. The detection of the optical touch 122 depends on the degree of the ultrasonic signal 114 reflected in the material layer 102 relative to the absorbed ultrasonic signal 116 transmitted through the second surface 108 of the material layer 102 into the input object 112. As used herein, a reflected ultrasonic signal (e.g., a reflected ultrasonic signal 114) may refer to a signal that has been reflected off the second surface 108 of the material layer 102, and an absorbed ultrasonic signal (e.g., an absorbed ultrasonic signal 116) may refer to a signal at least a portion of which has been absorbed by the input object 112 (e.g., a finger) contacting the second surface 108 of the material layer 102. The contact (e.g., based on pressure) of the input object 112 on the touch surface defines one or more contact areas 118 and the amount of reflection. The material layer 102 may be a single layer, or may be composed of multiple layers of materials with different properties. For example, in some implementations, the material layer 102 may be a uniform and isotropic material. In other implementations, the material layer 102 may be a composite material layer composed of multiple layers of different materials. The threshold may be set based on the contact area 118 of the touch for triggering the button and the impedance difference between the input object 112 and the material layer 102 as well as the geometric and acoustic properties of the entire material stack of the material layer 102.

[0102] The size of the contact areas 118 and the spaces between the contact areas 118 may indicate the size and spacing of the ridges of a finger and the size and spacing of the valleys of a finger print. Specific variations in the size and / or spacing between the contact areas 118 may indicate different fingers contacting the material layer 102. For example, a younger individual may have smaller valleys (e.g., smaller distances between the contact areas 118) than an older individual. In some cases, the detected size and / or spacing between the contact areas 118 may be used to detect or infer that a user is contacting the material layer 102. Such inferences may be used to customize applications (e.g., to cause a touch event to cause different actions for different users or to cause different users to have different sensing thresholds), test permissions (e.g., to allow an action only if the identified user is initiating a touch event or the user is touching the surface in a specific manner that is the same as a "password"), or to use the inference to perform other rule-based actions.

[0103] A hard touch 124 can be distinguished from a light touch 122 by determining that the transducer 104 receives fewer reflected signals or less unattenuated signals due to an increased amount of absorbed ultrasonic signals 126. If the touch pressure increases, such as when the contact surface becomes flat, the ultrasonic input device 100 and the input object 113 (e.g., a finger) will have a larger contact area 128. Figure 1As shown, a larger contact area 128 increases the amount of absorbed ultrasonic signals 126 transmitted into the input object 113 through the second surface 108 of the material layer 102. In the case of a user's finger, a larger contact area 128 may indicate that the ridges of the user's finger are flattened against the second surface 108 of the material layer 102. In some cases, where the input object 113 is not a finger or is covered by another material, the larger contact area 128 may be the result of a flattening of a texture element of the input object 113 against the second surface 108 of the material layer 102.

[0104] Figure 2 is a schematic diagram according to certain aspects of the present disclosure, which depicts an ultrasound input system in a non-contact state and a contact state. Figure 2 An ultrasonic input device 200 without touch (e.g., non-contact state) and an ultrasonic input device 250 with touch (e.g., contact state) are shown. The ultrasonic input device includes a transducer 202 coupled to a material layer 204. In this embodiment, the material layer 204 is shown as aluminum, but can be any material (e.g., glass, wood, leather, plastic, etc., or a composite material formed by a combination of materials). The transducer 202 is coupled to a first (inner) surface 206 of the material layer 204. A second (outer) surface 208 of the material layer 204 is in contact with air or some other environment similar to a liquid having an acoustic impedance different from that of a human finger.

[0105] For the contactless ultrasonic input device 200, the transducer 202 transmits an ultrasonic signal 210A, which is directed into the material layer 204 and toward the second surface 208. Air has an acoustic impedance of approximately zero and causes the second surface 208 to reflect a reflected ultrasonic signal 212A that is close to 100% (e.g., equal to or greater than 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.1%, 99.2%, 99.3%, 99.4%, 99.5%, 99.6%, 99.7%, 99.8%, 99.9%, 99.10%, 99.11%, 99.12%, 99.13%, 99.14%, 99.15%, 99.16%, 99.17%, 99.18%, 99.19 ...9%, 99.19%, 99.19%, 99.19%, 99.19%, 99.19%, 99.19%, 99.19%, 99.19%, 9 %. The reflected ultrasonic signal 212A may itself reflect off the first surface 206 to generate a reflected-transmitted signal 210B, which may be reflected off the second surface 208 to cause a second reflected ultrasonic signal 212B. In the case of a composite material stack of 204, the signal reflected from 208 may be reflected multiple times within the composite stack itself, and such an echo chain may be sensed by the transducer 202. In the case of composite materials, analysis of the received echo chain formed by reflections between 206 and 208 and / or internal reflections within multiple layers of 204 can be used directly to identify the material stack and / or the environment (e.g., air) of 204. Such information can be used only to identify the acoustic and / or geometric properties of the stack or as additional information for sensor calibration and threshold tuning of detection algorithms. Figure 2As depicted, four reflected ultrasonic signals 212A, 212B, 212C, 212D generate four corresponding reflection-transmitted signals 212B, 212C, 212D, 210E. Any number of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflection-transmitted signals 212B, 212C, 212D, 210E may be obtained from the initially transmitted ultrasonic signal 210A until the signals become too weak to be reflected and / or detected. Graph 214 shows a first amplitude 216 corresponding to the transmitted ultrasonic signal 210A and a set of subsequent amplitudes 218A, 218B, 218C, 218D, 218E corresponding to the reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E. The first subsequent amplitude 218A is less than the first amplitude 216 due to losses in the material layer 204. Each of the remaining subsequent amplitudes 218B, 218C, 218D, 218E is less than the amplitude of the previous subsequent amplitude 218A, 218B, 218C, 218D due to losses in the material layer 204.

[0106] In some cases, one or more frequencies selected for use with an ultrasonic input device may be selected to achieve a small or minimal attenuation in a non-contact state, thereby achieving a large or maximum number of reflected ultrasonic signals. In some cases, a group of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E from a single transmitted ultrasonic signal 210A may be referred to as a train of reflected signals. For illustration purposes, Figure 2 Various reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflected-transmitted signals 210B, 210C, 210D, 210E are depicted from left to right in FIG. However, it will be understood that these signals are separated in time and may not necessarily be separated in space. The echo signals may be analyzed individually and / or combined or integrated with another echo signal as a detection metric analysis.

[0107] For an ultrasonic input device 250 with touch, an input object 220 (a finger in this case) is in contact with the second surface 208 of the material layer 204. The local reflection loss from the area where the object (e.g., the ridges of the finger) is in contact depends on how much the touch input medium differs from the input object in terms of acoustic impedance. For example, the reflection loss (dB) can be expressed as Where Z1 is the impedance of the material layer 204 and Z2 is the impedance of the input object 220. Once the input object 220 contacts the material layer 204, the transmitted ultrasonic signal 210A is split into two parts. The first part, the echo, is the reflected ultrasonic signal 213A and is reflected back to the transducer. The second part 222 is the transmitted signal that penetrates into the input object 220. The reflected ultrasonic signal 213A itself can be reflected off the first surface 206 to generate a reflected-transmitted signal. The reflected-transmitted signal itself can be split into two parts, one of which is the second reflected ultrasonic signal 212B and the other is the second part 222 that penetrates into the input object 220. Figure 2 As depicted, four reflected ultrasonic signals 213A, 213B, 213C, 213D generate four corresponding reflection-transmitted signals. Any number of reflected ultrasonic signals 212A, 212B, 212C, 212D, 212E and reflection-transmitted signals may be generated from the initially transmitted ultrasonic signal 210A until the signals become too weak to be reflected and / or detected.

[0108] As shown in the graph 224, the first amplitude 226 corresponds to the transmitted ultrasonic signal 210A. As compared to the touchless ultrasonic input device, the first subsequent amplitude 228A corresponding to the reflected ultrasonic signal 213A is reduced due to the second portion 222 penetrating the input object 220. Due to losses in the material layer 204 and internal multipath reflections in the case of the composite material stack 204, each of the remaining subsequent amplitudes 228B, 228C, 228D, 228E is less than the amplitude of the previous subsequent amplitude 228A, 228B, 228C, 228D. For illustrative purposes, the graph 224 depicts the subsequent amplitudes 228A, 228B, 228C, 228D, 228E in solid lines, which overlap with the corresponding subsequent amplitudes 218A, 218B, 218C, 218D, 218E depicted in dashed lines. The total attenuation of the subsequent amplitudes 228A, 228B, 228C, 228D, and 228E of the ultrasonic input device in the contact state may be greater than the total attenuation of the subsequent amplitudes 218A, 218B, 218C, 218D, and 218E of the ultrasonic input device in the non-contact state. In addition, the attenuation between each of the subsequent amplitudes 228A, 228B, 228C, 228D, and 228E of the ultrasonic input device in the contact state may be greater than the attenuation of the subsequent amplitudes 218A, 218B, 218C, 218D, and 218E of the ultrasonic input device in the non-contact state.

[0109] It is noteworthy that the subsequent amplitudes 228A, 228B, 228C, 228D, 228E associated with touch events from graph 224 decay faster than the corresponding subsequent amplitudes 218A, 218B, 218C, 218D, 218E associated with no touch events from graph 214. In other words, the contrast between the subsequent amplitudes of touch events and the subsequent amplitudes of no touch events increases with each subsequent number of reflections n. In some cases, the ratio of the nth subsequent amplitude associated with no touch event to the nth subsequent amplitude associated with a touch event may be Γ. n :(1-Γ n ), where Γ is the percentage of the signal reflected back from the second surface 208. For example, the ratio of the subsequent amplitude 218A to the subsequent amplitude 228A may be 100:90; the ratio of the subsequent amplitude 218B to the subsequent amplitude 228B may be 100:81; the ratio of the subsequent amplitude 218C to the subsequent amplitude 228C may be 100:72; the ratio of the subsequent amplitude 218D to the subsequent amplitude 228D may be 100:63; and the ratio of the subsequent amplitude 218E to the subsequent amplitude 228E may be 100:54.

[0110] B. Ultrasonic touch input device

[0111] Figure 3 An ultrasonic input device according to certain aspects of the present disclosure is shown. The ultrasonic input device 300 can be attached to any surface to detect touch input. The ultrasonic input device 300 may include a sensor 302, such as a piezoelectric micromachined ultrasonic transducer (PMUT). A PMUT transducer is a piezoelectric ultrasonic transducer that includes a thin film coupled to a thin piezoelectric film to sense and / or sense ultrasonic signals. The sensor 302 can be integrated on an application specific integrated circuit (ASIC), such as a CMOS (complementary metal oxide semiconductor) ASIC 304 (all-in-one), and formed on a substrate 306. The ASIC 304 may include circuits and / or modules that can be used to perform various processes disclosed herein, such as at least reference to Figure 25-41 The various analog and / or digital processing described above. For example, ASIC 304 may be used to drive sensor 302, detect reflected ultrasonic signals using sensor 302, and determine the amplitude associated with the reflected ultrasonic signals (e.g., using various analog techniques such as accumulation and integration). In some cases, ASIC 304 may optionally determine a threshold value, and the determined amplitude may be compared with the threshold value to determine whether a touch event has occurred, in which case ASIC 304 may output a signal associated with the occurrence of a touch event.

[0112] In some cases, the circuitry of the ASIC 304 may perform certain processes in an analog manner, such as signal rectification, integration, mixing, modification, accumulation, etc. As used herein, analog circuitry may include any circuitry capable of performing an action (e.g., rectification, integration, etc.) on an analog signal without first digitizing the analog signal. In one example, the ASIC 304 may include analog circuitry capable of taking a received ultrasound signal, rectifying the signal, and integrating at least a portion of the rectified signal to provide an integrated signal, such as a reference signal. Fig.26 As described, in another example, the ASIC 304 may include analog circuitry capable of acquiring received ultrasound signals, calculating absolute values ​​of the signals, and accumulating the absolute values ​​to provide an accumulated signal, such as with reference to Fig.28 In another example, the ASIC 304 may include analog circuitry capable of acquiring a received ultrasound signal, squaring the signal by self-mixing, and integrating the squared signal to provide an integrated signal, such as with reference to FIG. Fig.29 As described.

[0113] In some cases, different forms of ultrasonic transducers can be used for sensor 302 instead of a PMUT sensor. In some cases, the ultrasonic sensor can be formed using deposited layers of piezoelectric materials (e.g., aluminum nitride, lead zirconate titanate (PZT), or polyvinylidene fluoride (PVDF)). In some cases, the ultrasonic sensor can be a capacitive micromachined ultrasonic transducer (CMUT). In some cases, the ultrasonic sensor can be a resonator array of a piezoelectric device (e.g., a piezoelectric cantilever or a piezoelectric post).

[0114] The substrate 306 may be bonded 310 to a flexible printed circuit / printed circuit board 308 (FPC / PCB) of a larger integrated device such as a mobile phone. In some embodiments, a contact area 312 on the sensor 302 may be bonded to a substrate contact 314. As shown, the size of the ultrasonic input device 300 may be equal to or less than 1.5 mm x 1.5 mm x 0.5 mm, although other sizes may also be used. In some cases, the FPC / PCB 308 to which the substrate 306 is attached may receive information associated with the amplitude of the detected reflected ultrasonic signal and perform some of the functions disclosed herein, such as determining a threshold and / or determining when a touch event has occurred. However, in some cases, the FPC / PCB 308 only receives a signal associated with the occurrence of a touch event, and therefore, there is no need to perform further analysis of the amplitude of the detected reflected ultrasonic signal to perform an action based on the touch event.

[0115] The integration of ASIC 304 and sensor 302 enables a small form factor, which results in the placement of buttons or other functions in many space-constrained applications. For example, the mechanical buttons on the side of a smartphone can be easily replaced by the ultrasound input device 300 under the casing. To implement a touch interface or other suitable functions of the system, the ultrasound input device 300 can be bonded to the surface 316 using adhesive 318.

[0116] Figure 4 4 is a cross-sectional view of two piezoelectric micromechanical ultrasonic transducers integrated into a CMOS wafer according to certain aspects of the present disclosure. Device 400 shows a cross-sectional view of two PMUTs bonded to a CMOS wafer 402, which can be used in an ultrasonic input device. Each PMUT can be formed on a MEMS wafer 401 bonded to a CMOS wafer 402. In this way, the PMUT can be connected to the necessary processing electronics of the CMOS wafer 402. It should be understood that each PMUT can have an active piezoelectric layer 404 and a first electrode 403 and a second electrode 405. The first electrode 403 and the second electrode 405 can be electrically coupled to the piezoelectric layer 404.

[0117] In some embodiments, the PMUT may include a first contact 422 electrically coupled to the first electrode 403, a second contact 424 electrically coupled to the second electrode 405, and a third electrode 426 electrically coupled to the CMOS wafer 402. Applying an alternating voltage through the first electrode 403 and the second electrode 405 may cause movement (e.g., bending motion) of the piezoelectric layer 404, which may result in generated acoustic waves. Similarly, the received acoustic waves causing movement in the piezoelectric layer 404 may be sensed as a voltage across the first electrode 403 and the second electrode 405. One or more vias (vertical interconnect channels) 410 may be formed in the PMUT. Each contact may be wire-bonded to an electronic board. In some embodiments, the PMUT may include a passivation layer 428 formed on a surface 420 and the contacts. An adhesive coupling surface 430 on the surface of the surface 420 or the passivation layer 428 may be coupled to a material layer of an ultrasonic input device.

[0118] In some embodiments, the passive electrical layer 408 may include SiO2 or any other suitable passive layer. The active piezoelectric layer 404 may be aluminum nitride of approximately 1 μm thick, and the passive elastic layer may be single crystal silicon of approximately 1 μm thick, but other sizes and materials may be used. In some embodiments, the active piezoelectric layer 404 may be aluminum nitride doped with scandium. Alternatively, the active piezoelectric layer 404 may be another suitable piezoelectric ceramic, such as PZT. Both the top and bottom electrodes 406 may include molybdenum. In order to bond the PMUT to the top metal 412 of the CMOS wafer 402, fusion bonding via through silicon vias (TSVs) as shown at the vias 410 may be used. This approach results in significant parasitic reduction, which in turn results in improved signal integrity and lower power consumption.

[0119] In some embodiments, chamber 414 can be formed with a vacuum or near vacuum to isolate the transducer from the processing electronics in CMOS wafer 402. The sound generated by the PMUTS will not propagate through the near vacuum of chamber 414, thereby minimizing reflections and interference that may be caused by the material interface with CMOS wafer 402. Chamber 414 can cause ultrasound 416 to travel away from the PMUT. Ultrasound 416 can travel through the adhesive coupling surface 430 and enter the material layer of the ultrasound input device. The material layer can reflect ultrasound 416, thereby causing the return echo to be reflected back to the PMUT. The return echo travels through the adhesive coupling interface and is received by the PMUT.

[0120] In some embodiments, the CMOS wafer 402 may be an application specific integrated circuit (ASIC) including one or more devices required to drive the transducer. The driving voltage for the PMUT array may be less than 4 volts. In some cases, the driving voltage may be less than 1.8 volts. In some cases, the driving voltage may be or less than 4, 3.5, 3, 2.5, 2, 1.9, 1.8, 1.7, 1.6, or 1.5 volts. The ASIC may be manufactured to meet the size requirements associated with the size of the associated PMUT. In some embodiments, the ASIC may include one or more modules to receive the measured signal. The ASIC may be configured to further process the signal. For example, the ASIC may include one or more rectifiers to generate an absolute value signal by obtaining the absolute value of the received signal (which may be an alternating current). The ASIC may also include an integrator and an analog-to-digital converter (ADC) to convert the reflected ultrasonic signal into a digital representation of the reflected signal. The integration of the ASIC and the PMUT further allows the embedding of a gain amplifier and an ADC in the ASIC and eliminates a separate ADC sensor controller chip. This opens up space on the associated circuit board and reduces the implementation cost of the touch input sensor. In some embodiments, the ASIC may transmit the digital signal to at least one or more of a memory, a processor, and a remote device. In other embodiments, the ASIC may include one or more signal processing modules.

[0121] The PMUT array can be compatible with CMOS semiconductor processes. In some embodiments, the PMUT materials and sizes can comply with the semiconductor equipment and materials international (SEMI) standard specifications. Because the PMUT can comply with the SEMI specifications, the transducer array can be used with existing CMOS semiconductor manufacturing tools and methods. For example, one or more PMUTs can be formed using photolithography. In contrast, current piezoelectric ultrasonic transducer arrays are formed using wafer saws that cannot match the precision of photolithography. As a result, the PMUT can be smaller, operate at lower voltages, and have lower parasitics.

[0122] C. Integration with circuit boards

[0123] Figure 5502, 504, 506, 508, depicting ultrasonic input devices 510, 512, 514, 516 coupled to various surfaces, according to certain aspects of the present disclosure. FIG. 502 depicts an ultrasonic input device 510 coupled to a metal surface by an adhesive. FIG. 504 depicts an ultrasonic input device 512 coupled to a glass surface by an adhesive. FIG. 506 depicts an ultrasonic input device 514 coupled to a plastic surface by an adhesive. FIG. 508 depicts an ultrasonic input device 516 coupled to a wooden surface by an adhesive. Any suitable material may be used as a sensing surface, such as a non-porous material or a semi-porous material. Porous materials may be used for sensing surfaces, but better results may be obtained with smaller pores, higher density, and more consistent density.

[0124] In addition, the ultrasound input devices 510, 512, 514, 516 may be coupled to a flexible PCB, such as on the opposite side of the ultrasound input devices 510, 512, 514, 516 that are coupled to the sensing surface. The ultrasound input devices 510, 512, 514, 516 may be used as a mechanical coupler between the sensing surface and the PCB, where the PCB is not attached to the sensing surface elsewhere, although this is not always required. In some cases, a flexible PCB may be used.

[0125] The use of a PCB may allow additional components to be integrated with the ultrasound input devices 510, 512, 514, 516 to expand the functionality of the ultrasound input devices 510, 512, 514, 516, such as with reference to Figure 6 As described.

[0126] Figure 6 6 is a schematic side view of an ultrasonic input system 600 having a common board assembly according to certain aspects of the present disclosure. The ultrasonic input system 600 may include an ultrasonic input device 602 electrically coupled to a circuit board 610, and any number of common board assemblies 612. Each common board assembly may be electrically coupled to the circuit board 610. In some cases, the ultrasonic input device 602 may be mechanically coupled to the circuit board 610, such as using an electrical connection (e.g., a solder joint) or other mechanical support. In some cases, one, some, or all of the common board assemblies 612 may be mechanically coupled to the circuit board 610. In some cases, the circuit board may be a printed circuit board, such as a flexible PCB, although this need not always be the case.

[0127] The entire ultrasound input system 600 may be contained within a single common housing, within multiple housings, or may not be contained within a housing. In some cases, two or more common board assemblies 612 may be contained within a single housing, with or without the ultrasound input device 602. In some cases, all of the common board assemblies 612 may be located on the same side of the circuit board 610 as the ultrasound input device 602, although this need not always be the case. When located on the same side as the ultrasound input device 602, the common board assembly 612 may be selected or designed to have a height that exceeds the height of the ultrasound input device 602.

[0128] In some cases, the ultrasound input system 600 may include a power assembly 604. The power assembly 604 may provide power to the ultrasound input device 602 and / or any other common board components 612. Examples of the power assembly 604 include a battery, a transformer (e.g., a transformer coupled to a mains), a capacitor (e.g., a supercapacitor), a solar cell, a fuel cell, and / or any other suitable power source.

[0129] In some cases, the ultrasound input system 600 may include a processor 606. The processor 606 may enable various processing functions to be performed within the ultrasound input system 600 based on signals received from the ultrasound input device 602. Examples of suitable processors 606 include microcontrollers, central processing units, or other suitable devices. The processor 606 may also be coupled to a memory to access processing routines, access stored data, and / or store data.

[0130] In some cases, the ultrasound input system 600 may include a communication component 608. The communication component 608 may interact with the ultrasound input device 602 and / or the processor 606 to send signals to or receive signals from an external device. Examples of suitable communication components 608 include wireless radios (e.g., Bluetooth, WiFi, Zigbee, Z-Wave, etc.), audio devices (e.g., microphones or speakers), visual devices (e.g., cameras, lights, or displays), tactile devices (e.g., tactile feedback devices such as motors and vibrators), or other devices suitable for sending or receiving signals.

[0131] In some cases, the ultrasound input system 600 can include a common board assembly 612 that includes a power assembly 604, a processor 606, and a communication assembly 608. In some cases, the ultrasound input system 600 can include more or fewer common board assemblies that include different types of components.

[0132] D. Example System Settings

[0133] Figure 7is a schematic diagram of an example ultrasound input system 700 according to certain aspects of the present disclosure. The ultrasound input system 700 may include an ultrasound sensor 702 and a processor 722. The ultrasound sensor 702 may be coupled to Figure 1 The processor 722 may be electrically coupled to the ultrasonic sensor 702 and may be optionally mechanically coupled to the ultrasonic sensor 702, similar to the transducer 104 of the present invention. In some cases, the processor 722 and the ultrasonic sensor 702 may be integrated into the same package, although this need not always be the case. The processor 722 may perform specific functions as disclosed herein, such as acquiring signals from the ultrasonic sensor 702 and / or detecting touch events. In some cases, an optional computing device 724 may be coupled to the processor 722 to exchange information, such as information related to touch events, information related to signals from the ultrasonic sensor 702, or information related to how the processor 722 interprets signal information. A data store 726 may be coupled to the processor 722 for storing information, such as information related to how the processor 722 interprets signal information. In some cases, the optional computing device 724 may be coupled to a data store 728 that may store information, such as information related to how to interpret signal information from the ultrasonic sensor 702 to determine a touch event. The computing device 724 may be any suitable computing device, such as a desktop computer, a laptop computer, a server, a smart phone, a tablet computer, or any other suitable computing device. The computing device 724 may be coupled to the processor 722 via a wired or wireless connection. The computing device 724 may be coupled to the processor 722 via a local or remote connection.

[0134] In some cases, the processor 722 may be an application specific integrated circuit (ASIC). In some cases, the ultrasonic sensor 702 may be a MUT. The processor 722 may be any suitable circuit designed to be able to drive and receive one or more transducers of the sensor 502. The processor 722 may drive the transducer to send and receive ultrasonic signals to implement the touch sensing capabilities described herein. In some cases, the processor 722 may output a measured energy level (e.g., an energy signal) associated with the sensor 502, which may be used later to determine whether a touch event has occurred. In some cases, the processor 722 may output a touch signal indicating the occurrence of a touch event. In this case, the processor 722 may perform the necessary processing to determine whether a touch event has occurred. In some cases, the processor 722 may further perform the necessary processing to determine additional information associated with the touch event, such as whether the touch event is initiated by a bare finger or a gloved finger, whether the touch event is initiated by a first user or a second user, or other aspects of the touch event. This additional information may take the form of an inference and may have different confidence levels, although this need not always be the case. In some cases, the processor 722 may have the ability to process signals and recognize the type of pattern that the user is entering (e.g., single click, double click, hold, etc.). This capability in the processor 722 may be implemented by a hardware processing block, or may be written into the chip memory as part of the firmware. In some cases, the processor 722 may have the ability to self-calibrate and tune its parameters for signal recognition and pattern recognition.

[0135] In some cases, processor 722 may send the energy signal and / or the touch signal to computing device 724. Computing device 724 may perform the necessary processing to determine whether a touch event has occurred and / or additional information associated with the touch event, such as whether the touch event was initiated by a bare finger or a gloved finger, whether the touch event was initiated by a first user or a second user, or other aspects of the touch event.

[0136] In some cases, data store 726 can store information related to how processor 722 determines whether a touch event has occurred or determines other information associated with a touch event. In some cases, data store 726 can store model information used by processor 722 to process energy signals and determine whether a touch event has occurred. In some cases, the model information stored in data store 726 can be provided and / or updated using computing device 724.

[0137] II. Fully Integrated System-on-Chip for Ultrasonic Touch Input

[0138] Embodiments of the present disclosure allow for a fully integrated system-on-chip for ultrasonic touch input. For example, an integrated ultrasonic input device may include an ultrasonic sensor and an application specific integrated circuit (ASIC). Various production techniques may be used to create an integrated ultrasonic input device wafer to allow for low profile size as well as improved noise immunity and lower power.

[0139] A. Overview of the integrated ultrasound input device

[0140] Figure 8 8 is a schematic side view of an integrated circuit 820 having an ultrasound sensor 802 and an application specific integrated circuit (ASIC) 822, according to certain aspects of the present disclosure. The ultrasound sensor 802 may be composed of one or more ultrasound transducers arranged in an array. In some cases, the ultrasound transducer is a MUT.

[0141] ASIC 822 may be any suitable circuit designed to be capable of driving and receiving one or more transducers of ultrasonic sensor 802. ASIC 822 may drive transducers to send and receive ultrasonic signals to implement the touch sensing capabilities described herein. In some cases, ASIC 822 may output a measured energy level associated with ultrasonic sensor 802, which may be used later to determine whether a touch event has occurred. In some cases, ASIC 822 may output a touch signal indicating the occurrence of a touch event. In this case, ASIC 822 may perform necessary processing to determine whether a touch event has occurred. In some cases, ASIC 822 may further perform necessary processing to determine additional information associated with the touch event, such as whether the touch event is initiated by a bare finger or a gloved finger, whether the touch event is initiated by a first user or a second user, or other aspects of the touch event. This additional information may take the form of an inference and may have different confidence levels, although this need not always be the case. In some cases, ASIC 822 may have the ability to process signals and identify the type of pattern that the user is inputting (e.g., single click, double click, hold, etc.). This capability in ASIC 822 may be implemented by a hardware processing block, or may be written into chip memory as part of firmware. In some cases, ASIC 822 may have the ability to self-calibrate and tune its parameters for signal recognition and pattern recognition.

[0142] The integrated ultrasonic input device 820 can be completely or partially encapsulated in the housing 824 to form a package. The housing 824 can take the form of any suitable material, such as a cured resin. In some cases, the housing 824 contains only the ultrasonic sensor 802 and the ASIC 822, and any electrical contacts necessary to connect the ASIC 822 to external components. In some cases, the housing 824 may contain additional components, such as additional sensors (e.g., thermal sensors, vibration sensors, or gyroscopes). In some cases, the material used for the housing 824 can be selected to perform well as part of the stacking of the ultrasonic input system. For example, a material with maximum energy transmission in the frequency range associated with a specific ultrasonic input device 820 can be used to maximize the signal. In some cases, additional materials can be used in the housing 824 or incorporated into the housing 824 itself to achieve the desired response of ultrasonic propagation into the stack. For example, a window can be assembled into the housing 824 adjacent to the ultrasonic sensor 802 to provide a path for transmitting ultrasonic signals to and from the ultrasonic sensor 802. The window may be made of optically transparent, translucent or opaque material and may be selected to allow the ultrasonic signal to pass through with little or no attenuation. Furthermore, materials may be used in the stack to enhance the acoustic matching between layers to facilitate transmission and / or reception of signals.

[0143] In some cases, the housing 824 may be applied after the ultrasonic sensor 802 and the ASIC 822 have been formed into a wafer and diced into individual chips. However, in some cases, the housing 824 may be applied while the ultrasonic sensor 802 and the ASIC 822 are still part of a wafer containing many chips. Any suitable chip packaging method may be used to package the ultrasonic sensor 802 and the ASIC 822.

[0144] In some cases, other types of processors or circuits may be used in place of the ASIC 822. For example, a general purpose programmable processor may be used in place of the ASIC 822 while still achieving many of the benefits associated with the integrated ultrasound input device 820. In some cases, the ASIC 822 may receive power as an input, which may be used to power the ASIC 822 itself and drive the transducer of the ultrasound sensor 802. In some cases, the general purpose programmable processor may be used to communicate between multiple chips with or without internal ASICs in a master and slave format.

[0145] In some cases, the height of the package of the integrated ultrasound input device 820 can be about 500 microns or less. In some cases, the ultrasound sensor 802 and ASIC 822 of the integrated ultrasound input device 820 can have a combined height of about 150 microns or less.

[0146] B.Production Technology

[0147] Fig. 9 is a combined set of schematic diagrams according to certain aspects of the present disclosure that depict two processing routes 926, 928 for producing an integrated ultrasound input device wafer 930. The first processing route 926 depicts the use of a monolithic technique to produce the wafer 930. The second processing route 928 depicts the use of a wafer bonding technique to produce the wafer 930. Any suitable process may be used to produce the wafer 930 containing the sensor and ASIC as described herein.

[0148] Under the first processing route 926, an ASIC wafer 932 is provided, and then a sensor layer 934 is built on the ASIC wafer 932 to produce a monolithic wafer 930 containing both the sensor and the ASIC. This type of wafer-level manufacturing can allow small form factors to be produced in an economical manner.

[0149]

[0066] Under the second processing route 928, a sensor layer 934 is provided, and an ASIC wafer 932 is provided. The provided sensor layer 934 may then be bonded to the ASIC wafer 932 using any suitable wafer bonding technique, with or without intermediate layers.

[0150] Wafer 930 resulting from first processing route 926, second processing route 928, or any other suitable processing route may include one or more instances of sensors and ASICs that may be used to create an integrated ultrasound input device.

[0151] Fig.10 1 is a set of schematic diagrams depicting a single integrated ultrasonic input device 1020 cut from a wafer 1030, PCB mounted, and stacked mounted according to certain aspects of the present disclosure. The wafer 1030 may be Fig.10 The wafer 1030 may be cut or diced into a plurality of pieces (e.g., wafers). Each wafer 1042 may contain the sensor 1002 and the ASIC 1022 for a single integrated ultrasonic input device 1020. If a housing has not been previously applied to the wafer 1030, each wafer 1042 may be packaged in a housing to produce an integrated ultrasonic input device 1020.

[0152] The ultrasound input device 1020 can be mounted on a printed circuit board (PCB) 1036 or otherwise electrically coupled to any other necessary electronics. For example, in some cases, the ultrasound input device 1020 can be electrically coupled to a battery or other power source. In some cases, the ultrasound input device 1020 can be mounted on a PCB 1036 that contains other electronic components 1038, such as a processor and a power source.

[0153] The ultrasonic input device 1020 can be mounted to a substrate 1040. The substrate 1040 can be any combination of one or more materials through which an ultrasonic signal can be transmitted to the sensor 1002. The housing of the ultrasonic input device 1020 can be coupled to the substrate 1040. The combination of materials through which the ultrasonic signal is transmitted from the outer surface of the substrate 1040 to the sensor 1002 can be referred to as a stack, which can include the housing of the ultrasonic input device 1020. The ultrasonic input device 1020 can be coupled to the substrate 1040 using any suitable technique, including the use of adhesives, mechanical coupling, active pressure, or any other suitable technique for acoustically coupling the ultrasonic input device 1020 and the substrate 1040.

[0154] C. Low profile size

[0155] Fig.11 1 is a schematic cross-sectional view of a consumer electronic product 1100 including an integrated ultrasonic input device 1120 according to certain aspects of the present disclosure. The consumer electronic product 1100 may be a smart phone or any other suitable device. The integrated ultrasonic input device 1120 may be attached to a substrate consisting of one or more layers of a display 1140, or to any other part of the consumer electronic device, such as a frame or back (1144) that may be made of metal, plastic, or other materials. The display 1140 may include multiple layers, including a display layer, an illumination layer, a protective layer, a sensing layer, and other suitable layers. Through its connection with the display 1140, the integrated ultrasonic input device 1120 may be used to register touch events associated with the display 1140. However, in some cases, the integrated ultrasonic input device 1120 may be connected to any surface of the consumer electronic product to detect touch events on the opposite side of the surface, such as the back side or side edge of the consumer electronic product.

[0156] As described herein, the integrated ultrasonic input device 1120 can be formed to have a very small height, for example, equal to or less than 500 microns. Due to the low profile of the integrated ultrasonic input device 1120, one or more such integrated ultrasonic input devices can be easily positioned within the consumer electronic product 1100, leaving ample space for other components. For example, the low profile of the integrated ultrasonic input device 1120 can occupy only a small portion of the total height of the consumer electronic product 1100, thereby allowing more space for other components, such as a larger battery 1144 with a larger capacity, or a more open space for airflow. In addition, due to the design and physical properties behind the operation of the device, the integrated ultrasonic input device can be operated in a small local area for transmitting and receiving ultrasonic information. This local operation greatly improves the performance robustness of the device to interference sources (such as touch or hold) caused outside the operating area.

[0157] D. Improved noise immunity and lower power

[0158] Fig.12 1 is a set of schematic cross-sectional views comparing a non-integrated ultrasonic input device 1200 with an integrated ultrasonic input device 1220 according to certain aspects of the present disclosure. The non-integrated ultrasonic input device 1200 is more susceptible to noise due, at least in part, to the relatively long length of wires required to connect the ASIC and the sensor. For example, the non-integrated ultrasonic input device 1200 may have an exposed electrical trace. Not only is power used to transmit the signal along the electrical trace, but the electrical trace may also be further susceptible to interference. Therefore, the overall signal-to-noise ratio of the non-integrated ultrasonic input device is relatively low. If a higher signal-to-noise ratio is required, the ASIC must provide more power to drive the sensor, in which case the entire system will have a relatively higher power consumption.

[0159] In contrast, the integrated ultrasonic input device 1220 of the present disclosure is an integrated chip encapsulated in a housing. The integrated ultrasonic input device 1220 does not have large exposed traces or wires between the sensor and the ASIC. Therefore, due at least in part to the minimal conductive traces between the sensor and the ASIC, there is little or no risk of interference and little or minimal energy depletion when transmitting signals from the sensor to the ASIC. Therefore, compared to a similar non-integrated ultrasonic input device 1200, the integrated ultrasonic input device 1220 can operate with an improved signal-to-noise ratio and / or with improved power efficiency.

[0160] III. Ultrasonic sensor design

[0161] The ultrasound input device may include multiple transducers, which may be configured as, for example, a sensor array. In some embodiments, multiple transducers may allow measurement of multiple frequencies. In addition, in other embodiments, multiple transducers may allow separation of transmission and reception capabilities. For example, some transducers may be configured to transmit ultrasound signals, while other transducers in the multiple transducers may be configured to receive ultrasound signals. In other embodiments, multiple transducers may allow beamforming.

[0162] A. Transducer Array

[0163] Fig.13A 1 is a top view of a sensor array 1302 of an ultrasound input device according to certain aspects of the present disclosure. The sensor array 1302 may include one or more transducers 1350 (eg, MUTs). In general, the sensor array 1302 may have a plurality of transducers 1350. Fig.13AThe sensor array 1302 is described as having 144 different transducers 1350 that are traversed across the approximately 1.2 mm square sensor array 1302, although other numbers of transducers 1350 and arrays of other sizes may be used. Various electrical traces in the sensor array 1302 may interconnect the different transducers 1350 to the ASIC. Each transducer 1350 may be independently addressable. In some cases, the use of a transducer 1350 for a particular purpose (e.g., as a transmitter or receiver, or having certain specific frequencies) may be set or changed by the ASIC, so that each transducer 1350 may perform any specific function performed by any other transducer 1350 of the sensor array 1302. However, in some cases, one or more transducers 1350 may be specifically selected or configured to perform a specific function more efficiently or effectively. For example, some transducers 1350 may be designed to achieve improved transmission, while other transducers 1350 may be designed to achieve improved reception.

[0164] Fig. 13B 1 is a top view of an alternative sensor array 1312 of an ultrasonic input device according to certain aspects of the present disclosure. The sensor array 1312 may include one or more transducers 1360. The sensor array 1312 depicts a sensor array including 36 ultrasonic transducers 1360. Various electrical traces in the sensor array 1312 may interconnect different transducers 1360 with the integrated circuit layer. One or more transducers 1360 of the sensor array 1312 may be transmitting ultrasonic transducers. One or more transducers 1360 of the transducers 1360 of the sensor array 1312 may be receiving ultrasonic transducers. As described herein, the transducers 1360 of the sensor array 1312 may transmit and receive at any suitable frequency. The relative size of the transducers 1360 may indicate the frequencies that can be transmitted / received by the transducers.

[0165] Various electrical traces (not shown) in the sensor array 1312 may interconnect different transducers 1360 with the integrated circuit. The various electrical traces may interconnect different transducers 1360 in any suitable manner. For example, the electrical traces may connect transducers 1360 in horizontal and vertical grids. As another example, the electrical traces may connect transducers 1360 that are positioned diagonally from each other.

[0166] Fig.14A According to certain aspects of the present disclosure Fig.13AFIG. 1 is a schematic diagram of a sensor array 1302 depicting an example configuration of the individual transducers of the sensor array 1302. In this example configuration, of the 144 different transducers in the sensor array 1302, 60 are configured to operate as low frequency transmitters, 8 are configured to operate as low frequency receivers, 56 are configured to operate as high frequency transmitters, and 20 are configured to operate as high frequency receivers. Fig.14A The configuration depicted in can be particularly useful for sensing touch events using multiple ultrasonic frequencies to, for example, better identify environmental changes relative to real touch events and / or improve the operating frequency bandwidth of the device to make the device more responsive over a wider frequency range.

[0167] Fig.14A A sensor array 1402 including four corner regions is further shown. The corner regions of the sensor array 1402 may include multiple transducers. For example, the sensor array 1402 includes four rotationally symmetrical corner regions, which mainly include (e.g., most) low-frequency transmission ultrasonic transducers, which can surround the low-frequency receiving ultrasonic transducer. The corner regions of the sensor array 1402 include 16 ultrasonic transducers in a 4x4 array. However, it should be understood that the corner regions of the sensor array 1402 may include up to one-quarter of the total number of ultrasonic transducers included in the sensor array 1402. For example, a square sensor array including 81 transducers may include four corner regions. Each of the four corner regions may include a transducer grid of 1x1, 2x2, 3x3, or 4x4. In some cases, the sensor array 1402 may include a high-frequency transmission ultrasonic transducer surrounding the low-frequency transmission ultrasonic transducer, for example, as shown. In some implementations, the high-frequency transmitting ultrasonic transducer may not be on the diagonal line of the corner region, but a high-frequency receiving ultrasonic transducer may be present, for example, Fig.14A As shown. In addition, the central region may mainly include low-frequency transmitting ultrasonic transducers. The central region may be surrounded by high-frequency transmitting ultrasonic transducers. In some cases, the central region may include transmitting ultrasonic transducers. In other cases, the central region may include receiving ultrasonic transducers. The central region of the sensor array may have any suitable size, for example, 1x1, 2x2, 3x3, 4x4, 5x5, 6x6, 7x7, etc.

[0168] In some cases, sensor array 1302 may have any number of transducers operating at any number of different frequencies. Fig.13AThe example configurations of may be useful in some cases, but other configurations may be used. In some cases, a single type of sensor array 1302 may be manufactured in batches and used with the same or different types of ASICs. For example, different types of ASICs may be configured to operate the same sensor array 1302 in different configurations (e.g., with more or fewer transmitters or receivers, different frequencies, or a greater or lesser number of different frequencies). In some cases, the same type of ASIC may also be programmed to operate in different configurations. In some cases, integrated versions of transducers and / or ASICs may be used in conjunction with non-integrated transducers and / or ASICs to achieve specific purposes, such as increasing transmission power.

[0169] exist Fig.13A and Fig.14A The sensor array depicted in 1312 may include one or more piezoelectric micromachined ultrasonic transducers, one or more capacitive micromachined ultrasonic transducers, one or more integrated block piezoelectric transducers, or one or more non-integrated block piezoelectric transducers. In some cases, the sensor array may include any suitable combination of the above transducers. In addition, the sensor array may have any suitable size. For example, the sensor array may include an array of ultrasonic transducers of 2x2, 3x3, 5x5, 9x9, 16x16, etc. For example, sensor array 1312 depicts a sensor array of 6 ultrasonic transducers by 6 ultrasonic transducers.

[0170] Fig. 14B is a schematic diagram of a sensor array according to certain aspects of the present disclosure, which depicts eight example configurations of individual transducers in the corner regions of the sensor array. Fig. 14B The sensor array depicted in FIG. 1 is 12x12 transducers in size and the corner regions are 4x4 transducers in size, however, it should be understood that embodiments may include sensor arrays and corner regions of any suitable size.

[0171] Each corner region of the sensor array 1410-1424 may include a transmitting ultrasonic transducer 1426 and a receiving ultrasonic transducer 1425. The transmitting ultrasonic transducers and the receiving ultrasonic transducers may be arranged as in the sensor arrays 1410-1424. Thus, in various combinations, the receiving transducers may be diagonal to each other, where the diagonal may be in various positions and have various lengths. The receiving transducers may be in blocks (e.g., 2x2) in various positions and may be other shapes, including an odd number of receiving transducers.

[0172] For example, the transmitting ultrasound transducers in the sensor array 1410 may each transmit at the same frequency. Similarly, the receiving ultrasound transducers in the sensor array 1410 may each receive the same frequency, which may be the same frequency transmitted from the transmitting ultrasound transducer. The corner regions need not all be the same and may appear in various combinations, for example, a combination may have one type selected from 1410, 1412, 1414, and 1416. The interior region may have Fig. 14C Various combinations shown in .

[0173] Fig. 14C is a schematic diagram of a sensor array according to certain aspects of the present disclosure, which depicts eight example configurations of individual transducers of the sensor array. Fig. 14C The sensor array depicted in 1430 depicts transducers inside the corner regions of the sensor array. The interior region may include transducers between at least two corner regions. The transducers shown in the sensor array may include a transmitting ultrasonic transducer 1446 and a receiving ultrasonic transducer 1445. The transmitting ultrasonic transducer and the receiving ultrasonic transducer may be arranged as in the sensor arrays 1430-1444. Fig. 14C Any arrangement in Fig. 14B Use with any corner arrangement in .

[0174] As depicted, the majority may be transmitting transducers, but they may be in the minority. The receiving transducers may contact each other to form a ring, for example, as in sensor arrays 1430-1438. As an alternative, the receiving transducers may form disjoint groups, as in sensor arrays 1440-1444. In such disjoint groups, there may be an even or odd number of receiving transducers. These groups may all be the same or may vary.

[0175] Fig.14D is a schematic diagram of a sensor array according to certain aspects of the present disclosure, which depicts eight example configurations of individual transducers of the sensor array. Fig.14D The sensor arrays depicted in 1450 depict sensor arrays that include different numbers of ultrasound transducers. For example, sensor array 1450 includes 36 transducers, while sensor array 1460 includes 64 transducers. Transmitting ultrasound transducers 1448 and receiving ultrasound transducers 1447 can be arranged as in sensor arrays 1450-1464.

[0176] Fig.14E is a schematic diagram of a sensor array depicting two example configurations of individual transducers of the sensor array in accordance with certain aspects of the present disclosure. Fig.14ETwo example sensor arrays are shown, which are 12x12 ultrasonic transducers in size. For example, both sensor array 1465 and sensor array 1466 include 144 transducers. Transmitting ultrasonic transducers 1492 and receiving ultrasonic transducers 1491 may be arranged as in sensor arrays 1465-1466. In some implementations, the central area of ​​sensor arrays 1465-1466 may not include ultrasonic transducers. Sensor array 1465 may then include, for example, 138 ultrasonic transducers. However, it should be understood that the central area may be larger or smaller than the size of 16 transducers in a square. In some implementations, ultrasonic transducers may form a ring around a central area that does not include ultrasonic transducers. Sensor arrays that do not include ultrasonic sensors may include a central area for routing space.

[0177] In some cases, the ultrasonic transducers in the sensor array can be grouped. For example, the sensor array 1465 can include 8 groups of ultrasonic transducers, each of which can include 16 ultrasonic transducers included in a square shape. The central area of ​​the sensor array 1465 does not include a group of ultrasonic transducers. Each group of ultrasonic transducers can be disconnected from each other. For example, there can be a gap between two or more groups of ultrasonic transducers. These groups can be disconnected horizontally, vertically, and diagonally.

[0178] Fig.14F is a schematic diagram of a sensor array according to certain aspects of the present disclosure, depicting example configurations of individual transducers of sensor arrays of various sizes. Fig.14F Ten example sensor arrays of varying sizes are shown. For example, sensor array 1467 includes 16 ultrasonic transducers, sensor array 1468 includes 25 ultrasonic transducers, sensor array 1469 includes 36 ultrasonic transducers, sensor array 1470 includes 49 ultrasonic transducers, sensor array 1471 includes 64 ultrasonic transducers, sensor array 1472 includes 81 ultrasonic transducers, sensor array 1473 includes 100 ultrasonic transducers, sensor array 1474 includes 131 ultrasonic transducers, sensor array 1475 includes 144 ultrasonic transducers, and sensor array 1476 includes 169 ultrasonic transducers. The transmitting ultrasonic transducer 1494 and the receiving ultrasonic transducer 1493 can be arranged as in the sensor arrays 1467-1476. However, it should be understood that the configuration of the transmitting ultrasonic transducer 1494 and the receiving ultrasonic transducer 1493 can be any suitable arrangement as described herein.

[0179] Figure 14G14 is a schematic diagram of a sensor array according to certain aspects of the present disclosure, which depicts twelve example configurations of individual transducers of a sensor array of transducers of size 8x8. For example, the sensor arrays 1477-1488 include 81 transducers. The transmitting ultrasound transducer 1496 and the receiving ultrasound transducer 1495 can be arranged as in the sensor arrays 1477-1488.

[0180] In some implementations, the sensor array may include the sensor arrays described herein and with reference to Figures 13A-13B Any suitable combination of sensor array characteristics (e.g., regions, groups, arrangements, etc.) described in 14A-14G. For example, the sensor array may include Fig. 14B The corner regions depicted in the sensor array 1420 and the Fig.14F The inner area and the central area depicted in the sensor array 1472. The arrangement of the ultrasound transducers in the sensor array can be based on the application of the sensor array, the frequency of operation, size restrictions, power constraints, etc.

[0181] Multiple embodiments provide many advantages. For example, different array sizes can be implemented depending on sensor area (physical size) limitations and power constraints. The total array size, the configuration of the transmit and receive ultrasonic transducers (e.g., pMUTs), and the size of the ultrasonic transducers can be used to determine the transmit and receive acoustic apertures and beam shapes. The transmit and receive acoustic apertures and beam shapes can be altered using at least the above-mentioned characteristics, for which different stack thicknesses and materials and the use of sensor arrays can be selected to produce optimal performance given constraints (e.g., size, power, sampling frequency, supply voltage, process breakdown voltage, etc.).

[0182] B. Multi-frequency measurement

[0183] Fig.15 1502, 1504, 1506 depict energy measurements from transducers of a single sensor array operating at different frequencies according to certain aspects of the present disclosure. Graphs 1502, 1504, 1506 show energy measurements over time for a pair of touch events. Graph 1502 depicts energy measurements for a transducer operating at 100 kHz, graph 1504 depicts energy measurements for a transducer operating at 1 MHz, and graph 1506 depicts energy measurements for a transducer operating at 10 MHz. Clearly, the measurements taken at these different frequencies have different energy traces, particularly with respect to temperature drift.

[0184] Since the drop in energy measurement associated with the ultrasonic transducer receiving the reflected ultrasonic signal is used as a factor in identifying a touch event, it may be desirable to find techniques to reduce any false touch events. Fig.15As depicted, energy measurements at different frequencies react differently to temperature changes (e.g., temperature changes that occur when heat is transferred from a finger to a substrate or from a substrate to air, or other such temperature changes). Thus, rather than simply relying on identifying a drop in energy measurements to infer a touch event, an ultrasonic touch input system can use energy measurements at multiple frequencies or other types of operating procedures, such as different ultrasound beam shapes, number of pulses, etc. to confirm or reject an inference of a touch event. For example, a perceived energy drop in graph 1502 may not register as a touch event because a concurrent energy drop was not identified in graphs 1504 or 1506. However, once all three graphs 1502, 1504, 1506 register a concurrent energy drop, it can be assumed that a touch event has occurred.

[0185] Fig.16 1600 is a graph depicting the temperature behavior of an ultrasonic transducer with respect to operating frequency according to certain aspects of the present disclosure. Graph 1600 includes four lines, each associated with an air signal or a target signal at a first or second frequency. The air signal may refer to the energy measured when there is no touch event, while the target signal may refer to the energy measured when a touch event is occurring. The first and second frequencies may be any suitable different frequencies. Graph 1600 shows. For all signals, the overall signal strength decreases as the temperature increases. Graph 1600 also shows the different behavior of each frequency with respect to temperature, which can therefore be used to help identify whether a touch event has occurred (e.g., identifying whether a change in energy measurement is associated with a touch event or simply associated with a temperature drift).

[0186] In an example, first and second measurements may be made by a transducer operating at a first frequency, which results in measurements at point 1610 and line 1612. At this point, it may not be clear whether the measurement at line 1612 is associated with a touch event (e.g., moving from point 1610 to point 1614) or a temperature change (e.g., moving from point 1610 to point 1616). First and second measurements may also be made with a transducer operating at a second frequency, which results in measurements at point 1618 and either line 1620 or line 1622. If the second measurement at the second frequency falls on line 1620, it may be inferred that the energy drop is associated with a temperature change from point 1618 to point 1626, and is therefore unlikely to be associated with a touch event. However, if the second measurement at the second frequency falls on line 1622, it may be inferred that the energy drop is associated with a touch event because the energy drops from point 1618 to point 1624. The measurements at the first and second frequencies may be made simultaneously, sequentially, or otherwise close in time to each other (e.g., within a few milliseconds, tens of milliseconds, or hundreds of milliseconds of each other). Therefore, by comparing changes in energy measurements over a period of time at multiple frequencies, it can be determined whether a touch event has occurred.

[0187] Although graph 1600 has been described with reference to frequency-dependent energy changes due to temperature changes, this technique may be used to identify and exploit frequency-dependent energy changes due to changes in other environmental conditions, such as humidity.

[0188] Fig.17 1700 is a graph depicting a frequency response for a stacked structure according to certain aspects of the present disclosure. Graph 1700 shows three lines, each associated with a different stack. Each different stack may be constructed of different materials or different combinations of materials. Due to inherent differences in each stack, each stack may have a unique response curve associated with the transmission frequency used by the ultrasonic input device. The response curve may be a measure of energy, received signal peak, or any other quality factor. Fig.17 As depicted, the frequency providing the highest response for stack / cover 1 is higher than the frequency providing the highest response for stack / cover 2 , which itself is higher than the frequency providing the highest response for stack / cover 3 .

[0189] Thus, specific frequencies and stack materials can be matched to provide optimal results. For example, given a known set of frequencies, the material of the housing of the integrated ultrasonic input device can be selected to maintain the highest possible energy measurement of the reflected ultrasonic signal from the initial transmission of the ultrasonic input device. As another example, given a known stack or known material (e.g., a specific display or a specific type of wood from a consumer product manufacturer), the ultrasonic input device can be set to operate at the frequency that provides the highest possible energy measurement. In some cases, the ultrasonic input device can automatically detect the best frequency to use based on measuring multiple frequencies that are close in time to each other.

[0190] C. Separate transmission and reception

[0191] Fig.18 1 is a schematic diagram depicting a circuit 1800 for receiving and transmitting signals through an ultrasonic transducer, the circuit being in a transmit state. The circuit 1800 drives the ultrasonic transducer to transmit and receive signals, and therefore requires a high voltage switch circuit to separate the high voltage transmitter from the low voltage receiver. When transmitting, the high voltage switch allows the high voltage transmitter circuit to drive the transducer while isolating the low voltage receiver. To move to the receive state, the switch must isolate the high voltage transmitter circuit and couple the transducer to the low voltage receive circuit.

[0192] Fig.19 It is a depiction Fig.18 Schematic diagram of a circuit 1800 for receiving and transmitting signals through an ultrasonic transducer, the circuit being in a receive state. When in the receive state, the high voltage switch isolates the high voltage transmitter circuit and couples the transducer to the low voltage receiving circuit. However, the high voltage switch typically has a large capacitance, which inherently attenuates the signal received at the transducer as it is conducted to the low voltage receiver. Thus, for example, an input voltage of 0.37 millivolts (370 microvolts) can be attenuated to less than 2 microvolts. This parasitic effect can dramatically reduce the available signal, thereby reducing the overall signal-to-noise ratio.

[0193] Fig. 20 is a schematic diagram illustrating isolation circuits 2000, 2002 for receiving and transmitting signals through an ultrasonic transducer according to certain aspects of the present disclosure. Figure 18 to Figure 19 The circuit 1800 is different, Fig. 20 The circuits 2000, 2002 eliminate the need for a high voltage switch. Thus, the circuits 2000, 2002 can provide efficient driving of a transducer configured as a transmitting transducer while also providing efficient reception of a transducer configured as a receiving transducer. The circuit 2000 includes a high voltage transmitter circuit that directly drives a transducer configured as a transmitting transducer. The circuit 2002 includes a low voltage receiver circuit that directly receives a signal from a transducer configured as a receiving transducer.

[0194] By separating the transmit and receive transducers, signal integrity can be improved, size can be reduced, and overall cost can be reduced. For example, by reducing or eliminating parasitic effects from electrical components (e.g., high voltage switches) embedded between the transducer and its low voltage receiver circuit, signal integrity can be improved, and power consumption can be improved. Since high voltage devices (e.g., high voltage switches) tend to be larger in size, the overall chip size can also be reduced. Therefore, by eliminating these switches, and optionally eliminating some high voltage transmitter circuits, the overall chip size and cost can be reduced.

[0195] D. Beamforming

[0196] Fig.21 is a set of schematic side views 2100 , 2102 , 2104 , 2106 illustrating beamforming achieved using an ultrasound transducer in accordance with certain aspects of the present disclosure.

[0197] Diagram 2100 depicts the beam pattern of a single ultrasonic transducer, such as a standard piezoelectric transducer. The beam is wide and fixed by the sensor size and sensor topology. There is no ability to adjust the beam for the transducer of Figure 2100.

[0198] Diagram 2102 depicts a focused beam achieved by activating a specific set of transducers. Using beamforming techniques, the activated transducers can focus the beam to a specific distance, which can improve the pressure sensitivity and accuracy of the ultrasonic sensor. For example, the focused beam can be used to provide fine point accuracy of touch events, as well as fine point accuracy for detecting other information associated with touch events, such as the ridges and valleys of a user's fingerprint.

[0199] Diagram 2104 depicts a wide beam achieved by activating a particular set of transducers. Using beamforming techniques, the activated transducers can focus the beam to a certain close distance to allow the beam to reach a certain point and expand again before reaching the target distance. Such a wide beam can improve the overall coverage of the sensor and can be used to obtain more average measurements over a larger area. Such a wide beam can be used to reduce target position sensitivity, which is advantageous in situations where a certain degree of variability is expected or desired, such as providing a large touch sensitive area and / or additional touch sensitive area on or around a button.

[0200] As depicted in graphs 2104 and 2106, the beam can be adjusted as needed and a tradeoff can be made between a more focused transmission pressure on the target and a larger effective area with lower target sensitivity.

[0201] Diagram 2106 depicts a multi-receiver configuration of activated transducers. In this configuration, a group of transmitting transducers can emit an ultrasonic signal that can be reflected and received at two or more groups of receiving transducers. For example, a first group of receiving transducers (e.g., one or more transducers) can be positioned to receive ultrasonic signals that have been reflected within a first zone, and a second group of receiving transducers can be positioned to receive ultrasonic signals that have been reflected within a second zone. As depicted in diagram 2106, the first zone can be smaller and enclosed within the second zone.

[0202] By performing beamforming using an array of ultrasound transducers, the energy may be limited to a particular region of interest, and thus the ultrasound transducer may be less sensitive to areas outside the region of interest.

[0203] IV. Micromachined Ultrasonic Transducers for Touch Input

[0204] In some cases, the ultrasonic input device may include an ultrasonic sensor that includes a micromachined ultrasonic transducer (MUT), such as a piezoelectric micromachined ultrasonic transducer (pMUT) or a capacitive micromachined ultrasonic transducer (cMUT). Other types of transducers besides pMUT and cMUT may include integrated (i.e., directly fabricated on CMOS) and non-integrated (i.e., fabricated separately and subsequently assembled with an on-board CMOS chip or directly communicated with a microprocessor / microcontroller or a field programmable gate array (FPGA) or any hardware with inter-integrated circuit (I2C) or serial peripheral interface (SPI) communication capabilities) bulk piezoelectric transducers. As described herein, a micromachined ultrasonic transducer for touch input may allow for an improved energy sensing area. In addition, the MUT may also reduce the overall power consumption of the ultrasonic input device.

[0205] A. Improved Energy Sensing Area

[0206] Fig. 22 2204, which depicts the operating modes of a micromachined ultrasonic transducer according to certain aspects of the present disclosure compared to a standard bulk transducer, which is depicted as average displacement for different frequencies. The graphs 2202, 2204 include lines depicting the average displacement over a frequency range of 0.5 MHz to 5 MHz, and axisymmetric cross-sectional visualization illustrations of the transducer modes.

[0207] Graph 2202 depicts the operating modes of a standard bulk transducer (e.g., a standard piezoelectric transducer) operating from 0.5 MHz to 5 MHz. Over the course of this relatively small frequency range, the number of peaks in the average displacement and the overall range of each of these peaks significantly depends on the various combinations of bulk modes, shear modes, bending modes, surface acoustic modes, and other modes experienced by the bulk transducer. As a result, shear waves and surface acoustic waves along different directions may be generated in addition to the normal longitudinal waves of interest. Therefore, a sensor utilizing such a bulk transducer may have uncontrollable beam patterns, unwanted crosstalk, more multipath reflections from different modes at different angles, parasitic modes and notches in the spectrum, a less clean received signal, more energy wasted on undesired modes, and other such problems.

[0208] In contrast, graph 2204 depicts a uniform and predictable bending mode shape present in a MUT (e.g., pMUT) over the same frequency span, and which serves to launch longitudinal acoustic waves in a normal direction toward the outer surface of the stack. As a result, the MUT is able to implement much improved performance over a standard bulk transducer.

[0209] Due to the nature of ultrasonic input devices, it is desirable to detect ultrasonic reflections based on longitudinal acoustic waves (e.g., propagating in a direction orthogonal to the sensor). Where MUTs are used as ultrasonic transducers for touch input, MUTs perform particularly well due to their inherent ability to perform bending mode displacements to generate such longitudinal acoustic waves without inadvertently generating many (if any) transverse or other undesirable waves. Thus, MUTs can be used for beamforming operations, such as those described herein, can be tightly packaged into sensor arrays, can be used with fewer filtering devices, and can achieve higher signal-to-noise ratios using the same or less power than would be the case with standard bulk piezoelectric transducers.

[0210] Fig.23 2302, 2304, 2306, which depict modes of operation of a standard body transducer for ultrasonic touch detection. When a standard body transducer is used for ultrasonic touch detection, driving the transducer to transmit a signal may cause the transducer to shift in multiple operating modes, which may cause erroneous signals to be transmitted into a receiving medium (e.g., a stack).

[0211] Figure 2302 depicts a longitudinal mode of operation, in which the drive of the transducer initiates a longitudinal signal in a direction orthogonal to the sensor. However, the same or similar drive of the transducer in Figure 2302 may cause a lateral displacement as depicted in Figure 2304. Such a lateral displacement (e.g., due to the lateral mode of operation) may initiate a lateral signal that is carried into the receiving medium in a direction different from the normal of the sensor, or may result in an undesirable normally traveling shear wave. As a result, the driven body transducer may generate a signal as depicted in Figure 2306, in which both normal and abnormal signals propagate from the body transducer. Since the sensing region (e.g., the region desired to be sensed) is typically located directly above the stack, the abnormal signal may cause interference with the signal received from the sensing region. In addition, the body transducer may be susceptible to the physical topology of the stacked region near the sensing region, because different topologies may initiate different reflections of the abnormal signal, which may result in false positives or false negatives.

[0212] Fig.24 2404, which depicts lateral signal rejection of a micromachined ultrasonic transducer according to certain aspects of the present disclosure. FIG. 2402 is a close-up view of a single transducer of a MUT array. The transducer can be composed of multiple layers, including a piezoelectric layer that, when excited, can initiate a bending displacement to cause a longitudinal wave to be emitted in a direction orthogonal to the sensor (e.g., a direction orthogonal to the MUT surface).

[0213] Figure 2404 depicts an ultrasound input device using a sensor with a MUT. The ultrasound input device is depicted as being coupled to an aluminum layer and a glass layer, but any other stacking configuration may be used. The nature of the MUT may allow ultrasound signals to be transmitted in a direction orthogonal to the sensor while minimizing or eliminating any signals that would otherwise propagate in a direction that is not orthogonal or substantially not orthogonal to the sensor if a body transducer were used. Thus, using a MUT as a transducer in an ultrasound input device may help focus energy into a desired sensing region and reduce sensitivity to false positives or false negatives due to false reflections.

[0214] B. Driving convenience

[0215] In addition to the above benefits of the MUT, when used with an ultrasound input device, the MUT can also reduce the overall power consumption of the ultrasound input device. Since the power required to drive a transducer is proportional to its capacitance multiplied by the square of its voltage, the low capacitance level of the MUT array (e.g., on the order of picofarads) results in much lower power consumption than the relatively higher capacitance level (e.g., on the order of nanofarads, which is three orders of magnitude greater than picofarads) of an equivalent standard bulk transducer.

[0216] V. Ultrasonic Signal Processing

[0217] The reflected ultrasound signals may be processed to produce an image and determine the range to the object.The embodiments described herein may process the reflected ultrasound signals to determine whether the object is in contact with a surface.

[0218] A. Detecting touch input by digitizing reflected signals

[0219] Fig.25 2500 is a schematic flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device according to certain aspects of the present disclosure. The flow chart 2500 includes transmitting and receiving ultrasonic signals as shown in a first graph 2502. The first graph 2502 shows analog measurements of a first signal 2503 of a transmitted ultrasonic signal associated with the ultrasonic input device and a set of subsequent signals 2504A, 2504B, 2504C, 2504D, 2504E of a set of reflected ultrasonic signals associated with the ultrasonic input device. The first signal 2503 and the subsequent signals 2504 can be measured using a high speed ADC 2506 to digitize the signals.

[0220] The output of the high speed ADC 2506 is shown in a second graph 2508. The second graph 2508 includes a first digital representation 2510 of the transmitted ultrasonic signal and subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E of the reflected ultrasonic signal associated with the ultrasonic input device. The first digital representation 2510 and the subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E may be processed by a digital processing module 2514 embedded in the ultrasonic input device and / or in a system coupled to the ultrasonic input device. The digital processing module 2514 may demodulate the digital representation of the data to extract the touch input information. For example, the digital processing module may process one or more of the subsequent digital representations 2512A, 2512B, 2512C, 2512D, 2512E to determine that the amplitude of the second digital representation is below a threshold value associated with an object contacting the surface of the ultrasonic input device.

[0221] B. Using energy integration to detect touch input

[0222] Fig.262600 is a schematic flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to certain aspects of the present disclosure. The flow chart 2600 includes transmitting and receiving ultrasonic signals, as shown in a first graph 2602. The first graph 2602 shows an analog measurement of a transmitted ultrasonic signal 2603 and a set of subsequent signals 2604A, 2604B, 2604C, 2604D, 2604E for a set of reflected ultrasonic signals associated with the ultrasonic input device. The flow chart 2600 may include an ultrasonic input device having an analog circuit that includes a rectifier 2606 to rectify the subsequent signals 2604A, 2604B, 2604C, 2604D, 2604E.

[0223] The second graph 2608 shows the first signal 2603 and a set of rectified signals 2610A, 2610B, 2610C, 2610D, 2610E, each rectified signal corresponding to a corresponding reflected ultrasonic signal in the set of reflected ultrasonic signals. The rectified signals 2610A, 2610B, 2610C, 2610D, 2610E can be processed by an analog integrator 2612 to output a direct current (DC) signal 2613, as shown in a third graph 2614, which is proportional to the amplitude of the reflected ultrasonic signal. The DC signal 2613 can be determined using an energy measurement window 2616. The DC signal 2613 can represent an energy value associated with the energy of the received signal measured during the energy measurement window 2616. The DC signal 2613 can be processed by a low-speed ADC 2618. The DC signal 2613 output by the rectifier 2606 and the integrator 2612 removes the need to generate a high frequency digital output, and as a result, the low speed ADC can use less power and can be manufactured on a smaller chip area.

[0224] Fig. 27 2700 is a schematic example flow chart for processing ultrasonic signals transmitted and received by an ultrasonic input device using energy integration according to certain aspects of the present disclosure. The flow chart 2700 includes transmitting and receiving ultrasonic signals as shown in a first graph 2702. The first graph 2702 shows an analog measurement of a first signal 2703 for the transmitted ultrasonic signal, and an analog measurement of a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E for a set of reflected ultrasonic signals associated with the ultrasonic input device. The flow chart 2700 may include an ultrasonic input device having an analog summing or integrating circuit 2720 and a summed voltage output 2722.

[0225] The second graph 2708 shows the first signal 2703 and a set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E, which correspond to the energy of the corresponding reflected ultrasound signals in the set of reflected ultrasound signals, respectively. For illustration purposes, the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E are depicted in solid lines, which overlap with the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E from the first graph 2702 shown in dotted lines.

[0226] The summing or integrating circuit 2720 may receive the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E from within the energy measurement window 2716. The summing or integrating circuit 2720 may generate a voltage output 2722 which is an analog value representing the summed / integrated energy within the energy measurement window 2716.

[0227] In some cases, an optional negative DC charging circuit 2724 may be applied to the summing or integrating circuit 2720 to offset information not associated with a touch event. Since a touch event is identified based on the difference between a signal received during a non-contact state and a signal received during a contact state, there is some information (e.g., a baseline signal) within the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E that is not associated with these differences. Removing this baseline signal can result in a more efficient sampling range during the analog-to-digital conversion. Since removing this analog baseline signal from the set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E would require precise phase alignment, it is difficult to apply such a correction. However, as Fig. 27 As depicted, an optional negative DC charging circuit 2724 applied to the summing or integrating circuit 2720 can offset a certain amount of energy associated with the baseline signal or a portion thereof, thereby increasing the amount of effective range available for analog-to-digital conversion. In this case, the voltage output 2722 can be proportional to the energy of the signal minus the energy of the negative DC charging circuit 2724.

[0228] The voltage output 2722 may be processed by the low speed ADC 2718. The voltage output 2722 of the summed / integrated energy within the energy measurement window 2716 may eliminate the need to generate a high frequency digital output, and as a result, the low speed ADC may use less power and may be fabricated on a smaller chip area.

[0229] Fig.282800 is a schematic flowchart for processing ultrasound signals transmitted and received by an ultrasound input device using energy integration by absolute value accumulation according to certain aspects of the present disclosure. Flowchart 2800 may be a flowchart for implementing Fig. 27 The flowchart 2700 of the present invention is a technique for transmitting and receiving an ultrasonic signal, as shown in a first graph 2802. The first graph 2802 shows a simulated measurement of a first signal of a transmitted ultrasonic signal and a set of subsequent signals of reflected ultrasonic signals associated with an ultrasonic input device. The first graph 2802 can depict a voltage (e.g., V(t)) as a function of time. The first graph 2802 can be Fig. 27 The first graph 2702 of . The flow chart 2800 may include an ultrasound input device having an analog sampling circuit 2806, an absolute value circuit 2814, an analog accumulator 2824, and a summed voltage output 2828.

[0230] The set of subsequent signals from the first graph 2802 may be passed through an analog sampling circuit 2806 to produce a sampled first signal 2810 and a set of sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E as depicted in a second graph 2808. The first signal may correspond to the initially emitted ultrasonic wave. The second graph 2808 may depict a voltage as a function of samples (e.g., V(n), where n is the number of samples). The sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E may be passed to an absolute value circuit 2814, which may generate a set of energy signals 2820A, 2820B, 2820C, 2820D, 2820E as depicted in a third graph 2816. The third graph 2816 may depict an absolute value of a voltage as a function of samples (e.g., |V(n)|). The absolute value circuit 2814 can pass all zero or positive values ​​of the set of sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E and invert the polarity of all negative values. A sampled first signal 2818 is also shown in the third graph 2816, and the sampled first signal 2818 can be similar to the sampled first signal 2810.

[0231] A switched capacitor analog accumulator 2824 may be used to sum the set of energy signals 2820A, 2820B, 2820C, 2820D, 2820E from within the energy measurement window 2822. The switched capacitor analog accumulator may generate a voltage output 2828, which is an analog value representing the sum of the energy within the energy measurement window 2822. In some cases, an analog integrator may be used instead of an accumulator.

[0232] In some cases, an optional negative clocked DC charging circuit 2826 may be applied to the switched capacitor analog accumulator 2824 to offset information not associated with a touch event. Since the sampling circuit 2806 is clocked according to the sampling rate, the optional negative clocked DC charging circuit 2826 may be clocked at the same rate to ensure that the bias voltage is applied at the appropriate intervals corresponding to the samples of the sampled subsequent signals 2812A, 2812B, 2812C, 2812D, 2812E. When the optional negative clocked DC charging circuit 2826 is used, the voltage output 2828 may be proportional to the energy of the signal minus the energy of the negative clocked DC charging circuit 2826.

[0233] The voltage output 2828 may be processed by a low speed ADC 2830. The voltage output 2828 of the summed energy within the energy measurement window 2822 may remove the need to generate a high frequency digital output, and as a result, the low speed ADC may use lower power and may be fabricated on a smaller chip area.

[0234] Fig.29 2900 is a schematic flowchart of processing ultrasound signals transmitted and received by an ultrasound input device using energy integration through self-mixing and integration according to certain aspects of the present disclosure. Flowchart 2900 may be used to implement Fig. 27 The flowchart 2700 of the present invention is a technique for transmitting and receiving an ultrasonic signal, as shown in a first graph 2902. The first graph 2902 shows a simulated measurement of a first signal for transmitting an ultrasonic signal and a set of subsequent signals for a set of reflected ultrasonic signals associated with an ultrasonic input device. The first graph 2802 can depict a voltage (e.g., V(t)) as a function of time. The first graph 2902 can be Fig. 27 The first graph 2702 of . The flow chart 2900 may include an ultrasound input device having a self-mixing circuit 2906, an analog integrator circuit 2920, and an integrated voltage output 2922.

[0235] The set of subsequent signals from the first graph 2902 may be passed through a self-mixing circuit 2906 to generate a set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E, as depicted in a second graph 2908. The self-mixing circuit 2906 may effectively multiply each analog value by itself over time. As a result, the second graph 2908 may depict a squared voltage (e.g., V 2 (t)). Due to the nature of squaring and therefore the nature of the self-mixing circuit 2906, the set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E will always be positive.

[0236] The set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E may be passed to an analog integrator circuit 2920. The analog integrator circuit 2920 may integrate the set of squared subsequent signals 2910A, 2910B, 2910C, 2910D, 2910E within the energy measurement window 2916 to generate an integrated voltage output 2922. The integrated voltage output 2922 may be an analog representation of the total energy over time within the energy measurement window 2916. In some cases, an accumulator may be used instead of the analog integrator circuit 2920.

[0237] In some cases, an optional negative bias current circuit 2924 can be applied to the analog integrator circuit 2920 to offset information not associated with a touch event. The negative bias current circuit 2924 can constantly drain charge out of the analog integrator circuit 2920 during integration. When the optional negative bias current circuit 2924 is used, the voltage output 2922 can be proportional to the energy of the signal minus the energy of the negative bias current circuit 2924.

[0238] The voltage output 2922 may be processed by a low speed ADC 2926. The voltage output 2922 of the integrated energy within the energy measurement window 2916 may remove the need to generate a high frequency digital output, and as a result, the low speed ADC may use lower power and may be fabricated on a smaller chip area.

[0239] Fig.30 is a schematic circuit diagram depicting an analog integrator 3000 with a negative bias current according to certain aspects of the present disclosure. The negative bias of the analog integrator 3000 may be Fig.29 Analog integrator circuit 2920 and optional negative bias current circuit 2924.

[0240] The analog integrator 3000 can be connected through a resistor (R in ) receives the input voltage (V in ) to obtain the input current (I in The capacitor (C) can be charged by the charging current (I f ) is charged to generate an integrated signal that can feed the voltage output (V out ). Item (A) is an operational amplifier. A negative bias current (I bias ) to discharge the charge out of the analog integrator 3000, thereby causing a reduced charging current (I f ). Therefore, the charging current can be calculated as I f =I in -I bias .

[0241] C. Energy measurement window

[0242] Fig.31 is a schematic flow chart for processing an ultrasonic signal according to certain aspects of the present disclosure, which depicts the reduced effect of flight time variations on touch input detection within an energy measurement window. In an ultrasonic imaging system or a proximity detection system, accurate flight time is critical to determining the distance of an object in the field of view from an ultrasonic transducer. In contrast to imaging and proximity systems, distances to first and second surfaces of a material layer in an ultrasonic input device can be provided, and touch input can be detected without considering flight time variations. Fig.31 A first graph 3102 is shown in which a first set of reflected ultrasonic signals 3104 are received beginning at a first time 3106, and a second graph 3108 is shown in which a second set of reflected ultrasonic signals 3110 are received at a second time 3112. The first signal 3103 may be associated with an emitted ultrasonic signal that occurs before the first time 3106 and the second time 3112 of the first graph 3102 and the second graph 3108, respectively. The first set of reflected ultrasonic signals 3104 are passed through an energy accumulator or integrator circuit 3120 to generate an output voltage 3122 (e.g., V sum1 ), which can be fed into a low speed ADC 3124 and processed to obtain an output value 3118 (e.g., 3000 LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3110 pass through an energy accumulator or integrator circuit 3120 to generate an output voltage 3123 (e.g., V sum2 ), the output voltage can be fed to the low-speed ADC 3124 and processed to obtain an output value 3119 (e.g., 3000LSB, where LSB represents the least significant bit). The output values ​​3118, 3119 can represent the pulse reflection energy during the energy measurement window 3116 of the graphs 3102, 3108. Although the start time of the first group of reflected ultrasonic signals 3104 and the second group of reflected ultrasonic signals 3110 is different (e.g., the first time 3106 and the second time 3112), the output values ​​3118, 3119 can be the same or substantially the same because the entire first group of reflected ultrasonic signals 3104 and the entire second group of reflected ultrasonic signals 3110 are each fitted within the energy measurement window 3116.

[0243] Therefore, the ultrasonic input device can be insensitive to flight time at least to a certain extent (e.g., within the energy measurement window). In some cases, advanced windowing techniques such as those disclosed herein can further improve the insensitivity of the ultrasonic input device to flight time. As a result, the surface of the ultrasonic input device (e.g., a material layer) does not need to be completely flat and / or the alignment of the ultrasonic input device relative to the material (e.g., a material layer) does not need to be exactly 90° (e.g., the angle between the propagation direction of the ultrasonic transducer and the surface of the material layer). In addition, insensitivity to flight time can allow a certain degree of insensitivity to the changing refractive index (e.g., a material layer with a slightly inconsistent refractive index as a whole) through which the ultrasonic signal passes.

[0244] For example, Figure 26 to Figure 29 and Fig.31 As shown, the energy of the reflected ultrasonic signal (e.g., reflected echo and standing wave) is summed or integrated over the energy measurement window. The energy is related to the condition of the touch input and can therefore be used for input touch detection. The size of the energy measurement window 3116 can be determined to include the pulse time of the ultrasonic signal and take into account changes in flight time due to temperature, stack changes (e.g., changes in the materials constituting the ultrasonic input device), etc. The energy measurement window 3116 can reduce errors caused by changes in flight time. The ultrasonic touch device can determine input touch contact based on a specific threshold.

[0245] Fig.32 is a schematic simplified flow chart for processing an ultrasound signal according to certain aspects of the present disclosure, which illustrates the effect of reflected ultrasound signal flight time variation on the enhancement of touch input detection outside the energy measurement window. Fig.32 A first graph 3202 is shown, where a first set of reflected ultrasonic signals 3204 are received starting at a first time 3206, and a second graph 3208 is shown, where a second set of reflected ultrasonic signals 3210 are received at a second time 3212. The first signal 3203 may be associated with an ultrasonic signal transmitted, which occurs before the first time 3206 and the second time 3212 of the first graph 3202 and the second graph 3208, respectively. The first set of reflected ultrasonic signals 3204 may be processed as disclosed herein to obtain output values ​​3218 (e.g., 3000 LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3210 may be processed as disclosed herein to obtain output values ​​3219 (e.g., 2500 LSB, where LSB represents the least significant bit). The output values ​​3218, 3219 may represent the pulse reflection energy during the energy measurement window 3216 of the graphs 3202, 3208.

[0246] like Fig.32As shown, because almost all of the first group of reflected ultrasonic signals 3204 fit within the energy measurement window 3216, but a smaller portion of the second group of reflected ultrasonic signals 3210 fits within the energy measurement window 3216, the output value 3218 is greater than the output value 3219. Fig.32 As depicted in FIG. 32, the output values ​​3218, 3219 differ by 500 LSB. If the reflected ultrasound signal falls outside the energy measurement window 3216, some of the measured pulses may be cut off and cannot be measured, and thus the ultrasound input device may be susceptible to flight time variations (e.g., variations that would cause a difference between the first time 3206 and the second time 3212).

[0247] Fig.33 is a schematic flow chart for processing ultrasound signals according to certain aspects of the present disclosure, which depicts the minimal impact of reflected ultrasound signal flight time variations on touch input detection outside the energy measurement window when window shaping is used. Fig.33 A first graph 3302 is shown, where a first set of reflected ultrasonic signals 3304 are received starting at a first time 3306, and a second graph 3308 is shown, where a second set of reflected ultrasonic signals 3310 are received at a second time 3312. The first signal 3303 may be associated with an ultrasonic signal transmitted, which occurs before the first time 3306 and the second time 3312 of the first graph 3302 and the second graph 3308, respectively. The first set of reflected ultrasonic signals 3304 may be processed as disclosed herein to obtain output values ​​3318 (e.g., 2500LSB, where LSB represents the least significant bit). The second set of reflected ultrasonic signals 3310 may be processed as disclosed herein to obtain output values ​​3319 (e.g., 2450LSB, where LSB represents the least significant bit). The output values ​​3318, 3319 may represent the pulse reflection energy during the energy measurement window 3316 of the graphs 3302, 3308.

[0248] and Fig.32 Differently, the energy measurement window envelope 3320 is used in conjunction with the energy measurement window 3316. The energy measurement window envelope 3320 scales portions of the signal within the energy measurement window 3316 so that portions near the edges of the energy measurement window 3316 have less weight than portions near the center of the energy measurement window 3316. Thus, despite small changes near the ends of the energy measurement window 3316, the resulting output value will be primarily based on the signal measured within the center of the energy measurement window 3316. The energy measurement window envelope 3320 is used in conjunction with the energy measurement window 3316. Fig.33 As depicted in FIG. 1 as having a particular bell shape, although any suitable shape may be used, including symmetrical and asymmetrical shapes. Fig.33The vertical extension of the energy measurement window envelope 3320 depicted in can represent any suitable scaling, such as 0% to 100%. In some cases, the energy measurement window envelope 3320 can include amplifying the signal near the center of the energy measurement window 3316 to a value such as above 100% of the original signal at that time.

[0249] As in Fig.33 As depicted in FIG, due to the use of the energy measurement window envelope 3320, the signals (e.g., the first set of reflected ultrasonic signals 3304 and the second set of reflected ultrasonic signals 3310) are weighted so that portions of the signal closest to the center of the energy measurement window 3316 are given greater weight than portions closest to the edge of the energy measurement window 3316, thereby de-emphasizing any portions cut off by the start or end of the energy measurement window 3316. As a result, the output values ​​3318 and 3319 are more Fig.32 The output values ​​3218 and 3219 are closer. Fig.33 As depicted in , the output values ​​3318, 3319 differ by only 50 LSB. Thus, as a result of the energy measurement window envelope 3320, the ultrasound input device may become less susceptible to flight time variations.

[0250] Fig.34 3400 is a schematic circuit diagram illustrating a window shaping circuit 3400 according to certain aspects of the present disclosure. The window shaping circuit 3400 may generate an energy measurement window having an energy measurement window envelope (e.g., having Fig.33 3404. The energy measurement window envelope 3320 of the energy measurement window 3316 is shown in FIG. 3405. With the addition of the adjustable capacitor 3402, the window shaping circuit 3400 can operate as a conventional analog accumulator circuit. The adjustable capacitor 3402 can take any suitable form, such as a switched ladder of capacitors of different sizes. The selection of capacitor size for the adjustable capacitor 3402 can cause the gain on the analog accumulator circuit to adjust over time. In some cases, the adjustable capacitor 3402 can be driven by a clock 3404 or other source to determine when to change the capacitance. In some cases, the adjustable capacitor 3402 can be used with an analog sampling circuit such as Fig.28 The analog sampling circuit 2806 of FIG. 28 is used together, and the adjustable capacitor 3402 can change with different sample numbers (e.g., n of V(n)).

[0251] Fig.35 is a schematic diagram illustrating a process 3500 for processing an ultrasonic signal to detect a touch input using the amplitude of a reflected ultrasonic signal according to certain aspects of the present disclosure. Fig.35The ultrasonic input device 3502 is shown without a touch input 3504 and with a touch input 3506. A first graph 3508 associated with the ultrasonic input device 3502 without a touch input 3504 shows a transmission signal 3510 and a first set of reflected signals 3512. The first set of reflected signals 3512 may be processed to generate an output voltage 3530 (e.g., V sum1 ), which output voltage can be provided to the low speed ADC 3534 and further processed to generate a first output 3536. The first output 3536 can represent the energy of the first set of reflected signals 3512 within the energy measurement window envelope 3516. The second graph 3520 shows the transmission signal 3522 and the second set of reflected signals 3524. As disclosed herein, the second set of reflected signals 3524 can be processed to generate an output voltage 3532 (e.g., V sum2 ), the output voltage can be provided to the low speed ADC 3534 and further processed to generate a second output 3538. The second output 3538 can represent the energy of the second group of reflected signals 3524 within the energy measurement window envelope 3516.

[0252] Energy measurement window envelope 3516 (e.g., similar to Fig.33 The energy measurement window envelope 3320 of the embodiment of the present invention can be applied to the first group of reflection signals 3512 and the second group of reflection signals 3524. In some embodiments, the energy measurement window envelope 3516 can be applied to the first group of reflection signals 3512 and the second group of reflection signals 3524 to weaken the signal at the edge of the energy measurement window envelope 3516.

[0253] The first output 3536 and the second output 3538 may be compared to determine whether a touch input (e.g., a touch event) has occurred. For example, if the second output 3538 is lower than the first output 3536 by a predetermined amount and / or if the second output 3538 is below a threshold, the ultrasonic input device 3502 may generate a signal indicating that a touch input is present on the surface. Since the output voltages 3530, 3532 indicate the first output 3536 and the second output 3520, respectively, the output voltages 3530, 3532 may be used to determine whether a touch input has occurred. In some embodiments, only a single output, such as the first output 3518, may be compared to a reference value. The reference value may be established at the time of manufacture and / or updated based on background characteristics (such as temperature) measured by or transmitted to the device.

[0254] Reference Fig.35The described techniques may be used to generate output signals from the ultrasound input device 3502, although other techniques may be used. Any technique that can convert a signal associated with the first set of reflection signals 3512 or the second set of reflection signals 3524 into a measure of total energy may be used.

[0255] Fig.36 3600 is a graph depicting a simplified example energy signal 3614 according to certain aspects of the present disclosure. When the ultrasound input system processes an input ultrasound signal received by an ultrasound transducer, the ultrasound input system may convert the ultrasound signal into an energy signal 3614. The energy signal 3614 may represent the overall energy associated with the input ultrasound signal. For example, as shown in Fig.35 As depicted in , the signals depicted in graphs 3508 and 3520 can be converted into outputs 3536 and 3538. These outputs can be recorded, plotted, or output as energy signal 3614 over time. Output 3536 associated with no touch events can generally be viewed as an area of ​​overall higher energy signal, while output 3538 associated with touch events can generally be viewed as an area of ​​overall lower energy signal. It should be understood that the continuous output of processed ultrasonic signals can be used to generate an energy signal, which can then be used to determine whether a touch event has occurred at a certain point in time. Fig.36 The energy signal 3614 is simplified for illustration purposes only.

[0256] D. Touch input error prevention

[0257] Fig.37 3700 is a diagram according to certain aspects of the present disclosure, which depicts reflected ultrasonic signal measurements made using an ultrasonic input device and illustrates techniques for improving touch input detection. A sensor readout (e.g., a DC signal or other sensor data) determined by the ultrasonic input device can be measured continuously or at a specific frequency according to the present application. In some embodiments, the sensor readout can be measured at a frequency of 100 Hz. A single measurement 3702 can correspond to an energy measurement within an energy measurement window. One or more single measurements can be used to determine a current state 3706. The current state can be defined by the current individual measurement 3702 or by a best fit line based on two or more single measurements. In some embodiments, the best fit line can be calculated using a least squares method. A moving average threshold 3704 can be determined using multiple single measurements.

[0258] The current state 3706 and the moving average threshold 3704 can be used to detect a touch event. The moving average threshold 3704 can be used to determine a sudden signal drop that can trigger a touch input event. For example, the system can detect a "hand touch" effect only when a "fast signal change" 3708 from the current state 3706 is detected. The fast signal change 3708 can be associated with a sudden signal drop on all or many channels and can be considered a touch input event. The threshold for detecting the fast signal change 3708 can be the moving average threshold 3704 (dynamic threshold) when no hand touch event is detected. In some embodiments, the fast signal change 3708 can be a pre-programmed static threshold. The fast signal change 3708 event can trigger a touch input event and cause the ultrasonic input device to generate a signal indicating a touch input on the surface of the device. For the fast signal change 3708 event, multiple measurements 3710 are taken to ensure that the signal does actually drop and does not jump back up, such as jumping back to its original value. For example, a heavy press by the user may result in a sensor readout that drops, but a continuous signal will still be provided. During multiple measurements 3710, if the signal quickly returns to a higher value (such as a value previously seen before a suspected touch event), the ultrasonic input device may recognize the temporary signal drop as a false touch event and not classify it as a touch event. Multiple measurements 3710 may occur within a very short time frame (e.g., on the order of tens or hundreds of milliseconds). In some embodiments, a "gradual signal change" may be considered a temperature change rather than a hand touch event, since the moving average will be adjusted with each individual measurement 3702 at a rate based on the number of measurements used to determine the moving average.

[0259] In some cases, the threshold 3704 can be based on a calculation other than a moving average calculation. In some cases, the threshold 3704 is simply a function of past history (e.g., historical measurements), such as a function of the past x measurements. In some cases, past measurements can be weighted, such as a recent measurement having a greater weight than a measurement taken longer ago. In this case, the response time of the ultrasonic input device can be adjusted based on the weight of the past x measurements. For example, the threshold can be calculated as a function of historical values ​​according to Threshold = f(X[n-1], X[n-2], ..., X[nm]) (where X[n] is the nth sensor reading (or current sensor reading)). In another example, the threshold can be calculated according to Threshold = w1X[n-1] + w2X[n-2], ..., w m X[nm](where w n is a weight parameter for the nth sensor reading) is calculated as a function of the weighted historical values. In some cases, the weight parameters can be trained using machine learning, such as described in more detail herein.

[0260] In some cases, in addition to or instead of determining rapid signal changes 3708 based on the measurement itself, the determination may be made using the slope of a set of measurements, such as the slopes of a current measurement and a number of past measurements.

[0261] Fig.38 is a diagram 3800 that depicts reflected ultrasonic signal measurements using an ultrasonic input device and illustrates additional techniques for improving touch input detection in accordance with certain aspects of the present disclosure. A portion of the diagram 3800 is depicted as Fig.37 Graph 3700 of FIG. Graph 3800 shows that the signal may change over time due to various factors (such as temperature changes), yet the ultrasonic input device may be able to discern that these changes are not touch events. However, a sudden signal drop between consecutive measurements may indicate a touch event. The current state 3806 may be similar to Fig.37 The current state 3706 of . The moving average threshold 3804 is similar to Fig.37 The threshold 3804 is based in part on a moving average of previous measurements of the current state 3806, such as a moving average of previous measurements offset by a given amount. This type of threshold 3804 may be referred to as a dynamic threshold, although other thresholding techniques may be used.

[0262] At region 3816, a touch event occurs. When a touch event occurs, the current state 3806 drops rapidly. As depicted in the labeled portion of chart 3800, various measurements 3802 are shown. Each measurement 3802 can be separated in time based on the measurement frequency. For example, each measurement 3802 can be separated by 0.01 seconds (e.g., at 100 Hz), although other frequencies can be used. A sudden drop can be detected between two or more consecutive measurements 3802. When the sudden drop in the current state 3806 drops below the threshold 3804, it can be considered that a touch event has occurred. Region 3817 depicts another touch event.

[0263] At regions 3818 and 3820, gradual changes in the temperature of the ultrasonic sensor and the temperature of the surface to which the sensor is coupled may cause gradual changes in the current state 3806. Due to the relatively slow changes in the current state 3806, the threshold 3804 based on the moving average of the current state 3806 will also change. Since the threshold 3804 can compensate for slow changes in the current state 3806, such as changes due to temperature changes, these slow changes in the current state 3806 will not exceed the threshold 3804, and therefore will not trigger a touch event. In addition, since the threshold 3804 is dynamically updated, the threshold 3804 can operate appropriately at different temperatures. In some cases, the changes in the current state 3806 caused by temperature changes can even be greater than the contrast caused by an actual hand touch, but since these temperature changes are much slower than the changes in the current state 3806 caused by a touch event, they are not detected as touch events.

[0264] VI. Multi-frequency touch detection

[0265] Fig.39 3904. The reflected ultrasonic signal received by the ultrasonic input device may include a main signal 3902 and any undesired signal 3904. The main signal travels through the material layer along a first path and is associated with a first time of flight (TOF), and any undesired signal 3904 travels through the material layer along a second path and is associated with a second TOF. The speed of sound in the material layer depends on the temperature of the material layer. Since the speed of sound changes as a result of temperature changes, the main signal 3902 and the undesired signal 3904 travel through different acoustic paths, and the associated first TOF and second TOF change by different amounts accordingly. This produces a net TOF difference Δt(T) 3906 between the main signal 3902 and the undesired signal 3904, which changes with temperature T. It is then converted into a phase delay difference Δφ(T) between the main signal 3902 and the undesired signal 3904. And thus produces a different integrated signal intensity difference Dout(T), as depicted by line 3910.

[0266] Fig.40 is a set of graphs depicting the TOF temperature dependence of two frequency methods for detecting touch input according to certain aspects of the present disclosure. These graphs may be similar to Fig.39In the multi-frequency ultrasonic input device, different frequencies will have different temperature effects, causing different TOFs for each signal. The multi-frequency ultrasonic input device can process a "finger touch" (e.g., a touch event) when a signal drop is detected in a threshold number of frequency channels. For example, two different methods can detect whether a finger touches the ultrasonic touch input device, and the device can only process a touch event when both methods agree that a finger touch has been detected.

[0267] In a multi-frequency ultrasonic touch input device, a first signal 4002 at a first frequency and a second signal 4004 at a second frequency have different background and temperature drift characteristics. For example, when the temperature changes, the first signal 4002 and the second signal 4004 experience the same Δt(T). Due to the different temperature drift characteristics, the same Δt(T) will be converted into a different phase delay for each frequency. For example, the first signal 4002 will have a first phase delay Δφ↓1(T)4006, and the second signal 4004 will have a second phase delay Δφ↓2(T)4008. The resulting phase delay difference can cause two different ADC output value patterns, which are at temperatures Dout↓1(T) and Dout↓2(T), respectively, as depicted by lines 4010 and 4012.

[0268] Therefore, the signal drop can be measured in multiple frequencies to improve touch detection reliability and reduce false trigger detection. If all frequency channels detect a sudden signal drop, the touch input event can be processed. Multiple measurements can occur very quickly (<1ms) to ensure that the sudden signal drop is not due to temperature effects.

[0269] The multi-frequency ultrasonic touch input device can avoid false triggering by reducing the noise associated with environmental conditions. The touch input device can immediately perform a fast pulse echo test to ensure that the touch event is real and not a false trigger caused by noise. In some embodiments, multiple tests can occur within 1ms.

[0270] Fig.41 41 is a multi-part diagram 4100 depicting reflected ultrasonic signal measurements across several frequencies using an ultrasonic input device and illustrating techniques for improving touch input detection according to certain aspects of the present disclosure. Different frequencies of the ultrasonic signal may exhibit different changes due to temperature changes. Therefore, by sensing using multiple ultrasonic frequencies, the ultrasonic input device may compare a suspected touch event with data from one or more other frequencies to ensure that the suspected touch event is confirmed by the one or more other frequencies. The use of multiple frequencies may reduce error rates.

[0271] Line 4106 may represent an energy signal associated with a frequency of 100 kHz, line 4105 may represent an energy signal associated with a frequency of 1 MHz, and line 4107 may represent an energy signal associated with a frequency of 10 MHz. Line 4104 may represent a moving average threshold, e.g. Fig.37 For purposes of illustration, the moving average threshold is depicted only for the 100 kHz frequency, but there may be a corresponding threshold for each frequency used (e.g., 1 MHz and 10 MHz). Fig.41 Frequencies of 100 kHz, 1 MHz, and 10 MHz are used, but any other suitable frequency may be used. Fig.41 Three different frequencies are used, but any number of different frequencies may be used, such as two or more than three. A touch event may be registered only if a touch event is detected on all, most, or at least a threshold percentage of the different frequencies used for detection.

[0272] In some cases, instead of or in addition to driving the ultrasound input device at different frequencies, the ultrasound input device can drive an ultrasound array with different phase delays to generate different beam patterns. Since different beam patterns can have different temperature characteristics, different beam patterns can be used similar to different frequencies to reduce errors and confirm suspected touch events.

[0273] Fig.42 4 is a schematic plan view according to certain aspects of the present disclosure, which depicts a dual-frequency PMUT 4200. In some embodiments, a circular PMUT design can be manufactured to realize a multi-frequency transducer. The circular PMUT design can be composed of multiple separate channels for transmission and reception per frequency. In some cases, multiple channels or transducers can be arranged concentrically. For example, the dual-frequency PMUT 4200 includes a first transmission / reception pair 4202 associated with a low frequency. The first transmission / reception pair 4202 may include a low-frequency transmission ring 4204 and a low-frequency receiving ring 4206. The dual-frequency PMUT 4200 also includes a second transmission / reception pair 4208 associated with a high frequency. The second transmission / reception pair 4208 may include a high-frequency transmission ring 4210 and a low-frequency receiving ring 4212. In various embodiments, the circular PMUT design may include a range of multiple frequencies from 2 to 10, and the frequency range may be from 1 MHz to 10 MHz. In some embodiments, a frequency less than 1 MHz may be used depending on the material layer and the specific application. A second PMUT array may be added for TOF measurements in the 1 MHz-3 MHz range. In some cases, the frequency range for any array may be from 30 kHz to 50 MHz.

[0274] Fig.43is a schematic plan view according to certain aspects of the present disclosure, which depicts a multi-frequency ultrasonic input device 4300 having a square design. The square sensor design can be composed of a square grid of multiple separate channels for transmission and reception per frequency. In some cases, one or more receive channels can be located between multiple transmit channels. In this case, the location of the receive channels between the multiple transmit channels can facilitate the reception and detection of reflected signals. In an example, the multi-frequency ultrasonic input device 4300 may include various low frequency transmitters 4302, low frequency receivers 4304, high frequency transmitters 4306, and high frequency receivers 4308. The square design may include a nested pattern, such as in Fig.43 The cross nesting pattern depicted in FIG. 1 may be used in any other suitable pattern. The various transmitters and receivers may be of any suitable frequency, such as between 30 kHz and 50 MHz, between 1 MHz and 10 MHz, or any other suitable range. It should be understood that reference is made to Fig.43 The frequencies described may be applicable to any suitable sensor array, for example, as described with reference to Figures 14A to 14G Described.

[0275] VII. Feature Extraction

[0276] Systems and methods according to embodiments may allow for the extraction of features from signals received, for example, by an ultrasound input device. The ultrasound input device is capable of extracting features such as energy signals as well as physical properties.

[0277] A. Identifiable Energy Signatures

[0278] Fig.44 4404, 4406 depicts example signals 4412, 4414, 4416 received by an ultrasound input system attributable to three different users, in accordance with certain aspects of the present disclosure. Each of the graphs 4402, 4404, 4406 depicts energy measurements over time associated with reflected signals detected by an ultrasound input device.

[0279] Signal 4412 of graph 4402 is an example of a dry finger pressing quickly with relatively little force. The dryness of the finger and the relatively small force show a relatively small dip in the energy measurement during the touch event. The speed of the press is seen in the relatively short duration of the dip in the energy measurement.

[0280] Signal 4414 of graph 4404 is an example of a wet finger pressing moderately with a relatively heavy force. Both the wetness of the finger and the intensity of the pressing can result in a greater damping effect on the reflected signal, and therefore a deeper dip in the energy measurement. The speed of the pressing can be seen in the moderately wide dip in the energy measurement. In addition, when the energy measurement first dips, the initial dip and subsequent dips appear more pronounced, indicating that little time was spent in contact with the surface before the full pressing force was initiated.

[0281] Signal 4416 of graph 4406 is an example of a touch event pattern in which a user taps the surface before pressing down and initiating a full touch event. The initial dip in energy measurement and the relatively long delay until the subsequent full dip indicate that the user placed a finger on the surface and waited a short time before pressing the finger down.

[0282] While signals 4412, 4414, 4416 may each be used to indicate an expected touch event due to the presence of sufficient dips in the energy measurement, each of signals 4412, 4414, 4416 contains various features that are discernible. Examples of discernible features include the depth of a dip in the energy measurement, the width of a dip in the energy measurement, the presence of an initial dip before a subsequent and deeper dip in the energy measurement, a delay between an initial dip and a subsequent and deeper dip in the energy measurement, the speed of decrease and / or increase of the energy measurement entering and exiting a dip (e.g., the speed of change of the energy signal at the edge of a dip), or any other feature of the energy measurement.

[0283] By extracting various features from the energy measurement signal, different users can be distinguished or even identified to enable additional user-based advanced functions. For example, after a training session, the ultrasonic input system is able to distinguish between a first user and a second user due to the specific way in which the user interacts with the ultrasonic input device, such as the way of touching (e.g., quick tapping or placement and pressing), the duration of the touch, the characteristics of the skin (e.g., the natural moisture or dryness of the finger), the intensity of the touch (e.g., light pressing or heavy pressing), or other characteristics that can be discerned from the energy measurement signal. Although the characteristics can be discerned from the energy measurement signal, they may not be easily perceived by the user because the energy measurement signal can be acquired at a high speed. Therefore, the difference between a quick tap and a placement and pressing can be easily discerned from the energy measurement signal, but may not be discernible or easily discernible from a visual inspection of the touch action.

[0284] Fig.45is a set of graphs depicting energy measurement signals associated with a human finger, a drop of water, and placing a device on a table (e.g., placing an object on the sensor). For a human finger, the energy measurement signal inevitably has slight movement or variation, even during the duration of a touch event, which can be detected and identified to confirm that a human finger is initiating a touch event. For a drop of liquid or water, if any, the energy measurement signal has specific characteristics, such as a sharp drop followed by a generally stable signal without much variation, if any. Detection of these characteristics can be used to distinguish actual intentional touch events from accidental contact by other objects (such as falling water). Placing a device or other object on a sensor (e.g., a sensor mounted on a table) can have an energy measurement signal with specific characteristics, such as a relatively shallow drop followed by a generally stable signal without much variation, if any.

[0285] Thus, a system as described herein can determine an energy signal associated with a set of reflected ultrasonic signals. The system can then extract feature information associated with the energy signal, and then determine an inference associated with an object based on the extracted feature information. Determining an inference can include using the feature information to determine whether a touch event is associated with a human finger or with dripping water. For example, Fig. 27 As shown, a drop of water (i.e., a water drop) can cause a larger drop in the energy signal determined by the system than a human finger (i.e., a finger). A finger may have peaks and valleys (i.e., a fingerprint) that reduce the amount of surface area placed on the sensor and therefore reduce the amount of ultrasonic signal absorbed by the object.

[0286] Thus, a criterion for the magnitude of the energy signal (e.g., corresponding to a sharp drop) can be used to distinguish between a finger touch and a water drop. Additionally, the energy signal is more consistent over time than a human finger. Thus, a criterion that the energy signal is within a specified range for a specified amount of time can be used to distinguish between a water drop and a human finger. Such a measurement can be performed using a change in the energy signal over time (e.g., a standard deviation). Thus, the characteristic information can include the magnitude of the energy signal and / or the change in the energy signal. The inferred determination can include comparing the magnitude and / or the change to a corresponding threshold to determine whether the touch event is associated with a human finger or a water drop.

[0287] Fig.46 The invention is a combined schematic diagram and a set of diagrams depicting how temperature can be used to further identify whether a human finger is initiating a touch event. The energy measurement signal output by the sensor (e.g., the sensor chip and / or substrate) depends to some extent on the temperature of the sensor. As the temperature increases, the energy measurement signal tends to decrease.

[0288] Typically, the chip will be at room temperature (e.g., at or about 20 or 21°C), while a human finger will be at body temperature (e.g., at or about 30°C). When a living tissue (e.g., a human finger) initiates a touch event, heat will be transferred between the tissue (e.g., the finger) and the chip. When the finger is hotter, it may cause the chip temperature to increase slightly. Since the energy measurement signal as a whole depends in part on the temperature of the chip and / or substrate, temperature fluctuations of the chip and / or substrate may be detected as a potentially stable increase or decrease in the energy measurement signal over time. As in Fig.46 As depicted in the lower left graph, when a warm finger is placed on a cooler sensor, the heat transfer will cause the energy measurement signal to exhibit an overall downward slope. Fig.46 As depicted in the lower middle graph of , when a cool finger is placed on the softer sensor, the heat transfer will cause the energy measurement signal to exhibit a generally upward slope (i.e., an upward trend). Fig.46 As depicted in the lower right graph, when something other than living tissue (e.g., a finger) is placed on the sensor and the other object has a temperature at or near the same temperature of the sensor (e.g., both are at room temperature), the lack of heat transfer will cause the energy measurement signal to exhibit a generally flat slope. In summary, this temperature effect on the energy measurement signal can be used to identify when something contacting the sensor is at or near body temperature, or at or near other temperatures. In some cases, the approximate temperature of the object initiating the touch event can be discerned by analyzing the general slope of the energy measurement signal.

[0289] In some cases, one or more temperature sensors may be used to measure the temperature of the chip and / or substrate. Knowledge of the temperature of the chip and / or substrate may help inform the determination of whether the object initiating the touch event is a human finger.

[0290] Fig.47 is a combined schematic diagram and diagram depicting finger touches and associated temperature information in accordance with certain aspects of the present disclosure. In some cases, the ultrasonic input system may include a temperature sensor, such as within, on, or near the chip. The temperature sensor may provide a temperature signal (e.g., a temperature sensor readout) associated with the temperature of the ultrasonic input system. Typically, when no touch event is initiated, there will be minimal or no change in the temperature signal because the ultrasonic input system will remain at or near ambient temperature, such as room temperature. However, if a touch event is initiated using a human finger, an expected temperature change toward body temperature may occur (e.g., a temperature rise from room temperature to body temperature). As shown in Fig.47 As depicted in the lower left graph of , a human finger touch can be detected or confirmed by identifying a change in the temperature signal toward body temperature (e.g., at or near 30° C.). Fig.47As depicted in the lower right graph of , a touch event initiated by an object other than a human finger (e.g., a room temperature object) will not cause the temperature of the ultrasound input system to change toward body temperature.

[0291] B. Identifiable physical characteristics

[0292] Fig.48 4802 is a schematic combined side view 4804 and signal graph depicting ridges 4806 and valleys 4808 of a fingerprint initiating a touch event on an ultrasonic input system in accordance with certain aspects of the present disclosure. When a user places a finger on a surface associated with ultrasonic input device 4810, ultrasonic input device 4810 is able to detect a portion of the user's fingerprint. Typically, ultrasonic input device 4810 can sense an area that is less than the user's entire fingerprint, although this is not always required.

[0293] The ultrasonic input device 4810 may identify ridges 4806 and valleys 4808 of a user's fingerprint (e.g., a portion of a user's fingerprint). At the ridges 4806, the ultrasonic input device 4810 will detect a decrease in the energy measurement of the reflected signal due to the damping effect of the flesh of the ridges 4806. However, at the valleys 4808, the same damping effect does not exist.

[0294] Thus, the ultrasonic input device 4810 measuring the finger as depicted in the schematic side view 4802 can generate a signal graph 4804 showing ridges 4806 and valleys 4808. As can be seen in the signal graph 4804, the darker areas represent dips in the energy measurement of the reflected signal, while the lighter areas represent signals closer to the baseline energy measurement. Although the entire fingerprint cannot be discerned from the field of view of the ultrasonic input device 4810, many ridges 4806 and valleys 4808 can be discerned. By measuring the width of the ridges 4806 and valleys 4808 and the inter-valley distance and the inter-ridge distance (e.g., the inter-ridge distance 4812), the ultrasonic input device 4810 is able to discern one finger from another. In an example case, the fingers of an adult may present wider ridges 4806 and valleys 4808 than the fingers of a young person. Therefore, in a family with adults and children, the ultrasonic input device 4810 is able to discern between two users based on discernible physical characteristics of the user's fingers, such as fingerprint characteristics. In some cases, the presence of a repeating line pattern (eg, a pattern of ridges 4806 and valleys 4808) may be used to confirm or determine whether the object initiating the touch event is a human finger.

[0295] In some cases, a discernible physical characteristic, such as a fingerprint, may be used along with a discernible energy signature to further identify a user.

[0296] Fig.494908 and the like, and may have additional attenuation due to the presence of gloves 4912. The first set of reflected signals 4924, 4925 associated with the ultrasonic input device 3502 with touch input from the user not wearing gloves 4906 shows a transmission signal 4922 and a first set of reflected signals 4924. The first set of reflected signals 4924 shows the characteristic attenuation of the reflected signals associated with the touch event. The second set of reflected signals 4925, which are associated with the ultrasonic input device 3502 with touch input from the user wearing gloves 4908, shows the transmission signal 4922 and the second set of reflected signals 4925. The second set of reflected signals 4925 shows the characteristic attenuation of the reflected signals associated with the touch event, which is somewhat similar to the first set of reflected signals 4925, but may have additional attenuation due to the presence of gloves 4912. The first set of reflected signals 4924 can be processed to generate a first output voltage 4932. Similarly, the second set of reflected signals 4925 may be processed to generate a second output voltage 4933 .

[0297] Thus, the ultrasound input system 4902 can distinguish between gloved hands and ungloved hands. In some cases, specific actions may or may not be available depending on whether the user is wearing gloves. For example, in a doctor's office, specific functions associated with the ultrasound input system may not be available unless the user is wearing gloves to ensure that proper protection is in place.

[0298] C. Extracting and using features

[0299] Fig.50 is a flow chart depicting a method 5000 for extracting features from a signal input to an ultrasound system, in accordance with certain aspects of the present disclosure. Fig.50 The method shown in will be described in the context of a system including an ultrasonic input device and one or more data processors that determine an energy signal from a touch event. However, it should be understood that the present invention may be applied to other situations.

[0300] At optional block 5002, a baseline signal may be received by the ultrasound input system. The baseline signal may be an energy measurement associated with a no-touch event (e.g., when no user is touching a surface coupled to the ultrasound input device). Removing such a baseline signal may result in a more efficient sampling range during analog-to-digital conversion, for example, as described herein at least with reference to Fig. 27For example, the ultrasonic input system may transmit a first signal. Subsequently, any appropriate number of reflected ultrasonic signals and reflected-transmitted signals may be measured by the ultrasonic input system. Based on the characteristics of the received signal, it can be determined that the signal is not associated with a touch event (e.g., a finger touching an outer surface). For example, the received signal may indicate a baseline signal associated with the air signal. Further example details of the baseline signal are described herein.

[0301] At block 5004, the system can transmit the transmitted signal using an ultrasonic input device. The ultrasonic input device can be coupled to a material layer having an outer surface positioned opposite the material layer of the ultrasonic input device. The transmitted signal can pass through the material layer toward the outer surface. As described in detail herein, any number of reflected ultrasonic signals and reflected-transmitted signals can be generated from the initial transmitted ultrasonic signal until the signal becomes too weak to be reflected and / or detected.

[0302] At block 5006, a signal associated with the touch event is received. For example, the system may receive a set of reflected ultrasound signals associated with the transmitted signal. The received signal may be a measurement of the energy associated with the reflected ultrasound. The signal received at block 5004 may depend on how the touch event was initiated (e.g., the timing of the touch, the manner of the touch, the amount of force of the touch, the physical characteristics of the object initiating the touch).

[0303] At box 5008, one or more data processors of the system can determine an energy signal associated with a set of reflected ultrasonic signals, which are associated with a touch event between the object and the outer surface of the material layer coupled to the ultrasonic input device.

[0304] As an example, see Fig. 27 , flow chart 2700 includes transmitting and receiving ultrasonic signals, as shown in a first graph 2702. The first graph 2702 shows an analog measurement of a first signal 2703 for a transmitted ultrasonic signal, and an analog measurement of a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E for a set of reflected ultrasonic signals associated with the ultrasonic input device. Flow chart 2700 may include an ultrasonic input device having an analog summing or integrating circuit 2720 and a summed voltage output 2722.

[0305] The second graph 2708 shows the first signal 2703 and a set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E, which correspond to the energy of a corresponding one of the set of reflected ultrasound signals, respectively. For illustration purposes, the set of energy signals 2710A, 2710B, 2710C, 2710D, 2710E are depicted in solid lines, which overlap with a set of subsequent signals 2704A, 2704B, 2704C, 2704D, 2704E from the first graph 2702 shown in dashed lines.

[0306] At block 5010, after determining an energy signal associated with the set of reflected ultrasonic signals, features may be extracted from the signal associated with the touch event. The extracted features may be any suitable characteristics of the signal that may be discernible and / or capable of informing inferences. One or more data processors may be configured to extract feature information associated with the energy signal in any suitable manner.

[0307] In some embodiments, extracting the characteristic information may include identifying a pattern in the energy signal associated with a dip in energy measurement associated with a touch event. For example, when an individual places their finger on the system, particularly on an external surface, the individual's finger may absorb at least a portion of the transmitted ultrasonic signal, thereby causing a dip in energy measurement.

[0308] The pattern may be identified in any suitable manner as described herein. For example, in some embodiments, identifying a pattern in the energy signal may include identifying the depth of a dip, the duration of a dip, the presence of a subsequent dip after a dip, the delay between a dip and another dip, and / or the rate of change of the energy signal at the edge of a dip (e.g., during finger land or removal). In other embodiments, identifying a pattern may include identifying changes in the energy signal attributable to temperature drift in a material layer, as described in detail herein.

[0309] In some cases, extracting features (i.e., feature information) at block 5010 may include comparing the signal to (one or more) stored historical signals, such as to determine whether the signal received at block 5010 matches a stored signal associated with a particular user. In some cases, extracting features at block 5010 may include identifying patterns in the received signal, such as to identify that the received signal is attributable to a distinct tap or a place and press action. In some cases, extracting features at block 5010 may include measuring a characteristic of the received signal. Any discernible characteristic of the received signal may be measured and used to make a determination or inference about the source of the touch event.

[0310] At block 5012, an inference may be determined based on the extracted feature information.The one or more data processors may be configured to determine an inference associated with the object based on the extracted feature information in any suitable manner.

[0311] For example, in some embodiments, determining the inference may include estimating the relative temperature of the object based on the identified changes in the energy signal attributable to the temperature drift in the material layer. For example, an individual contacting the outer surface of the material layer may have a body temperature that is higher than the ambient temperature and / or the temperature of the material layer. As described herein, the determined energy signal may be affected by temperature and thus allow one or more data processors to determine an inference of temperature measurement and / or temperature drift (e.g., as measured by a temperature sensor as described below).

[0312] In other embodiments, one or more data processors can determine the inference by comparing the identified pattern with stored data. The stored data can be associated with a previous touch event of the external surface. For example, a previous touch event of the external surface may have been performed by an individual. As described herein, the current touch event can be compared with the previous touch event to determine whether the current touch event was also performed by an individual.

[0313] In other embodiments, one or more data processors may determine inferences using the feature information to determine that a touch event is associated with a human finger, a bare human finger, a wet human finger, a dry human finger, and / or a gloved human finger. For example, as described herein, the determined energy signal may be affected by one or more characteristics of an individual finger(s) placed on the outer surface of the material layer. One or more data processors may also determine inferences using the feature information to determine the manner of touch (e.g., tap, double tap, placement, press, etc.) of the touch event, the touch intensity associated with the touch event, and / or the physical characteristics of the object.

[0314] In some embodiments, determining the inference may include identifying that the object is associated with one of the plurality of users based on associating the touch event with a touch pattern of the touch event, a touch intensity associated with the touch event, and / or a physical characteristic of the object. The physical characteristic of the object may include a measurement associated with a portion of a fingerprint contacting the external surface.

[0315] In some embodiments, one or more data processors may determine additional sensors associated with the ultrasound input device (e.g., Fig.29 The one or more data processors may then further use the additional signals to determine the inference. The additional sensors may include any suitable additional sensors associated with the ultrasonic input device. For example, the additional sensors may include a temperature sensor, a pressure sensor, a charge coupled device, etc.

[0316] For example, the system may include a temperature sensor. The temperature sensor may record, for example, the temperature of an outer surface of the system over time. Since human fingertips have specific physical dimensions and temperature ranges, when a human touches an outer surface, one or more data processors may determine that the touch event is caused by a human finger. As an illustrative example, the temperature sensor may record the temperature of at least a portion of the outer surface at predetermined intervals (e.g., 1ms, 0.1s, 1s, etc.). The temperature sensor may record an ambient temperature (e.g., 70°F). When a user touches the outer surface during a touch event, the system may record an energy signal, which may include, for example, a sink of energy. During the touch event, the temperature sensor may continue to measure the temperature of the outer surface. A human finger in contact with the outer surface may increase the temperature of the outer surface, thereby causing the temperature sensor to record an increase in the temperature. For example, a human finger may be approximately 98°F. The temperature sensor may record a temperature between an ambient temperature of 70°F and a human finger temperature of 98°F, because the finger will heat the outer surface and the temperature sensor.

[0317] The temperature measured by the temperature sensor may be an additional signal associated with an additional sensor (e.g., a temperature sensor) associated with the ultrasonic input device. One or more data processors may use the additional signal in conjunction with the energy signal to determine an inference. For example, one or more data processors may determine that a dip in the energy signal and a temperature rise from ambient temperature to a higher temperature between ambient temperature and average human body temperature indicate that a touch event indicates that a human finger touched the external surface. In some cases, one or more data processors may use temperature data from the temperature sensor to determine whether the signal change is the result of a human touch or from another object (e.g., a table, pocket fabric, pen / stylus, etc.) in contact with the external surface. For example, when contacted with a table, pocket fabric, pen / stylus, etc., the temperature sensor may not measure as much of a temperature increase as when contacted by a human finger.

[0318] In some cases, the temperature sensor may be a known (i.e., predetermined) distance from the finger. For example, the temperature sensor may be located on a side of the outer surface opposite the finger. In this case, during processing of additional signals associated with the additional sensors (e.g., the temperature sensor), a heat transfer problem with known boundary conditions and initial values ​​may be solved to determine what the temperature at the outer surface is.

[0319] In some embodiments, the additional sensors may include pressure sensors and / or strain gauges. For example, a typical touch from a human finger may apply a specific force and strain on the outer surface that may be propagated to the additional sensors. The pressure sensor and / or strain gauge can measure the force and / or strain applied to the system by the finger. One or more data processors can determine that the force and / or strain measured by the pressure sensor and / or strain gauge indicates a typical force and / or strain of the touch of the finger. The one or more data processors can also determine whether the energy signal indicates a touch of the finger. If both the additional signal from the pressure sensor and / or strain gauge and the energy signal indicate a touch of the finger, the one or more data processors can determine that the touch event is a touch of the finger.

[0320] In some cases, the additional sensor may include a strain gauge. The strain gauge may detect the deflection of the surface associated with a touch event and may output an electrical signal. The stronger the touch event (e.g., the greater the force applied by an object such as a finger on the outer surface), the greater the deflection applied to the strain gauge. Therefore, the strain gauge may output a larger electrical signal.

[0321] At block 5014, one or more data processors may generate an output signal associated with the determined inference. The output signal may include any suitable output generated based on the determined inference. In some embodiments, the output signal may indicate a specific action that may be performed by one or more data processors and / or an external device.

[0322] In some embodiments, one or more data processors may perform an action based on the extracted (one or more) features. The action may include any suitable process that may occur based on the output signal. In an example, if the extracted features are used to identify a specific use, the action performed may be to authenticate or authorize the user to access a resource. In another example, if multiple users have preset customizations for a specific ultrasound input system, the extracted feature information may be used to determine which user is interacting with the ultrasound input system and therefore perform customized actions for that particular user. In some cases, performing an action may include allowing or denying access to a resource, such as denying access to a room or tool when the extracted features indicate that the user is not wearing gloves (when gloves are required).

[0323] VIII. Machine Learning Decision Algorithms

[0324] Fig.51 is a diagram 5100 depicting a machine learning decision algorithm for improving touch detection according to certain aspects of the present disclosure. Fig.37As described above, the weight parameters can be used to drive various decisions about when a touch event is detected or not detected. In some cases, the machine learning method can consider the sensor output value and the slope between the sensor value and the previous sensor value to generate an inference that a touch event has occurred or not occurred. The machine learning method can use a decision function (f) such as:

[0325] f=w0X[n]+w1X[n-1]+w2X[n-2]+…+w m X[nm]+w s0 S[n]+W s1 S[n-1]+…+w sm S[nm]

[0326] where w n and w sn is a weight parameter, X[n] is the current sensor output, X[n-1] is the previous sensor output, X[nm] is the mth previous sensor output, S[n] is the slope of the current sensor output (e.g., compared to the immediately previous sensor output), S[n-1] is the slope of the previous sensor output, and S[nm] is the slope of the mth previous sensor output. In some cases, other parameters may be used in the decision function.

[0327] The weight parameters of the decision function can be trained on a corpus of data to generate a decision boundary between inputs that are considered to be touch events and inputs that are not considered to be touch events, as depicted in graph 5100. Thus, for any given sensor output and slope of the sensor output, a point on graph 5100 can be identified, and if that point falls above the decision boundary, then those sensor outputs and slope of the sensor output can be considered to be indicative of a touch event.

[0328] IX. Intelligent touch event detection

[0329] Systems and methods according to embodiments may allow for a touch event detection framework. Embodiments allow for an adaptive threshold for touch event detection. The adaptive threshold scheme may involve identifying a touch event from an energy signal of a sensor using a continuously adaptive threshold. Embodiments may also allow for a recursive neural network for touch event detection and / or a recursive neural network for state classification.

[0330] A. General touch event detection framework

[0331] Fig.52 is a flow chart depicting a process 5200 for detecting a touch event according to certain aspects of the present disclosure. The process 5200 may be performed by any suitable device, including Figure 7Processor 722 and / or computing device 724. In some cases, data from multiple sensors can be used in any of blocks 5202, 5204, 5206.

[0332] At block 5202, energy signal data is accessed. Energy signal data is signal data from an ultrasonic sensor that indicates the amount of energy sensed by the ultrasonic sensor during a period of time, such as reference Fig.36 Energy signal 3614 is depicted and described. Any suitable time period may be used.

[0333] At block 5204, a touch event may be identified based on the energy signal. Identifying the touch event may include determining whether a touch event has occurred based on the energy signal. In some cases, identifying the touch event at block 5204 may include outputting a touch signal. The touch signal may indicate whether the associated energy signal is inferred to be associated with a touch event.

[0334] At optional box 5206, a state classification can be identified from the touch event data (e.g., the touch signal from box 5204). In some cases, the state classification can be identified from the touch event data and the associated energy signal data. The state classification can be a classification associated with the touch event. Any suitable classification can be determined, such as the type of touch event that has occurred. Examples of suitable state classifications related to the type of touch event that has occurred include single click, double click, triple click, n-times tap, hold (e.g., touch and hold), tap and hold (e.g., tap followed by touch and hold), press (e.g., longer than tap), double press, press and hold (e.g., press followed by touch and hold), hold and press (e.g., touch and hold for a duration followed by press) and grip (e.g., hold with more surface area or other characteristics). A state classification can be determined, which can be associated with a touch event based on a trigger value. Examples of suitable state classifications related to other information associated with a touch event may include whether the user is wearing gloves, whether the user looks older or younger (e.g., based on the distance between fingerprint ridges), whether the user appears to be a pre-identified user, or other such classifications.

[0335] As an example, another classification may include hydration and / or perspiration of the user's fingers and / or body. The system may detect hydration and / or perspiration of the user, for example, by determining an ultrasonic signal absorption that is lower than a typical ultrasonic signal absorption of the user. When the user's fingers are drier, the fingers will absorb less ultrasonic signals. Therefore, different levels of thresholds for amplitude and change over time may be used. For example, a wet finger may cause a more uniform drop in the energy signal than a dry finger. Therefore, a criterion for the magnitude of the energy signal (e.g., corresponding to a sharp drop) may be used to distinguish between dry and wet fingers. In addition, due to the presence of additional water in a wet finger, the energy signal may be more consistent over time in the case of a wet finger than in the case of a dry finger. Therefore, a criterion that the energy signal is within a specified range for a specified amount of time may be used to distinguish between wet and dry fingers. Such measurements may be performed using changes in the energy signal over time (e.g., standard deviation). Therefore, the feature information may include the magnitude of the energy signal and / or the change in the energy signal. The inferred determination may include comparing the magnitude and / or change with a corresponding threshold to determine whether the touch event is associated with a wet finger or with a dry finger.

[0336] In some cases, any number of classifications may be used, depending on the orientation and placement of the sensor. In some cases, the state classification may be trained so that identification of the state classification at block 5206 may be made with reference to training data or a model generated using training data.

[0337] B. Adaptive Threshold for Touch Event Detection

[0338] Fig.53 is a schematic diagram illustrating an adaptive threshold scheme 5300 for identifying touch events according to certain aspects of the present disclosure. The adaptive threshold scheme 5300 may be executed in part or in whole on a processor coupled to an ultrasonic sensor, such as Figure 7 Processor 722. Fig.53 The method described in can be performed by an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other suitable device and / or controller described herein. The scheme 5300 does not require machine training, however, one-time tuning for each new cover material to which the sensor is coupled can improve detection. The adaptive threshold scheme 5300 can operate over discrete time periods or frames, so that at any time, the scheme 5300 can analyze current data from the current observation and historical data from any number of past observations.

[0339] Adaptive threshold scheme 5300 involves identifying touch events from a sensor's energy signal using a continuous adaptive threshold. The threshold is a continuously tracked version of the energy signal that has been filtered through an adaptive threshold update function, whose filter parameters are adaptively updated based on the energy signal, historical thresholds, and optional trigger data (e.g., whether a touch event is occurring or has occurred recently). Whenever the energy signal exceeds the adaptive threshold (i.e., less than the adaptive threshold, greater than the adaptive threshold, less than or equal to the adaptive threshold, or greater than or equal to the adaptive threshold), a sensor trigger (e.g., a touch event) can be identified. Optionally, scheme 5300 can analyze current and past trigger history (e.g., based on a specific number of recently observed touch event signals) to identify a current state classification, such as determining whether the touch event is a tap, press, hold, or other action. For example, analyzing the touch event signal (e.g., the trigger history) can show how many times the energy signal has dropped below the threshold within a specific time frame, how far the energy signal has dropped, how long the energy signal has been below the threshold, and other such characteristics that can be used by the scheme to make a determination about the current state of the sensor.

[0340] Sensor data 5302 may include current and historical sensor data from sources such as Figure 7 The energy signal (eg, energy data) of the sensor of the ultrasonic sensor 702 or based on the energy signal (eg, energy data) from a sensor such as Figure 7 The energy signal (e.g., energy data) of the sensor of the ultrasonic sensor 702. The sensor data 5302 can be provided to an adaptive threshold update function 5304, which can use the data to generate threshold data 5306. The adaptive threshold update function 5304 can use only the current sensor data or both the current sensor data and the historical sensor data, as well as any additional parameters, as appropriate. The adaptive threshold update function 5304 acts as a low-pass filter that allows slow changes in the energy signal to have a significant effect on the adaptive threshold, while faster changes in the energy signal have minimal or negligible effects on the adaptive threshold. Therefore, factors that cause slow changes in the energy signal, such as temperature changes in a room, can be automatically compensated in the adaptive threshold, while factors that cause fast changes in the energy signal, such as a finger press, will be correctly detected as exceeding the threshold.

[0341] Threshold data 5306 may represent a threshold value above which (e.g., below which) the energy signal should be considered to have triggered a touch event. Condition analyzer 5308 may compare sensor data 5302 (e.g., energy data) and threshold data 5306 to determine whether a trigger event (e.g., a touch event) has occurred. For example, when the current energy signal from sensor data 5302 drops below the current threshold in threshold data 5306, it may be assumed that a touch event has occurred. In some cases, condition analyzer 5308 may also provide feedback to adaptive threshold update function 5304 to update the parameters of adaptive threshold update function 5304. For example, the speed, degree, or number of times the energy signal drops below the threshold may be used to manipulate how adaptive threshold update function 5304 generates threshold data 5306 from sensor data 5302. If a trigger event (e.g., a touch event) is detected, condition analyzer 5308 may output one or more trigger values ​​in trigger data 5310. The trigger data 5310 may include current trigger data and historical trigger data indicating whether a trigger event (e.g., a touch event) has occurred. For example, the threshold data may be updated based on the energy data, the trigger data, and the threshold data, wherein updating the threshold data includes generating a subsequent threshold.

[0342] In some cases, trigger data 5310 may be used by condition analyzer 5308 to further inform a decision of condition analyzer 5308 regarding whether a trigger event has occurred. For example, historical trigger data may be used by condition analyzer 5308 to confirm or refute a possible trigger event. However, in some cases, trigger data 5310 may be used by condition analyzer 5308 to provide updated parameters to adaptive threshold update function 5304, such that adaptive threshold update function 5304 is further updated based on current and / or historical trigger data.

[0343] In some cases, the trigger data 5310 may be passed to a trigger analyzer 5312 to determine a sensor state 5314. The trigger analyzer 5312 may obtain information from the trigger data 5310 to determine whether the most recent trigger event was a touch, a tap, or other such classification of a trigger event. The trigger analyzer 5312 may then output its determination as a sensor state 5314. The sensor state 5314 may indicate not only a trigger event, but also a classification of a state associated with the trigger event. For example, while the trigger data 5310 may take a form indicating whether a trigger has occurred (e.g., a binary signal), the sensor state 5314 may also take a form indicating what state the sensor is in. Example sensor states include hold, tap, press, double-click, etc. The classification of sensor states may be selected from a predetermined list, each list having a different pattern of energy signals.

[0344] As used herein, various signals may be considered to include multiple data points, including a current data point (e.g., a recent data point) and any number of previous data points. As used herein, the term historical data may include current data points and past data points. Data points may be represented as analog or digital signals.

[0345] Fig.54 is an example graph 5400 depicting an energy signal 5402 and an adaptive threshold 5404 associated with identifying a touch event according to certain aspects of the present disclosure. Fig.53 As described above, energy signal 5402 may be from any suitable sensor data 5302, such as Figure 7 Ultrasonic sensor 702. Such as reference Fig.53 As described, the adaptive threshold 5404 may be the threshold data 5306 generated from the energy signal 5402 .

[0346] As in Fig.54 , the gradual change in the intensity of the energy signal 5402 throughout the course of the graph 5400 is reflected in the adaptive threshold 5404. Specifically, as can be seen in the relatively constant segments when no fast spikes occur, the average intensity of the energy signal 5402 steadily decreases over time, which is then reflected in the adaptive threshold 5404, which itself steadily decreases over time. However, the rapid changes associated with the touch event, depicted as the fast negative spikes in the energy signal 5402, are not fully reflected in the adaptive threshold 5404, which allows the energy signal 5402 to drop below the adaptive threshold 5404. Each time the energy signal 5402 drops below the adaptive threshold 5404, the system can register a touch event. Based on various attributes of the detected touch event, such as frequency, intensity, duration, and other such attributes as disclosed herein, a determination can be made regarding the classification of the touch event (e.g., the state of the sensor), such as whether the sensor is being tapped, pressed, held, double-clicked, or otherwise manipulated.

[0347] C. Recurrent Neural Network for Touch Event Detection

[0348] Fig.55 5500 is a schematic diagram of a general recurrent neural network 5500 according to certain aspects of the present disclosure. The recurrent neural network 5500 is a data analysis technique that operates to transform input data 5502 into output data 5508. The recurrent neural network 5500 can be used to identify touch events from energy signals (e.g., as in Fig.52 ) and identifying state classifications from touch events and / or energy signals (e.g., as in Fig.52For example, the input data 5502 may be an energy signal or energy data from any suitable sensor, such as Figure 7 The ultrasonic sensor 702 of the present invention can be used, and the output data can be a trigger signal or a state classification signal. For example, the ultrasonic sensor can provide energy data to the recurrent neural network to generate output data indicating the occurrence of a touch event.

[0349] The recurrent neural network 5500 may pass input data 5502 through any number of nodes across any number of layers until output data 5508 is generated. In some cases, the output data (e.g., output data 5508) may include state classification information associated with the touch event. One or more hidden layers 5504, 5506 may be located between the input data 5502 and the output data 5508. Within each hidden layer 5504, 5506, a node 5510 may process the input data into outgoing data. In the node 5510, any number of inputs may be received and processed (e.g., summed and passed through a function) to generate an output. In the example node 5514, three inputs are received (e.g., weighted versions of other layers, such as w i k,1 ;w i k,2 ;w i k,m((i-1) ) and sum them and pass them through a function to produce a single output (e.g., a i k ). In other words, in some cases, the output of a node may be a decision function that is a linear combination of the outputs of the previous layer, optionally with additional feedback as described below with reference to a tapped delay line. In other cases, the inputs to node 5514 may be linearly combined and then passed to another function f. The function f may be an activation function, which may be linear or nonlinear. For example, the activation function may include a sigmoid function, a hyperbolic tangent function, a rectified linear unit (ReLU) function, a recognition function, and / or any suitable activation function. The activation function may constrain the output to a probabilistic form between any suitable limits (e.g., 0 to 1, -1 to 1, -0.5 to 0.5, etc.). Subsequently, the output from node 5510 may be passed to one, some, or all of the nodes in the subsequent layers, or, in the case of the final layer, may be passed to the output and used to generate output data 5508 (e.g., by summing or other functions) together with other outputs from the same layer.

[0350] As in Fig.55As depicted in , the recurrent neural network 5500 may also utilize tapped delay lines 5512. Each tapped delay line 5512 may be used to provide input to one, some, or all nodes 5510 of a particular layer (e.g., layer 5504), which may receive a current output or a delayed output from that layer or a subsequent layer (e.g., layer 5506 or output 5508) via the tapped delay line 5512. In some cases, the tapped delay line 5512 may also provide a delayed version of the input data 5502 as input to each node 5510 of a particular layer. Example tapped delay line 5516 depicts a vector of inputs from layer j 5506 (e.g., a j ), which is delayed and output as input to the various layers of layer i 5504 (e.g., wd i,j ). In this way, outputs from subsequent layers (e.g., historical outputs) can inform earlier layers in the recurrent neural network 5500. The tapped delay line 5512 may include data from any suitable length of time. For example, the tapped delay line 5512 may provide data from a single frame in the past or from multiple frames.

[0351] Fig.55 The recursive neural network 5500 in is depicted as having a single input (e.g., input data 5502), a hidden layer i 5504 containing m(i) different nodes, a hidden layer j 5506 containing m(j) different nodes, and a single output (e.g., output data 5508). In some cases, the recursive neural network 5500 used in accordance with certain aspects of the present disclosure may include any suitable number of inputs, layers, nodes, and outputs. The recursive neural network 5500 can be pre-trained through supervised machine learning by providing labeled sensor data to the system to allow the function of each node (e.g., the weight value of each node) to be updated until the recursive neural network 5500 performs as desired.

[0352] In some cases, a recurrent neural network can be trained using historical energy data associated with multiple historical touch events. The historical energy data may include previous energy data recorded and stored in an appropriate memory and / or database. Multiple historical touch events may include data about previous touch events. For example, the historical energy data may include a sink of energy that may be associated with a historical touch event (e.g., "0" indicates no touch, "1" indicates touch). The historical touch event may include one or more of each of a set of state classifications. For example, a touch event "1" may be associated with a state classification of "tap".

[0353] In some embodiments, the group of state classifications can be selected from a plurality of available state classifications by user input. The plurality of available state classifications may include, for example, a list of state classifications that can be selected by the user. For example, in some cases, a plurality of available state classifications may include tapping, double-clicking, pressing, and holding. In other cases, a plurality of available state classifications may include tapping, pressing, double pressing, and gripping. A plurality of available state classifications may include any suitable combination of state classifications.

[0354] In other embodiments, the plurality of historical touch events may further include a plurality of non-touch events. Non-touch events may facilitate training of additional recursive neural networks to reject false positive events. Non-touch events may include, for example, touch events indicating no touch. For example, a non-touch event may be associated with a dip in the energy signal, but may be associated with an event in which a drop of water touches the outer surface rather than through a finger, as described herein. In some cases, the user may be prompted to touch the outer surface against other objects (e.g., pens, fabrics, etc.) that the user wishes to classify as non-touch events. In this way, the user may provide non-touch event data associated with situations in which the user does not wish the device to determine a touch event. In some cases, the device may provide output data from the recursive neural network to an additional recursive neural network to generate state classification information associated with the touch event.

[0355] Fig.56 is a schematic diagram depicting an example recurrent neural network 5600 for identifying trigger events in accordance with certain aspects of the present disclosure. The recurrent neural network 5600 may be particularly useful for efficiently and accurately detecting trigger events from ultrasound energy signals.

[0356] At input 5602, sensor data may come from, for example, Figure 7The energy signal of the ultrasonic sensor of the ultrasonic sensor 702 of the ultrasonic sensor 702 is provided to the recurrent neural network 5600. The energy signal is transmitted on two nodes 5610, 5612 of the hidden layer 1 5604 and on the tapped delay line 5618, which provides one or more delayed signals to the nodes 5610, 5612 of the hidden layer 1 5604 based on the energy signal from the input 5602. For example, the tapped delay line 5618 can be set to provide the energy signal of the last three or four frames to the hidden layer 1 5604. In addition, the nodes 5610, 5612 of the hidden layer 1 5604 can take as additional input the output of the tapped delay line 5620, which can be configured to output a set number of past frames of the output 5608 of the recurrent neural network 5600. For example, the tapped delay line 5620 can be configured to provide the immediately previous frame of data output via the output 5608 of the recurrent neural network 5600 as input to the hidden layer 1 5604. The outputs from nodes 5610, 5612 of hidden layer 1 5604 may then be passed as inputs to nodes 5614, 5616 of hidden layer 2 5606. The outputs of nodes 5614, 5616 of hidden layer 2 5606 may then be passed as trigger data to output 5608 (e.g., combined and output).

[0357] In some cases, a strong combination of efficiency and accuracy for identifying trigger data from an ultrasonic energy signal may be to use a recurrent neural network 5600 having a first layer that receives some combination of sensor data, past sensor data, and past trigger data and a second layer that receives the output of the first layer. The output from the second layer may be used to generate a trigger data output.

[0358] The recursive neural network 5600 may be trained in advance and / or by a user. Training of the recursive neural network 5600 may include providing energy signals that are appropriately labeled as touch events or not touch events. The training data may be provided by a supplementary input device (e.g., a physical button or electrical contact) that simultaneously records the touch event when the ultrasonic sensor detects the energy signal associated with the touch event; or by associating the recorded energy signal with the touch event, such as by instructing the user to initiate a touch event at a specific time or at a specific rhythm. Once the training data has been obtained, the recursive neural network can be programmed or trained by supervised machine learning, which allows the function of each node (e.g., the weight value of each node) to be updated until the recursive neural network 5600 performs as desired (e.g., accurately identifies the trigger event). In some cases, the recursive neural network 5600 can be retrained each time the ultrasonic sensor is connected to a new material stack.

[0359] Due to the nature of a typical recurrent neural network, the output 5608 may take the form of a number. Inferring appropriate trigger data from the number may include applying a threshold to the actual output 5608 of the recurrent neural network 5600. For example, if the recurrent neural network 5600 outputs a number between 0 and 1.0, a threshold may be set between the two numbers, and if above the threshold, the output may be considered a trigger event (e.g., a touch event), and if at or below the threshold, the output may be considered not to be a trigger event (e.g., no touch event), or vice versa. In this example, the threshold may be set to 0.5, so an output of 0.55 may be considered a touch event. In some cases, the recurrent neural network 5600 may adjust its sensitivity rather than simply retraining the entire neural network by adjusting the threshold. Therefore, in order to reduce the possibility of false triggering (e.g., reducing sensitivity), the threshold may be moved from 0.5 to 0.6. Therefore, the same output of 0.55 will not be considered not to be a touch event.

[0360] In some cases, the unsupervised machine learning model can analyze data that has not yet been labeled. The unsupervised machine learning model may include any suitable type of unsupervised machine learning model, such as clustering (e.g., k-means, hierarchical clustering, etc.), anomaly detection, etc. The multiple trigger values ​​can be measured by the device while being used by the user. For example, the user can perform any appropriate number of touch events that can be recorded. At this point, the multiple trigger values ​​from the touch event may not yet be labeled as, for example, tap, hold, press, etc. The multiple trigger values ​​may include at least 0s and 1s, which indicate a touch detected at a specific time. Data items including continuous trigger values ​​can be used to determine a state.

[0361] The unsupervised machine learning model can group multiple trigger values ​​or data items created thereby (e.g., using a clustering approach). As an illustrative example, the unsupervised machine learning model can group data items similar to (0, 0, 0, 1, 1, 1, 0, 0, 0) into a first cluster. The unsupervised machine learning model can group data items similar to (0, 0, 1, 1, 0, 0, 1, 1, 0, 0) into a second cluster. The unsupervised machine learning model can create any suitable number of clusters based on multiple trigger data.

[0362] The user may be prompted to provide supervised data (e.g., to provide a desired touch event). In some embodiments, recurrent neural network 5600 may further determine classifications for clusters determined from an unsupervised machine learning model based on the supervised data as part of training recurrent neural network 5600. For example, a cluster with data items similar to (0, 0, 0, 1, 1, 1, 0, 0, 0) may be labeled as a "tap," while a cluster with data items similar to (0, 0, 1, 1, 0, 0, 1, 1, 0, 0) may be labeled as a "double tap."

[0363] D. Recurrent Neural Network for State Classification

[0364] Fig.57 5700 is a schematic diagram according to certain aspects of the present disclosure, which depicts an example environment 5700 for touch detection and state classification using a set of recurrent neural networks 5706, 5708. The environment 5700 shows an example of a touch detection and state classification system that can be presented on a computing device such as Figure 7 A user interface 5702 on a computing device 724 of the device is used to generate a user interface associated with an ultrasonic sensor (e.g., Figure 7 The user interface 5702 allows the user to select those states to be detected and identified. Subsequently, the recurrent neural networks 5706, 5708 can be trained to recognize the selected states. However, it should be understood that the recurrent neural networks 5706, 5708 are not limited to recognizing the selected states. For example, the recurrent neural networks 5706, 5708 can be trained to recognize the trigger output based on the trigger data.

[0365] In some cases, a single recurrent neural network may be used to generate an output indicative of a state based on a received energy signal as input. Fig.57 As depicted in , a first recurrent neural network 5706 may receive an energy signal as input 5704 and output a trigger signal, which may then be passed as input to a second recurrent neural network 5708, which may then output a state signal as output 5710. Fig.57 , output 5710 is depicted as a graph of an imaginary feature space. In the imaginary feature space, different possible states are distinguishable based on their positions in the imaginary feature space. The imaginary feature space can be depicted as two-dimensional, although it can actually be based on any number of dimensions, including one or more than two dimensions. The output 5710 of environment 5700 can indicate a specific state of the ultrasonic sensor, such as being tapped, touched, pressed, double-clicked, or any other suitable state.

[0366] As in Fig.57The environment 5700 depicted in shows a single input and depicts four possible energy patterns that can be placed into the input. However, in some cases, a single environment 5700 can utilize multiple sensors to provide multiple energy signals to (one or more) recurrent neural networks.

[0367] Model information may be stored when training the recursive neural network(s) of the environment 5700. In some cases, the model information may be stored locally at the sensor (e.g., on a data store associated with a processor driving the ultrasonic transducer), although this need not always be the case. In some cases, the model information may be stored remotely (e.g., on a computing device separate from the sensor) or may be split, such as storing model information locally at the sensor for determining whether a trigger event has occurred, and storing model information remotely for determining the state of the sensor based on the trigger signal. The model information may be any information that may be used to generate, and optionally interpret, an output from an input energy signal. For example, the model information may include information about the structure and weights found in any recursive neural network(s) of the environment 5700.

[0368] During an example training session, the user can select a set of states to train into the model information. Fig.57 As depicted in , the selected states include single click, double click, and hold. When prompted to do so, the user can engage in each action associated with each state, thereby generating an energy signal as input data. As the user is prompted to engage in a particular action, environment 5700 can associate the detected energy signal as being associated with a particular state (e.g., single click or double click). As in Fig.57 As depicted in , information associated with a single click is shown in green, a double click is shown in blue, a hold is shown in yellow, and an error event is shown in red. For example, an error event may be generated by prompting the user to touch the periphery of the sensor rather than directly above the sensor to make the algorithm less susceptible to such undesired error inputs and thus make it more sensitive to local inputs located immediately above the sensor. Training data may be collected from one or more users once or repeatedly until the recurrent neural network(s) are adequately trained.

[0369] In some cases, training can be offline (e.g., performed on a set of test sensors and optimal network parameters written for all sensors for that particular application), or the user can be prompted to perform a training session by the user when the system is initialized (e.g., this can be similar to a fingerprint login on a phone). The method can also be a combination of the two methods. For example, the method can include offline training and some optimization during user use. In other cases, data (e.g., energy data, state data, trigger data, etc.) can be shared on the cloud or through other suitable communication channels to enhance the training data database in order to improve network models, training and optimization.

[0370] In some cases, recurrent neural networks can be particularly useful for time series data and can be more easily optimized for different material stacks and different environmental conditions. In some cases, an environment with multiple recurrent neural networks can allow different types of useful information to be output from sensors (e.g., from a processor driving an ultrasonic transducer). For example, a sensor can output energy signals, trigger signals, and state information from different points in the environment 5700. Therefore, the same sensor can be mass-produced and quickly used in a variety of different ways. Although some customers may prefer to use trigger signals, other customers may want to use state information. Therefore, the same mass-produced sensors can meet the needs of different customers. In addition, if multiple sensors are installed in a unit, the host / client can also decide how to combine information from the sensor network into bundled events when an action is triggered and / or even when training, such an example can be a slider or a mouse pad. Generally, when multiple sensors are installed in a unit, information from multiple sensors can be used to enhance the performance of the algorithm and improve its robustness.

[0371] X. Application

[0372] Fig.58 58 is a schematic diagram according to certain aspects of the present disclosure, which depicts an electronic device with an ultrasonic input device. The electronic device 5800 may include a housing 5802, a screen 5804, one or more front buttons 5806, a pair of ultrasonic input devices 5808, and a separate ultrasonic input device 5810. The electronic device 5800 may include a processor, a memory, and a network interface. In some embodiments, the ultrasonic input device may be coupled to the processor of the electronic device 5800.

[0373] In some embodiments, the pair of ultrasonic input devices 5808 can define an input touch area 5812 to detect user input. For example, a user can contact the input touch area 5812 to adjust the volume, brightness, etc. of the electronic device. In some embodiments, an array of ultrasonic input devices can be positioned below the screen or other locations such as the side or back of the electronic device to detect touch input and replace or enhance the capacitive touch or force touch capabilities or mechanical buttons of the electronic device. A single ultrasonic input device 5810 can define an input touch area 5814 to detect user input. The input touch area 5814 can be configured to control device power, screen on / off, etc.

[0374] In some embodiments, an ultrasonic input device can be used to detect touch input at each of the one or more front buttons 5806. The ultrasonic input device can replace capacitive sensing for detecting touch input on the fingerprint sensor. The ultrasonic input device provides a low-power solution to detect touch input on the fingerprint sensor. In some embodiments, one or more ultrasonic input devices can be positioned below the logo 5822 on the back 5820 of the housing 5802 to detect user input. They can also be placed below the side of the electronic device to replace the commonly used side mechanical buttons, such as for power or volume.

[0375] Fig.59 5 is a schematic diagram of a steering wheel 5902 with an ultrasonic input device 5904 according to certain aspects of the present disclosure. The ultrasonic input device 5904 can be used to form a touch input area on the steering wheel 5902 to detect touch input. The flexibility of the ultrasonic input device 5904 facilitates the detection of touch input through a variety of materials used to make steering wheels (such as plastic, leather, wood, etc.). A cross-section 5906 of the steering wheel 5902 shows the ultrasonic input device coupled to a surface 5908 to form a touch input area 5910. The touch input area can be combined with multiple touch input areas for applications such as cruise control, infotainment input control, cellular communication control, volume, and driver detection systems. For example, the ultrasonic input device 5904 can be used in a driver detection system to determine whether the driver's hand is in contact with the steering wheel.

[0376] Fig.60 6000 using an ultrasonic input device according to certain aspects of the present disclosure. The shapes and materials that can be used to design the touch area under which the ultrasonic input device is located are limited only by the creativity of the designer. Fig.60 A 12-key standard telephone keypad is shown in FIG. The keypad 6000 may include 12 ultrasonic input devices 6002 to form a touch area 6004 for each key. As another example, the keypad 6000 may include 23 or fewer ultrasonic input devices 6002.

[0377] Fig.61 A schematic diagram according to certain aspects of the present disclosure depicts a robotic arm using an ultrasonic input device. The robotic arm 6100 may include a first finger 6102 and a second finger 6104. The ultrasonic input device may be implemented as a robotic finger input device. The first finger 6102 and the second finger 6104 may include a first ultrasonic input device 6106 and a second ultrasonic input device 6108, respectively. The first ultrasonic input device 6106 may form a contact area 6110 on the surface of the first finger 6102, and the second ultrasonic input device 6108 may form a second contact area 6112 on the second finger. The ultrasonic input devices improve the detection capabilities of the robotic arm because they can be integrated into fingers comprising any material. In addition, the ultrasonic input device can detect touch input without the need for cutouts and / or different materials integrated into the fingers.

[0378] In some cases, the ultrasonic input device can identify the type of material being touched by analyzing the energy measurement signal. In some cases, the ultrasonic input device can identify the elasticity of the object being grasped. For example, an object with less elasticity will generally absorb less ultrasound than an object with greater elasticity, thus causing an overall higher energy measurement signal. In some cases, the determination of the elasticity of the object can be used to adjust the behavior of the robot arm, such as adjusting the force with which the robot arm grasps the object. In some cases, the ultrasonic input device is capable of detecting the texture or other mechanical properties of the object based on analyzing the energy measurement signal associated with the object. In some cases, the analysis of the energy measurement signal from the ultrasonic input device can be combined with other inputs (such as machine vision) to confirm or make a determination about the object with which the robot arm will interact or is interacting.

[0379] Fig.626 is a schematic diagram according to certain aspects of the present disclosure, which depicts a piece of furniture 6202 using an ultrasonic input device 6204. The ultrasonic input device 6204 can be coupled to the furniture 6202 in any suitable manner. A user touching the furniture 6202 at or near the location of the ultrasonic input device 6204 can be detected by the ultrasonic input device 6204 (e.g., through an ultrasonic touch sensor 6212). Upon detecting the touch, the ultrasonic input device 6204 can perform any preprogrammed function. For example, a communication module 6214 of the ultrasonic input device 6204 can send a signal (e.g., a wireless signal) to a control module 6206 spaced apart from the ultrasonic input device 6204 and / or the furniture 6202. The control module 6206 can control another device, such as a power switch 6208 coupled to a light bulb 6210. Thus, when a location on the furniture 6202 located at or adjacent to the ultrasonic input device 6204 is pressed, the light bulb 6210 can be turned on, off, or otherwise controlled. The controlled device (e.g., light bulb 6210) can be in the same environment as the ultrasound input device 6204, although this need not always be the case. In some cases, the controlled device can be located in a neighboring environment or even in a remote environment.

[0380] In some embodiments, a piece of furniture or IoT can be equipped with one or more of these ultrasonic input devices, which can operate individually or in the form of a sensor network that communicates with each other to perform multiple tasks. The sensors can also communicate with other sensors on other devices to exchange information, either through the IoT device itself or through the sensor's general programmable processor.

[0381] The ultrasonic input device 6204 according to certain aspects of the present disclosure can operate on very low power, such as from an internal battery 6216. Such battery powered low power operation can allow the ultrasonic input device 6204 to be used in locations that are otherwise difficult to access or inconvenient. For example, a light switch can be incorporated into a table or desk, or a television remote control can be incorporated into the armrest of a chair.

[0382] In some cases, the ultrasonic input device 6204 can be positioned on a hidden surface 6218 so as to hide the ultrasonic input device 6204 from view during normal operation. The hidden surface 6218 can be the underside of a table (e.g., furniture 6202), the inside of a piece of furniture, the inside of a wall, or any other suitable location hidden from view. Thus, the hidden ultrasonic input device can only be actuated by those who know its location, and it will be hidden from view for others.

[0383] Fig.63is a set of graphs according to certain aspects of the present disclosure, which depict energy measurement signals of ultrasonic input devices demonstrating material detection. Characteristics of the energy measurement signal, such as shape, duration, slope or other characteristics, can be used to determine the material interacting with the ultrasonic input device. For example, a bare human finger can cause a different energy measurement signal than a human finger wearing a plastic glove. The top graph depicts an example of an energy measurement signal from a bare human finger contacting an ultrasonic input device, where the characteristics drop rapidly and increase back to the baseline signal relatively quickly when the finger is removed. However, the bottom graph depicts an example of an energy measurement signal from a human finger wearing a plastic glove. When the plastic glove is worn, the energy measurement signal has different characteristics than when the plastic glove is not worn. For example, when the plastic glove is worn, the energy measurement signal increases discernibly slower to return to the baseline signal after the finger is removed. This relatively slow increase is due in part to the slight stickiness between the interaction of the plastic glove with the substrate of the ultrasonic input device. Although the human finger can be pulled apart faster, the plastic glove around the finger will tend to remain on the substrate for a short duration, which is discernible in the resulting energy measurement signal. This type of comparison can be used to determine the material that initiated the touch event.

[0384] XI. Additional Piezoelectric Array Designs

[0385] Fig.64 6400 is a schematic diagram of a piezoelectric resonator array 6400 including piezoelectric cantilevers 6402 that can be used for an ultrasound input device according to certain aspects of the present disclosure. The piezoelectric resonator array 6400 can include a set of piezoelectric cantilevers 6402 on a substrate 6404. The piezoelectric resonator array 6400 can be acoustically coupled to a material layer (e.g., Figure 1 The piezoelectric cantilever 6402 may be driven to bend and thereby cause an emitted signal.

[0386] Fig.65 65 is a schematic diagram of a piezoelectric resonator array 6500 including piezoelectric posts 6502 that can be used for an ultrasound input device according to certain aspects of the present disclosure. The piezoelectric resonator array 6500 can include a set of piezoelectric posts 6502 on a substrate 6504. When acoustically coupled to a material layer (e.g., Figure 1When a touch event is occurring, the touch event may cause the piezoelectric resonator array 6500 to resonate differently. This change in acoustic resonance due to the touch event may be detected and used as a sensor signal in an ultrasonic input device, such as in place of a PMUT. In addition, the piezoelectric posts 6502 may be driven to bend and thereby cause an emitted signal. The piezoelectric posts 6502 may be arranged in any suitable pattern, such as a hexagonal grid.

[0387] Aspects of the embodiments may be implemented in a modular or integrated manner using hardware circuits (e.g., application specific integrated circuits or field programmable gate arrays) and / or using computer software with a generally programmable processor in the form of control logic. As used herein, the processor may include a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units and dedicated hardware on a single circuit board or networked. Based on the disclosure and teachings provided herein, those of ordinary skill in the art will know and understand other ways and / or methods of implementing embodiments of the present invention using hardware and combinations of hardware and software.

[0388] Any software component or function described in this application can be implemented as a software code, which can be executed by a processor using any suitable computer language (e.g., Java, C, C++, C#, Objective-C, Swift or scripting language, such as Perl or Python using, for example, conventional or object-oriented techniques). The software code can be stored on a computer-readable medium as a series of instructions or commands for storage and / or transmission. Suitable non-temporary computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media (such as a hard drive or floppy disk) or optical media (such as a compact disc (CD) or DVD (digital versatile disc)), flash memory, etc. Computer-readable media can be any combination of such storage or transmission devices.

[0389] Such program can also be encoded and transmitted using carrier signals, which are suitable for transmission through wired, optical and / or wireless networks (including the Internet) that meet multiple protocols. Therefore, computer-readable media can be set up using data signals utilizing such program encoding. Computer-readable media utilizing program code encoding can be packaged with compatible devices or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable media can be present on or in a single computer product (e.g., hard drive, CD or entire computer system), and can be present on or in different computer products within a system or network. A computer system may include a monitor, printer or other suitable display, to provide any result mentioned herein to a user.

[0390] Any method described herein can be performed in whole or in part using a computer system, and the computer system includes one or more processors that can be configured to perform a plurality of steps. Therefore, an embodiment can relate to a computer system configured to perform the steps of any method described herein, which potentially utilizes different components to perform corresponding steps or corresponding step groups. Although the steps of the method herein are presented as numbered steps, they can be performed simultaneously or at different times or in different orders. In addition, a plurality of parts of these steps can be used in conjunction with a plurality of parts of other steps of other methods. And all or a plurality of parts of the step can be optional. In addition, any step of any method can be performed using the module, unit, circuit or other means that perform these steps.

[0391] The specific details of the specific embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the invention. However, other embodiments of the invention may be directed to specific embodiments relating to each individual aspect or specific combinations of these individual aspects.

[0392] The above description of example embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the above teaching.

[0393] Unless specifically indicated to the contrary, the reference to "a," "an," or "the" means "one or more." Unless specifically indicated to the contrary, the use of "or" means "inclusive-or," not "exclusive-or." Reference to a "first" component does not necessarily require the presence of a second component. Furthermore, reference to a "first" or "second" component does not limit the referenced components to specific locations unless explicitly stated. The term "based on" is intended to mean "based at least in part on."

[0394] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. No admission is made that they are prior art.

Claims

1. A method for touch input detection, the method comprising: transmitting, by a transducer coupled to a first surface of the material layer, an ultrasonic signal directed toward a second surface of the material layer, the material layer having a distance between the first surface and the second surface; detecting the reflected ultrasonic signal by the transducer; determining the amplitude of the reflected ultrasonic signal; determining that the amplitude exceeds a threshold associated with a portion of the ultrasound signal that penetrates the second surface; generating a signal indicating a touch input on the second surface if the amplitude exceeds the threshold; detecting, by the transducer, an additional reflected ultrasonic signal, wherein the additional reflected ultrasonic signal is associated with an echo of the reflected ultrasonic signal; determining an additional amplitude of the additional reflected ultrasonic signal; and The amplitude is updated by adding the additional amplitude before determining that the amplitude exceeds the threshold.

2. The method according to claim 1, wherein: The ultrasonic signal comprises a first frequency and a second frequency, and the method further comprises: determining a first amplitude associated with the first frequency and a second amplitude associated with the second frequency; determining that the first amplitude and the second amplitude are less than the threshold; and In the event that the first amplitude and the second amplitude are less than the threshold, a signal is generated indicating an input touch on the second surface.

3. The method according to claim 1, wherein: The material layer includes at least one or more of glass, metal, leather, wood, ceramic, plastic and stone.

4. The method according to claim 1, wherein: Determining the amplitude of the reflected ultrasound signal comprises one of the following steps: rectifying and integrating the detected ultrasonic signal; Accumulating the absolute values ​​of the detected ultrasonic signals; and The detected ultrasonic signal is squared and integrated.

5. The method according to claim 4, wherein: Determining the amplitude of the reflected ultrasonic signal further includes applying an energy measurement window envelope to the reflected ultrasonic signal.

6. The method according to claim 4, wherein: Determining the amplitude of the reflected ultrasonic signal includes accumulating absolute values ​​of the detected ultrasonic signal, and wherein accumulating absolute values ​​of the detected ultrasonic signal includes applying a negative clocked DC charge to a switched capacitor analog accumulator.

7. The method according to claim 1, wherein: Determining the amplitude of the reflected ultrasonic signal is performed by analog circuitry coupled to an application specific integrated circuit of the transducer.

8. The method according to claim 1, further comprising: Calculating a moving average of the amplitude of the reflected ultrasonic signal; and The threshold is calculated based on the moving average.

9. A method for touch input detection, the method comprising: transmitting an ultrasonic signal directed toward a second surface of the material layer by a transducer, the transducer being coupled to a first surface of the material layer, the material layer having a distance between the first surface and the second surface; detecting the reflected ultrasonic signal by the transducer; determining an energy value associated with the reflected ultrasound signal; determining that the energy value exceeds a threshold associated with a portion of the ultrasound signal that penetrates the second surface; generating a signal indicative of a touch input on the second surface if the energy value exceeds the threshold; detecting, by the transducer, an additional reflected ultrasonic signal, wherein the additional reflected ultrasonic signal is associated with an echo of the reflected ultrasonic signal; determining an additional energy value of the additional reflected ultrasonic signal; and The energy value is updated by adding the additional energy value before determining that the energy value exceeds the threshold.

10. The method according to claim 9, wherein: Determining the energy value is performed by analog circuitry coupled to an application specific integrated circuit of the transducer.

11. The method according to claim 9, wherein: Determining the energy value includes: rectifying the reflected ultrasonic signal to provide a rectified signal; integrating a portion of the rectified signal to provide an integrated signal; and The integrated signal is measured by an analog-to-digital converter to determine the energy value.

12. The method according to claim 9, wherein: Determining the energy value includes: calculating the absolute value of the detected reflected ultrasonic signal; accumulating absolute values ​​associated with the detected reflected ultrasonic signals to provide an accumulated signal; and The accumulated signal is measured by an analog-to-digital converter to determine the energy value.

13. The method according to claim 9, wherein: Determining the energy value includes: squaring the detected reflected ultrasonic signal by passing the reflected ultrasonic signal through a self-mixing circuit to provide a squared signal; integrating the squared signal to provide an integrated signal; and The integrated signal is measured by an analog-to-digital converter to determine the energy value.

14. The method according to claim 9, further comprising: calculating a moving average of the energy values ​​of the reflected ultrasonic signal; as well as The threshold is calculated based on the moving average.

15. A device for touch input detection, comprising: a body, including an inner surface and an outer surface; a transducer coupled to the inner surface; as well as A circuit coupled to the transducer and configured to: emitting an ultrasonic signal directed toward the outer surface; detecting a set of at least one reflected ultrasonic signal using the transducer; determining an amplitude of the set of at least one reflected ultrasonic signal; determining that the amplitude exceeds a threshold, wherein the threshold is associated with a portion of the at least one ultrasound signal that penetrates the outer surface; generating a signal indicative of a touch input on the outer surface if the amplitude exceeds the threshold; detecting, by the transducer, an additional reflected ultrasonic signal, wherein the additional reflected ultrasonic signal is associated with an echo of the reflected ultrasonic signal; determining an additional amplitude of the additional reflected ultrasonic signal; and The amplitude is updated by adding the additional amplitude before determining that the amplitude exceeds the threshold.

16. The device according to claim 15, wherein: The device comprises a mobile phone.

17. The device according to claim 15, wherein: The device includes at least one or more of a steering wheel, an infotainment input, a center console control, and a keypad.

18. The device according to claim 15, wherein: The main body includes at least one or more of glass, metal, leather, wood and stone.

19. The device according to claim 15, wherein: The device includes a robotic finger input device.

20. The device according to claim 15, wherein: The circuit includes one or more processors and a memory coupled to the transducer, wherein the memory includes a plurality of instructions for detecting the touch input, and the plurality of instructions, when executed by the one or more processors, cause the device to perform the steps that the circuit is configured to perform.

21. The apparatus of claim 15, further comprising an additional transducer coupled to the inner surface, wherein: The circuit is also coupled to the additional transducer and is configured to: emitting an additional ultrasonic signal directed toward the outer surface; detecting, using the additional transducer, an additional reflected ultrasonic signal associated with the additional ultrasonic signal; determining an additional amplitude of the additional reflected ultrasonic signal; determining that the additional amplitude exceeds an additional threshold, wherein the additional threshold is associated with a portion of the additional ultrasonic signal that penetrates the outer surface; and In case the additional amplitude exceeds the additional threshold and in case the amplitude exceeds the threshold, a signal is generated indicative of a touch input on the outer surface.

22. The device according to claim 21, wherein The additional ultrasound signal has a frequency different from that of the ultrasound signal.

23. The device according to claim 21, wherein The transducer and the additional transducer are concentric.

24. The device according to claim 21, wherein The transducer and the additional transducer are coupled to a single circuit board.

25. The device according to claim 15, wherein: The transducer is a piezoelectric micromechanical ultrasonic transducer.

26. The device according to claim 15, wherein: The transducer includes a plurality of channels including a set of transmit channels and at least one receive channel, wherein the at least one receive channel is positioned between a subset of the set of transmit channels.

27. The device according to claim 15, wherein: The circuit comprises an application specific integrated circuit including analog circuitry, and wherein determining the amplitude is performed using the analog circuitry of the application specific integrated circuit.

Images