Piezoelectric Micro-Electromechanical Systems (MEMS) Signal Processing for Contact Detection

Abstract

Aspects of this disclosure relate to micro-electromechanical systems (MEMS) and related detection and classification of surface impacts using MEMS systems and signals. One aspect is a device comprising a memory configured to store audio signals and motion signals, and one or more processors. The processors are configured to acquire audio signals, which are generated based on sound detection by a microphone; acquire motion signals, which are generated based on motion detection by a motion sensor mounted on the surface of an object; perform similarity measurements based on the audio and motion signals; and determine the context of the contact type on the surface of the object based on the similarity measurements.

Description

Technical Field

[0001] The present disclosure generally relates to piezoelectric acoustic transducers, and more particularly to piezoelectric microelectromechanical system (MEMS) vibration sensing devices that detect vibrations associated with an object's surface.

Background Art

[0002] MEMS technology has enabled the development of smaller microphones and other acoustic transducers using wafer deposition techniques. Generally, MEMS microphones can take various forms, including, for example, capacitive microphones and piezoelectric microphones. MEMS capacitive microphones and electric condenser microphones (ECMs) currently dominate the consumer microphone market. However, piezoelectric MEMS systems, such as microphones, are a growing market and offer various advantages. For example, piezoelectric MEMS microphones may not require a backplate to eliminate squeeze film damping (a source of inherent noise in capacitive MEMS microphones). Further, piezoelectric MEMS microphones are reflow compatible and can be mounted on a printed circuit board (PCB) using lead-free soldering processes that can cause irreparable damage to other types of microphones. These and other advantages can be more fully realized with improved piezoelectric MEMS microphones.

Summary of the Invention

[0003] Aspects of the present disclosure describe microelectromechanical system (MEMS) devices, systems, methods, and other aspects related to detecting and classifying contact with a surface.

[0004] In some embodiments, the techniques described herein relate to a device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors are configured to acquire audio signals based on sound detection by two or more microphones, acquire motion signals based on motion detection by two or more motion sensors mounted on the surface of a first object, perform one or more comparisons of the audio signals and motion signals to generate comparison data, determine the contact type context of the surface of the first object based on the comparison data, and determine the location of a second object within a threshold distance to the first object based on the context determined by the one or more processors.

[0005] In some embodiments, the techniques described herein relate to a device in which multiple cantilever beams are configured as a membrane surrounding a sensor area.

[0006] In some embodiments, the techniques described herein relate to a device in which the first object is a car door.

[0007] In some embodiments, the techniques described herein relate to devices in which the second object is a person, a key, or a balloon.

[0008] In some embodiments, the techniques described herein relate to a device that includes an area in which the contact type is associated with a first object.

[0009] In some embodiments, the techniques described herein relate to a device in which the first object is a door, and the area associated with the first object is one of the following: the upper right portion of the door, the lower right portion of the door, the upper left portion of the door, the lower left portion of the door, or the center of the door.

[0010] In some embodiments, the techniques described herein relate to a device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors are configured to acquire audio signals based on sound detection by a microphone, acquire motion signals based on motion detection by a motion sensor mounted on the surface of an object, normalize the audio signals and motion signals to generate normalized audio signals and normalized motion signals, generate correlation data from the normalized audio signals and normalized motion signals, and use the correlation data to determine contact classification.

[0011] In some embodiments, the techniques described herein relate to a device in which one or more processors are configured as a classification circuit that determines contact classification using correlation data.

[0012] In some embodiments, the techniques described herein relate to a device in which a classification circuit is further configured to receive audio signals and motion signals as training data in a training mode and to match the training data with provided training classification values.

[0013] In some embodiments, the techniques described herein relate to a device, where the object is an automotive surface, the surface is an outward-facing surface of the automotive surface, and the provided training classification values ​​are a set of collision classification values ​​having known force and damage type characteristics.

[0014] In some embodiments, the techniques described herein include a memory and a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, the first piezoelectric MEMS transducer being mechanically coupled to the surface of an object and configured to generate a first analog signal at the first output when the first analog signal is converted from vibrations propagating through the object by the first piezoelectric MEMS transducer, and a second piezoelectric MEMS transducer having a second output, the second piezoelectric MEMS transducer being configured to generate a second analog signal at the second analog signal. The present invention relates to a device comprising: a second piezoelectric MEMS converter configured to generate at a second output when a signal is converted from acoustic vibrations at the location of an object by a second piezoelectric MEMS converter; and a memory, the output of the first piezoelectric MEMS converter, and a classification circuit coupled to the output of the second piezoelectric MEMS converter, the classification circuit configured to process data from a first analog signal and data from a second analog signal, and to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0015] In some embodiments, the techniques described herein relate to a device that further includes storing categorization data from a classification circuit into memory.

[0016] In some embodiments, the techniques described herein relate to a device in which a first piezoelectric MEMS transducer has a noise floor that defines noise at a given frequency related to a signal output in gravitational units (g), and the noise floor is between 100 parts per million of gravitational units (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 μg / sqrt(Hz).

[0017] In some embodiments, the techniques described herein relate to a device in which a first piezoelectric MEMS converter has a conversion bandwidth for detecting vibrations propagating through an object at frequencies between 0.5 kilohertz (kHz) and 15 kHz.

[0018] In some embodiments, the techniques described herein relate to a device in which data from a first analog signal includes frequency data of vibrations propagating through an object and magnitude data of vibrations propagating through an object, the magnitude data being associated with the severity of contact with the object.

[0019] In some embodiments, the techniques described herein relate to a device in which one or more time frames are captured at a rate greater than 60 frames per second.

[0020] In some embodiments, the techniques described herein further include a first sensor package, the first sensor package including a substrate base and a lid, and a device comprising a first piezoelectric MEMS transducer, a second piezoelectric MEMS transducer, and an application-specific integrated circuit (ASIC) mounted on the substrate base.

[0021] In some embodiments, the techniques described herein relate to a device in which the ASIC includes an analog-to-digital converter (ADC), a digital signal processor (DSP), and a controller, wherein the output of the ADC is coupled to the input of the controller via the digital signal processor.

[0022] In some embodiments, the techniques described herein further include a device comprising a second sensor package including a third MEMS transducer and a fourth MEMS transducer, wherein the first sensor package is positioned at a first location on the surface of an object, and the second sensor package is positioned at a second location on the surface of the object at a predetermined distance from the first location.

[0023] In some embodiments, the techniques described herein relate to a device in which the classification circuit is further configured to detect the location of an impact on the surface of an object based on a time delay or magnitude difference between vibrations detected in a first sensor package and vibrations detected in a second sensor package within the same time frame.

[0024] The above matters will become clearer, together with other features and embodiments, by referring to the following description, claims, and attached drawings.

Brief Description of the Drawings

[0025] [Figure 1A] FIG. showing an example of an acoustic transducer system for contact detection and classification according to the aspects described herein. [Figure 1B] FIG. showing aspects of a piezoelectric microelectromechanical system (MEMS) sensor system according to the aspects described herein. [Figure 1C] FIG. showing aspects of a piezoelectric MEMS sensor device according to the aspects described herein. [Figure 1D] FIG. showing aspects of a piezoelectric MEMS sensor device according to the aspects described herein. [Figure 1E] FIG. showing aspects of a piezoelectric MEMS sensor device according to the aspects described herein. [Figure 2] FIG. is a plan view of a piezoelectric MEMS transducer that can be used according to the aspects described herein. [Figure 3] FIG. is a cross-sectional view of a portion of a piezoelectric MEMS beam that can be used according to the aspects described herein. [Figure 4] FIG. is an isometric view of a piezoelectric MEMS beam that can be used according to the aspects described herein. [Figure 5] FIG. showing aspects of a system including a piezoelectric MEMS transducer according to the aspects described herein. [Figure 6A] FIG. showing aspects of a system including a piezoelectric MEMS transducer according to the aspects described herein. [Figure 6B] FIG. showing aspects of a system including one or more arrays of piezoelectric MEMS transducers according to the aspects described herein. [Figure 7] FIG. showing a method related to contact detection and classification using a MEMS transducer according to the aspects described herein. [Figure 8A] This figure shows a method related to contact detection and classification using a MEMS transducer, according to an embodiment described herein. [Figure 8B] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 8C] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 9] This figure shows a method related to contact detection and classification using a MEMS transducer, according to an embodiment described herein. [Figure 10A] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 10B] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 10C] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 10D] This figure shows an embodiment of contact detection and classification using a MEMS converter, as described herein. [Figure 11] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 12] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 13] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 14] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 15] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 16]This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 17] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 18] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 19] This figure shows an embodiment of a system that supports contact detection and classification using a MEMS converter, according to the embodiments described herein. [Figure 20] This is a functional block diagram of a piezoelectric MEMS contact detection and classification system according to the embodiments described herein. [Figure 21] This figure shows some embodiments of the machine learning engine described herein. [Figure 22] This is a block diagram of a computing device that may be used in conjunction with an implementation of a piezoelectric MEMS contact detection and classification system according to the embodiments described herein.

[0026] Similar reference numerals in various drawings indicate the same elements. [Modes for carrying out the invention]

[0027] In relation to the attached drawings, the “Modes for Carrying Out the Invention” described below are intended to describe exemplary embodiments and implementations and are not intended to represent the only possible implementations of the invention. The “Modes for Carrying Out the Invention” include specific details intended to provide a complete understanding of the exemplary embodiments and implementations. In some cases, some devices are shown in block diagram form. Drawing elements common to the following figures can be identified using the same reference numerals.

[0028] Embodiments described herein include contact detection and classification systems using piezoelectric microelectromechanical (MEMS) transducers. Such transducers convert kinetic energy into electrical signals. An example of a MEMS transducer is a MEMS microphone, which converts sound pressure into voltage. Another example of a MEMS transducer is a motion detector, which converts motion into voltage. The size and low power associated with such MEMS transducers can enable their use in environments where other such sensors are unavailable. Embodiments described herein include systems for detecting vibrations associated with the surface of an object in order to detect and classify surface contacts associated with detected vibrations.

[0029] Some embodiments include a combination of piezoelectric MEMS acoustic and piezoelectric MEMS motion detectors coupled to the surface of an object to detect incident motion (e.g., mechanical vibration) and sound (e.g., acoustic vibration). Data derived from the electrical signals output by the MEMS detectors can, in some embodiments, be processed by a classifier or machine learning engine to generate additional system actions associated with a given type of signal. For example, the system may include data patterns corresponding to surface contact associated with collisions or key scratch paint on the surface of an object. In some embodiments, such data patterns can be generated by a machine learning system connected to the surface of an object, which can record data generated by specific actions (e.g., key scratching a car door, collision with a car bumper, etc.). The data can be used to train a classifier, neural network, or other such machine learning engine. A device can then be created having sensors in the same arrangement within a similar object used for data generation (e.g., a mass-produced car door). Electrical connections from the sensors within the object to control and processing circuits can be used to generate warnings or actions based on the classification of the sensed vibrations.

[0030] In some embodiments, multiple piezoelectric MEMS transducers of the same type (e.g., multiple microphones and multiple motion detectors) may be arranged at different locations on a surface. The time differences and other variations of the signals detected by each MEMS transducer can be used to determine where on the surface of the object contact occurred (e.g., based on time delays, amplitude variations, or other differences between electrical signals generated from the same contact).

[0031] In some embodiments, the contact sensing system is configured to distinguish and / or characterize different types of contact on a surface. To this end, some embodiments include a motion detector and a microphone that, in combination with a machine learning engine, produce the desired results. In some embodiments, the motion detector has a low noise floor, a high bandwidth (e.g., a wide bandwidth of detected vibration frequencies), or both. In some embodiments, these elements are formed on a shared die. In other embodiments, the elements are formed on separate dies. Further details of various exemplary embodiments are described below.

[0032] Figure 1A shows an example of a system for contact detection and classification using a MEMS transducer according to an embodiment described herein. Figure 1A schematically shows a cross-sectional view of an acoustic sensor 10A. As shown, the sensor 10A of Figure 1A includes a MEMS chip 12 which may include a die having a piezoelectric structure 14 (e.g., a cantilever or diaphragm that converts vibration energy into an electrical signal), and an application-specific integrated circuit (ASIC) chip 16 that buffers and amplifies the electrical signal generated by the MEMS chip 12. The MEMS chip 12 and the ASIC chip 16 are electrically connected by wire bonding 18 and mounted in an internal chamber of a package (although other packaging and connection techniques are also possible). The package has a lid 28 and a substrate 22 (e.g., a printed circuit board). The PCB substrate 22 and the MEMS substrate of the MEMS chip 12 form an acoustic port 24 that allows sound pressure to access the piezoelectric structure(s) 14 of the MEMS chip 12. Multiple solder pads 26 are arranged on the bottom surface of the PCB board 22 for soldering the MEMS transducer 10 as an additional device element. The MEMS transducer of the MEMS chip 12 can be used as a microphone or other sensor in, for example, a mobile phone, laptop computer, portable microphone, smart home accessory, or any other such device. The lid 28 forms the housing of the MEMS chip 12 and can be used to reduce electromagnetic interference (EMI) by providing an air pocket that provides one side of the air pressure difference causing deflection and signal generation within the MEMS chip 12. As described above, in some embodiments, the sensor 10 may be implemented without an acoustic port 24 to implement an accelerometer, and the piezoelectric structure 14 generates an electrical signal based on the movement of the MEMS transducer 10 rather than based on an incident acoustic (e.g., ultrasonic) signal from the acoustic port 24.

[0033] Figure 1A shows a structure comprising a MEMS chip 12 having an acoustic port 24 formed on a MEMS substrate. In other embodiments, the MEMS substrate can be closed off by a pocket similar to the pocket formed by the cavity beneath the piezoelectric structure 14 and by the acoustic port 24 on the opposite side of the piezoelectric structure(s) 14 from the substrate 22. In other embodiments, other such configurations of the acoustic port 24 can be used, as long as there is a path for sound pressure to reach the piezoelectric structure 14.

[0034] Figure 1A further illustrates the machine learning engine 7 and control circuit 8 coupled to the ASIC chip 16 via a data path 9. In some embodiments, the machine learning engine 7 may be a neural network or classification circuit separate from additional processing circuits of the system, such as the control circuit 8. In some embodiments, the machine learning engine 7 may include a decision tree circuit, a support vector machine circuit, a convolutional neural network circuit, or other such classification or contact detection and characterization circuit. In some embodiments, the control circuit 8 and the machine learning engine 7 may be implemented using one or more processors of a device using shared resources within a computing architecture as shown in Figure 11, where the control circuit and the machine learning engine are implemented by processor 1110, and sensor 10A functions as input device 945. The machine learning engine 7 can process signals output from the ASIC chip 16 (e.g., generated from analog signals provided by the MEMS chip 12) to determine the type of motion detected by the MEMS chip 12. The classification or type determination performed by the machine learning engine can generate an output provided to the control circuit 8. The control circuit 8 can then perform a selected action based on the classification information provided by the machine learning engine 7.

[0035] In some embodiments, rather than implementing the system using two separate chips, some embodiments may implement both the MEMS chip 12 and the ASIC 16 as part of the same die. Therefore, the description of separate chips is illustrative. Furthermore, in other embodiments, the ASIC 16 may be implemented on a die in a separate package having one or more interconnections electrically coupling the MEMS chip 12 to the ASIC 16. Similarly, the amplifier used for feedback conversion in the feedback conversion loop described above may, in some embodiments, be implemented on an ASIC 16 separate from the MEMS chip 12. In other embodiments, the amplifier may be implemented as part of a composite IC having both MEMS and ASIC components of the MEMS chip 12 and the ASIC 16.

[0036] Furthermore, as shown below, the sensor can be implemented using multiple piezoelectric MEMS transducers, either on a single MEMS chip or on separate MEMS chips.

[0037] Figure 1B shows an embodiment of a piezoelectric microelectromechanical system (MEMS) sensor 10B according to the embodiments described herein. As shown, the sensor 10B includes a piezoelectric MEMS converter 5. The piezoelectric MEMS converter can be mounted on a MEMS chip such as the MEMS chip 12 in Figure 1A. The output of the converter 5 is coupled to an analog-to-digital converter (ADC) 54, which receives an analog signal from the output of the converter 5 and converts the analog signal (e.g., a converted signal from motion vibration detected by the piezoelectric MEMS converter 5) into a digital signal. The output of the ADC 54 is provided to a digital signal processor (DSP) 56, which can perform preprocessing, digital filtering, or other signal adjustments on the information from the converter 5 and provide the output signal to a controller 58. The controller 58 can further process the information from the converter 5 to generate a digital data signal corresponding to the analog signal output from the converter 5. The digital data signal can be stored in the memory 60 on the sensor 10B, or output to the data path 9 via the application-specific integrated circuit (ASIC) input / output (I / O) circuit 62.

[0038] As shown in the figure, the transducer 5 does not have the associated acoustic port 24. In some embodiments, a similar MEMS chip 12 as shown in Figure 1A can be used for the transducer 5 acting as a motion detector, but the associated sensor does not have the acoustic port 24. In such embodiments, the PCB substrate 22 can be closed without capping the acoustic port.

[0039] Figure 1C shows details of the MEMS transducer 10C according to the embodiments described herein. As shown, the transducer may include a transducer 6 having an acoustic port 24. Furthermore, in contrast to the embodiment of Figure 1B which has a motion detector transducer 5 configured only to receive vibration signals, the transducer 6 of the sensor 10C can transmit signals in addition to receiving them. The sensor 10C can allow sound waves to be transmitted from the transducer 6 in transmit mode or sensed in receive mode. A switching circuit 50 allows the controller 58 to select between receive (Rx) operation and transmit (Tx) operation. In Tx mode, the electrical signal associated with the sound wave generated by the transducer 6 is received as input at the ASIC input / output (I / O) 62 and passed to the controller 58. The signal (e.g., modified by the controller 58 to shape this signal for the transducer 6) can be stored in memory 60 for later use or passed to the Tx circuit 52 for transmission. As part of its transmission operation, the Tx circuit 52 can perform additional waveform adjustment and amplification (e.g., via a power amplifier) ​​before being transmitted to the converter 6 and converted into an acoustic signal.

[0040] In receive mode, the MEMS chip 12 receives incident sound waves via the acoustic port 24, and the incident sound waves are converted into electrical signals by the converter 6. Similar to the motion sensor converter 5 described above, the ADC 54 and DSP 56 convert the analog electrical signals from the MEMS chip 12 into a format acceptable to the controller 58, which can then store the signals in the memory 60 or transmit the signals to additional processing circuits of a larger device via the ASIC I / O 62.

[0041] As described herein, embodiments may include transducer signals for both acoustic vibrations (e.g., microphones) and mechanical vibrations (e.g., motion sensors) used to detect and classify contact with a surface. In some embodiments, separate sensors 10 can be used for acoustic detection and motion detection. Such embodiments may include separate packages placed together on the surface of an object to generate analog signals and corresponding data associated with similar locations on the surface of the object. In other embodiments, a shared package can be used for multiple transducers (e.g., on a shared PCB board such as a PCB board 22 having the same lid, such as a lid 28).

[0042] Figure 1D shows an embodiment of the piezoelectric MEMS sensor 10D according to the embodiments described herein. The sensor 10D includes two transducers, indicated as transducer 6 and transducer 5. Transducer 6 may be a microphone that receives an acoustic signal via the acoustic port 24, and transducer 5 may be a motion detector that is not exposed to the acoustic port as described above. In some embodiments, transducers 5 and 6 may be mounted on a single MEMS chip, such as MEMS chip 12. In other embodiments, multiple different MEMS chips may be used. For example, in one embodiment, two MEMS chips may be arranged on a shared substrate under a shared cover, as illustrated in Figure 1A, but having a second MEMS chip in addition to MEMS chip 1A. Such embodiments may also include multiple ASICs, or a single ASIC may be used to process analog signals from multiple transducers. Figure 1D shows the sensor 10D with two transducers 6 and 5. Other embodiments may include additional transducers, such as transducers for different frequency ranges (e.g., two or more microphones that detect different acoustic frequency ranges, or two or more motion sensors that detect different ranges of mechanical vibration frequencies).

[0043] Figure 1E shows an embodiment of the piezoelectric MEMS sensor 10E according to the embodiments described herein. Figure 1E schematically shows further details of the sensor described above. Sensor 10E shows an implementation embodiment having a single MEMS chip 12E (for example, an implementation embodiment of MEMS chip 12 in Figure 1A) which includes a first die having a motion sensor (e.g., a first piezoelectric MEMS transducer such as transducer 5) configured to detect motion, a microphone (e.g., a second piezoelectric MEMS transducer such as transducer 6) configured to detect sound, and a second die implementing a machine learning engine in a separate ASIC chip 16E. The ASIC chip 16E is configured to use data from the microphone and the motion detector to determine information about contact (e.g., an impact on the surface of an object including or attached to sensor 10E). Sensor 10E can be implemented in a package having a base (e.g., PCB substrate 22) on which all three components are mounted. Thus, the motion sensor should detect the motion of the surface on which it is fixed. However, alternative embodiments may use two or more packages to form a single sensor (e.g., on a printed circuit board). For example, the motion sensor and microphone may be in a first package, while the machine learning engine implemented using the ASIC chip 16E may be in a second package. Other embodiments may divide all three elements into three different packages. As described herein, the MEMS chip 12E may be a shared MEMS die (e.g., a MEMS chip 12 having a microphone and a motion detector mounted on a PCB substrate 22 configured as a package substrate). Such a configuration, having two sensors on the same die and a machine learning engine (e.g., ML engine 7) integrated on an ASIC chip (e.g., ASIC chip 16), provides a device with an improved compact form factor compared to a device in which each component is configured on separate chips (e.g., two separate MEMS sensors, a separate ASIC, and a separate ML IC).

[0044] Figure 2 shows a plan view of a piezoelectric MEMS transducer that may be used according to the embodiments described herein. Figure 2 schematically shows a plan view of a piezoelectric MEMS acoustic transducer of a MEMS chip (e.g., MEMS chip 12) using eight MEMS cantilevers (e.g., also known as “sensing arms,” “sensing members,” “beams,” or “cantilevered beams”) formed as piezoelectric triangular cantilevers 30. Together these members form an octagonal MEMS transducer that can be used to mount a microphone (e.g., having associated acoustic ports) or a motion sensor (e.g., without associated acoustic ports). In the embodiments described above, one implementation of the transducer in Figure 2 may mount transducer 5, and a second implementation of the transducer in Figure 2 may mount transducer 6. In other embodiments, different transducer designs may be used for acoustic or mechanical signal detection.

[0045] In Figure 2, each cantilever 30 has a piezoelectric structure formed in the piezoelectric layer 34, and the structure of each of the eight cantilevers 30 has an associated fixed end and an associated central end. The central ends of each cantilever 30 in Figure 2 intersect near the center, and the edges of each cantilever 30 are separated from adjacent cantilevers by a pap between the cantilevers 30, as shown. During operation, the fixed end remains stationary, and pressure from an acoustic signal incident on the cantilever 30 (e.g., from the acoustic port 24) causes a pressure difference, which deflects the cantilever 30 inward and outward (e.g., via a slight rotation around the fixed end). This deflection generates an electrical signal from the sensing electrodes 36 / 38, which can be amplified by the analog front end and sent to the processing circuit as an audio signal. Mechanical electrodes 36 / 40 provide a mechanical structure at the central end of each cantilever 30.

[0046] Each cantilever 30 is positioned using a side adjacent to the side of another cantilever beam among the cantilevers separated by a gap between the cantilevers. The positions of the eight cantilevers 30 with gaps form a symmetrical polygonal shape (e.g., an octagon with one outer side for each of the cantilevers 30) bounded by a fixed base around the outer edge of the symmetrical polygon. In other embodiments, other shapes may be used. In other embodiments, the MEMS acoustic transducer may include cantilevers with different beam shapes for the same transducer, insofar as a fixed outer edge attached to the substrate uses the cantilevers (e.g., cantilevers 30) and the gaps between the beams to form a sealed transducer that separates the air on one side (e.g., the pocket side) from the air on the other side (e.g., the acoustic port side, similar to the acoustic port 24). This separation allows the pressure difference between the two sides of the MEMS transducer to force the beam and generate a signal that can be communicated to the analog front end and then to additional processing circuits via the bond pad 48. Similarly, an electrical signal provided from a transmitting circuit (for example, the Tx circuit 52 in Figure 1C) can deflect the cantilever 30 to generate an acoustic signal.

[0047] As shown in Figure 2, the cantilever 30 has a corresponding length determined by a line segment from the tip of the central end perpendicular to the fixed outermost end of the fixed end. The line segment extends from the fixed end of the substrate to the tip of the central end. As described above, when sound vibrations are present on the surface of the deflection beam, the cantilever moves due to pressure (e.g., z-direction movement in and out of the xy-plane shown in Figure 2). This in-and-out movement in this plane is referred to herein as vertical deflection. The deflection at the fixed end is smaller than the deflection at the central end, and the amount of deflection increases along the distance of the line segment away from the substrate towards the tip of the central end. Electrodes that generate electrical signals at the bond pad 48 in response to acoustic vibrations on the cantilever 30 can add rigidity to the cantilever 30, and therefore, in some implementations, the placement of the sensing electrodes 36 / 38 may be limited to a space of about two-thirds of the line segment distance from the fixed mounting to the substrate at the fixed end to the tip of the central end (e.g., limited to the fixed end). In some implementations, the electrode layer can cover the entire surface or xy-plane cross section of each illustrated fixed end of the cantilever. In other embodiments, a smaller electrode shape can be used for a portion of each fixed end of the cantilever 30. In some embodiments, the central end of each cantilever does not include an electrode layer. In some embodiments, the electrode layer does not extend to the tip of the central end (e.g., the free-moving end) of each cantilever 30 to avoid sensing lower movement of the free end at the deflection end (e.g., a signal proportional to the stress in the cantilever).

[0048] Figure 3 shows a cross-sectional view of a portion of the MEMS microphone shown in Figure 2, according to an embodiment described herein. Figure 3 shows an exemplary cross-sectional view of one of these cantilevers 30. Other embodiments of the piezoelectric MEMS acoustic transducer may use more or fewer cantilevers 30. Therefore, as with other features, the description of the eight cantilevers 30 is for illustrative purposes only. These triangular cantilevers 30 have their bases fixed to the substrate 50 (e.g., a silicon substrate) and are configured to move freely in response to the incoming / incident sound pressure (i.e., sound wave). The intersection of the substrate 50 and the piezoelectric layer (e.g., as well as the electrodes of the substrate 50) is the fixed end of the cantilever 30(s). A triangular cantilever 30 can offer advantages over a rectangular cantilever because it can be more easily configured to form a gap-control shape that isolates the acoustic port on one side of the cantilever of the piezoelectric MEMS acoustic transducer (e.g., acoustic port 24) from the air pocket on the other side of the cantilever. Specifically, when the cantilever 30 bends up and down due to either sound pressure or residual stress, the gap between adjacent cantilevers 30 remains relatively small and uniform, typically in exemplary symmetrical configurations with fixed ends using triangular cantilevers 30.

[0049] Electrode 36 is identified collectively by reference number 36. However, the electrode used to sense the signal is called the “sensing electrode” and is identified by reference number 38. These electrodes are electrically connected in series to achieve the desired capacitance and sensitivity values. In addition to the sensing electrode 38, the rest of the cantilever 30 may also be covered with metal to maintain a certain mechanical strength of the structure. However, these “mechanical electrodes 40” do not contribute to the electrical signal of the microphone output. As described above, some embodiments may include a cantilever 30 without mechanical electrodes 40.

[0050] As described above, when the cantilever 30 bends or flexes around the fixed end, the sensing electrodes 36 / 38 generate an electrical signal. The electrical signal from an upward bend (for example, relative to the position shown in Figure 3) is inverted compared to the signal from a downward bend. In some embodiments, the signals from each cantilever 30 of the piezoelectric MEMS acoustic transducer may be connected to the same signal path (e.g., a shared bond pad 48) so that the electrical signals from each cantilever 30 are combined. In other embodiments, each cantilever 30 may have a separate signal path that allows the signals from each cantilever 30 to be processed separately. In some embodiments, groups of cantilevers 30 may be connected in different combinations. In some embodiments, a switching circuit or group of switches may be used to reconfigure the connections between multiple cantilevers 30 to provide different characteristics for different operating modes, such as transmit mode and receive mode.

[0051] In one embodiment, adjacent cantilevers 30 can be connected to separate electrical paths such that every other cantilever 30 has a shared path. The electrical connections in such a configuration can be inverted to generate differential signals. In this embodiment, when an acoustic signal incident on the piezoelectric MEMS acoustic transducer bends all the cantilevers 30 upward, half of the cantilevers 30 can generate a positive signal and the other half cantilevers 30 can generate a negative signal. The two separate signals can be connected to opposite sides of the inverting and non-inverting ends of an amplifier in the analog front end. Similarly, when the same acoustic vibration bends the cantilevers 30 downward, the two groups of signals reverse polarity to provide differential electrical signals from the piezoelectric MEMS acoustic transducer.

[0052] Alternatively, instead of alternating the cantilever 30 within a single piezoelectric MEMS converter to generate a differential signal, multiple piezoelectric MEMS converters can be used to generate a differential signal by reversing the connections to the analog front-end amplifier and coupling the different inverting and non-inverting inputs of the analog front-end differential amplifier, with the same MEMS converters arranged across a shared acoustic port (e.g., acoustic port 24).

[0053] The cantilever 30 can be fabricated from one or more layers of piezoelectric material sandwiched between upper and lower metal electrodes 36. Figure 3 schematically shows an example of this structure. The piezoelectric layer 34 can be fabricated from one or more piezoelectric materials used in MEMS devices, such as aluminum nitride (AlN), scandium aluminum nitride (AlScN), zinc oxide (ZnO), and lead zirconate titanate (PZT). The electrodes 36 can be fabricated from metallic materials used in MEMS devices, such as one or more molybdenum (Mo), platinum (Pi), nickel (Ni), and aluminum (Al). Alternatively, the electrodes 36 can be formed from nonmetals, such as doped polysilicon. These electrodes 36 can cover only a portion of the cantilever 30, for example, up to about one-third from the base of the cantilever 30, because these regions generate electrical energy in the piezoelectric layer 34 more efficiently than regions near the central end (e.g., the free-moving end) of each cantilever 30. Specifically, the high stress concentration in these regions near the base, induced by the incident sound pressure, is directly converted into an electrical signal by the piezoelectric effect.

[0054] Figure 4 is a perspective view of a portion 420 of a piezoelectric transducer according to an embodiment described herein. Figure 4 shows a single cantilever that may be part of a larger acoustic transducer as described herein. The cantilever of portion 420 is mounted on a substrate 437. In various embodiments, the surface of the illustrated substrate 437 coupled to the cantilever may be the top, bottom, or any other surface that positions the cantilever over an acoustic port, allowing sound waves to contact the beam and displace it based on a pressure difference on either side of the cantilever. The cantilever of portion 420 has three electrode layers 436 that separate the portions of piezoelectric material constituting the first piezoelectric structure. The cantilever has a first end 434 and a second end 432. The top surface visible in the perspective view of Figure 4 has a portion of the top surface at a distance of two-thirds from the first end, covered by one of the electrode layers 436. The first side 435 faces the viewpoint in Figure 4, and the second side 430 (not visible) faces away from the viewpoint in Figure 4. The first end 434 is coupled to the substrate 437, and the second end 432 is cantilevered away from the substrate and the first end. The first and second sides 430 can be separated, respectively, from adjacent cantilevers having the same structure as the cantilever in Figure 4. Multiple such beams may be configured to enclose a symmetrical polygonal shape, and each corresponding piezoelectric structure has the same triangular shape in a shared piezoelectric layer (combined to form a piezoelectric acoustic transducer, e.g., the transducer in Figure 2). Each of the cantilevers of such transducers comprises a cantilever, such as the beam in Figure 4, and these cantilevers are separated from adjacent beams by a gap between their sides. In such a configuration, the first cantilever is positioned adjacent to a second cantilever separated by a gap between the first side of the first piezoelectric structure and the first side of the second piezoelectric structure, and the corner of the first end of the first cantilever coupled to the substrate is separated by an initial gap from the corner of the first end of the second cantilever coupled to the substrate. A sealed membrane configuration having a central portion of the membrane where the tip of the cantilever beam makes contact can be used, in particular, for acoustic transducers where pressure fluctuations across the membrane over time (e.g., each beam of the membrane) can affect the low-frequency performance for converting acoustic signals from the acoustic port.In the case of motion detection transducers, other beam shapes without a sealed membrane structure (such as the sealed structure shown in Figure 2) can be used with little to no impact on performance. Such shapes may include beams with a rectangular top profile (for example, instead of the triangular top profile in Figure 4).

[0055] Figure 5 shows an embodiment of a system including a piezoelectric MEMS converter according to an embodiment described herein. Figure 5 schematically shows a surface 501 having a plurality of sensors configured according to an exemplary embodiment. For example, the surface 501 may be a car panel such as a car door or fender. As shown, the surface 501 has sensors 510 and 520 mounted thereon. In some embodiments, the surface 501 having sensors 510, 520 mounted on it may be an inner surface protecting the sensors. In other embodiments, the sensors may be mounted on an outward-facing surface of an object to improve response time and the quality of vibration signals received by the sensors, and the outward-facing sensors may be configured to be replaced after a damaging contact, and the system may use a plurality of sensors to identify when a collision may damage an outward-facing sensor or when a non-damaging contact or collision occurs. In other embodiments, the sensors may be at any position on the surface of an object to which vibrations are transmitted to the sensors. In some embodiments, a single sensor may be used, or more than two sensors may be used. Sensors 510 and 520 may be any of the sensors shown in Figures 1A to 1E, Figure 2, Figure 3, etc., or any similar sensors.

[0056] Sensors 510, 520 may include an internal control unit or a closely connected control unit (e.g., controlled by a controller such as controller 58, which enables operation in a low-power mode until a vibration having a threshold energy value is detected). When the vibration energy detected by one or more of the sensors exceeds a threshold energy value, the controller may switch to an operating mode configured to detect contact with surface 501. The sensors can then generate output data for a classification circuit that can be used to determine whether the type of contact is associated with one or more actions performed by a control circuit (e.g., control circuit 8 or processor 2210). The classification circuit can, for example, distinguish between types of contact and / or make other determinations related to the contact. Such determinations may relate to the severity or magnitude of the contact with an object or surface 501 of the object (e.g., including non-contact events associated with acoustic vibrations) (e.g., hard or soft contact, or non-contact acoustic signals), and whether the contact damaged surface 501 of the object or another surface (e.g., a scratch or dent on a car panel).

[0057] As described above, each of the multiple sensors 510, 520, and any additional sensors may include multiple transducers that generate data used by the classification circuit to make such determinations. In some embodiments, each of the multiple sensors includes a first piezoelectric MEMS transducer and a second piezoelectric MEMS transducer (similar to any of the above-described transducers, such as the transducer in Figure 2, or a transducer having a piezoelectric beam as described in Figure 3 or 4, or any similar piezoelectric beam for electromechanical signal conversion). The first piezoelectric microelectromechanical system (MEMS) transducer has a first output and is mechanically coupled to the surface of an object having surface 501 and / or additional surfaces, and the first piezoelectric MEMS transducer is configured to generate a first analog signal at the first output when the first analog signal is converted by the first piezoelectric MEMS transducer from vibrations propagating through the object. Similarly, the second piezoelectric MEMS converter has a second output and is configured to generate a second analog signal at its second output when the second analog signal is converted by the second piezoelectric MEMS converter from acoustic vibrations incident on the surface of an object. A classification circuit coupled to the outputs of the first piezoelectric MEMS converter and the second piezoelectric MEMS converter operates to process data from the first analog signal and data from the second analog signal (modified from the analog signal by, for example, an ADC, DAC, controller, etc.). The classification circuit can operate using various thresholds or categorization mechanisms to generate an output that categorizes combinations of the first and second analog signals received in one or more time frames.

[0058] As shown in Figure 5, sensors 510 and 520 are located in different locations. Sensor 510 is located at a first position on the surface 501 of the object, and sensor 520 is located at a second position on the surface of the object at a predetermined distance from the first position. Sensors 510, 520, and any additional sensors present on the object may cooperate to determine the location and / or direction of contact. For example, if a key is dragged across a car door, the sensors may be configured to recognize the contact, duration, speed, location, size, and / or direction of the key / door contact. Thus, various embodiments can be considered to have formed a touch surface similar to a touchscreen having that function. However, as mentioned above, some embodiments may have only a single sensor on the surface. Therefore, such embodiments may not provide the functionality of embodiments having two or more sensors on a single surface. In some embodiments, the classification circuit is configured to detect the location of an impact on the surface of an object based on the known positions of sensors 510 and 520, and based on the time delay or magnitude difference between vibrations detected by different sensors such as sensor 510 and sensor 520.

[0059] Figure 6A shows an embodiment of System 600 including a piezoelectric MEMS transducer according to the embodiments described herein. System 600 is a vehicle comprising a plurality of objects that constitute the vehicle. Figure 6A shows a plurality of different objects 601, 602, 603, 604, and 605 that partially constitute the vehicle of System 600. Object 601 is the upper door panel of the vehicle, object 602 is the lower door panel of the vehicle, objects 603 and 604 are the upper bumper, and object 605 is the hood panel. Each of objects 601 to 605 may be a solid panel or object having consistent mechanical properties related to the transmission of mechanical or acoustic vibrations through the object. For any of objects 601 to 605, a collision or impact to an outward-facing surface generates both acoustic and mechanical vibrations on the internal surface of the object on which a sensor may be mounted, such as surface 501 in Figure 5.

[0060] Furthermore, while multiple sensors, such as sensors 510, 520 within a single panel on surface 501, may have similar signals due to the sensors being mounted on the same surface of the panel, sensors mounted within different objects, such as object 601 and object 602, can also provide data to a classification circuit that can be used to classify the data. For example, a sensor such as sensor 10D in Figure 1D may have high sensitivity and can detect vibrations from, for example, the closing of another car door near object 602 without any physical contact with object 602. Similarly, if sensors within object 602 and object 601 detect similar acoustic vibrations, but much stronger mechanical or kinetic vibrations are detected by the sensor within object 601, the data from the transducer signal can be used by the classification circuit to analyze possible impacts on the outer surface of object 601. If object 602 has multiple sensors, the sensors can help provide an estimated location of a collision on object 601 or on another object that does not have sensors, such as a glass window on the door panel of object 601. Similarly, sensors within bumper objects 604 and 603 can provide data to assist in classifying the impact on either bumper.

[0061] A test mode for the system, such as the system in Figure 1A, is used with system 600 to generate machine learning data that can be used by the classification system to identify patterns of data associated with control system alerts (e.g., automatically generated alerts managed by control circuit 8) and patterns of signals that distinguish them from important classifiers. For example, in training mode, sensors of system 600 (e.g., mounted on the surfaces of objects 601 to 606) can record data. The data can be matched with known events or past context types associated with training data that can be used to train a machine learning circuit (e.g., machine learning engine 7). Such data and matched known events may be, for example, bumper collisions, a car door hitting a panel or other object of system 600 and generating a paint scratch, rain entering system 600, or key scratches. After training the classification circuit (e.g., machine learning engine 7), an operating mode can be used to detect signals that match known events from the training. The control circuit can then be configured to take action when a known event associated with an action occurs. For example, if data matching the door of an adjacent car colliding with an object in system 600 is generated, the camera can be activated to capture an image of the object, confirm whether any damage or paint scratches occurred, or classify it as undamaged, and capture details of the adjacent car that caused the collision or the area surrounding the collision. Similarly, data matching a key scratch can be used to initiate a video capture and / or transmit a wireless signal to a mobile device associated with system 600. In contrast, data associated with a balloon bursting can be identified as a non-contact event, or an event without an associated action to be initiated by the system.

[0062] In a system such as system 600 in Figure 6A, each sensor can provide data to a central system or a computing device similar to the computing device in Figure 11. Such a central computing device can receive data generated from the converter analog signal (for example, using ADC, DAC, and controller circuits in the sensor package), process the aggregated sensor data to determine if contact has occurred, classify the contact, and perform any control system action instructed for a given contact type.

[0063] Training data generated using System 600 may be provided with a copy of System 600 so that similar systems can access memory storing similar known information. For example, an automotive manufacturer may have training data, provide access to that data, and include a system that generates and updates training data from users. The data can be generated using representative samples of a particular sensor itself (e.g., a sample of a sensor system including motion sensors and microphones). Other embodiments may use sensors trained to generate known contacts and record the system's responses. In any case, these responses are stored and used by a classification circuit and / or associated machine learning engine. As described below, these responses generate a number of motion data features (sometimes referred to herein as “motion data” or “representative data features”) that correlate to a particular contact or event type (e.g., as further detailed below in Figures 7, 8, and 9).

[0064] Furthermore, while automotive applications will be described, various embodiments can be applied to other applications. For example, surface 501 may be the surface of an object such as a robot element, a storage container, a building wall, a ship's hull, or an airplane panel. Any such system may include one or more sensors according to the embodiments described herein. A sensor in any such system may include a package comprising a motion sensor configured to detect motion, a microphone configured to detect sound, and a machine learning engine configured to determine information about contact using data from the microphone and the motion detector. The package has a base on which all three components are mounted. Thus, the motion sensor should detect the motion of the surface on which it is fixed. However, alternative embodiments may use two or more packages to form a single sensor (e.g., on a printed circuit board). For example, the motion sensor and microphone may be in a first package, and the machine learning engine may be in a second package. Other embodiments may divide all three elements into different packages.

[0065] In some embodiments, the sensor may be configured as a low-power wake-up, high-bandwidth, and / or low-noise-floor sensor. The low noise floor of the piezoelectric MEMS transducer allows for the collection of large amounts of data, but there is a risk of false alerts being automatically generated at an excessive rate without a contact signal threshold, and the classification circuit limits the excessive signaling that may occur if user alerts or notifications are generated for all sensor signals that exceed the noise floor. In some embodiments, the piezoelectric MEMS transducer of the sensor (e.g., sensors 510, 520) has a noise floor of about 100 micrograms (μg) per square root of the vibration frequency (sqrt(Hz)) (μg / sqrt(Hz)). Other sensors may have a noise floor of about 0.5 μg / sqrt(Hz) at 1 kHz, and 50 μg / sqrt(Hz) to 5 μg / sqrt(Hz) at 1 kHz. In some embodiments, different transducers sensing acoustic and mechanical vibrations may have different characteristics. (For example, a motion sensor may have a noise floor between 100 μg / sqrt(Hz) and 0.05 μg / sqrt(Hz) at device resonance, and / or a noise floor between 5 μg / sqrt(Hz) and 0.05 μg / sqrt(Hz) at resonance with an acoustic sensor having a different noise floor.) Furthermore, in some embodiments, the sensor may have a vibration detection bandwidth between 1 kHz and 8 kHz. In other examples, other frequency ranges may be used, with reduced bandwidth resulting in less data for the ML algorithm, and increased bandwidth requiring more ML processing resources for additional ML data. In some embodiments, the sensor may be configured to operate with a total sensor power consumption of 20 microwatts or less during a low-power pre-wake-up mode. Different implementation environments can use various sensor designs according to the embodiments described herein. Here, the noise floor has units of acceleration given in standard gravitational units (g), where 1g is 1x the gravitational acceleration of Earth (1g = approximately 9.8 m / s²).The difference from conventional transducers is that while conventional systems have a noise floor of approximately 300 / 1,000,000 (μg / sqrt(Hz)) per square root of cycles at 1 kHz, the example described herein can operate at approximately 13 μg / sqrt(Hz). Within a narrow bandwidth around the resonance, our noise floor is less than 1 μg / sqrt(Hz).

[0066] As illustrated, an object containing MEMS sensors may be a component of a car. In some embodiments, sensors may be positioned in a portion of the object such that a single panel may have MEMS sensors within components such as separate sensors in a portion of the panel defined by the relative positioning and orientation of the sensors contained within the panel, such as the upper right portion of the door, the lower right portion of the door, the upper left portion of the door, the lower left portion of the door, the center of the door, or a portion of the panel defined by the relative positioning and orientation of the sensors contained within the panel. In some embodiments, as described below with respect to Figure 6B, directional MEMS sensors may be used, and both relative positioning and orientation may be used to configure the sensor arrangement (e.g., to have 360-degree directional sensing), and thus the location of an area which is a section may represent sectors depending on the number of sensors, in addition to the arrangement of various MEMS sensors. Similarly, in some embodiments, a three-dimensional space within an object volume (e.g., the space between a car door and a second object or contact location) may be represented by zones, different fields of view, or sectors (e.g., for the positional location of a sound occurring away from the system, such as a car door slamming shut at a distance from the car in Figure 6A).

[0067] Furthermore, although different embodiments are described in the context of different packaging configurations, it will become clear that a wide variety of integrated packaging of multiple or single transducers and support circuits can be used in different applications. For example, some embodiments described above show motion detectors, acoustic detection microphones, and machine learning engines in a single die integrated package, while other embodiments can operate using separate dies and packages for each of these components.

[0068] Figure 6B shows an embodiment of a system comprising one or more arrays of piezoelectric MEMS transducers according to the embodiments described herein. Figure 6B shows the automotive system of Figure 6A having a sensor array of MEMS detectors (e.g., which may include a pair of motion sensors and acoustic sensors). As described above, the MEMS sensors can be configured to have directionality due to the orientation of the cantilever, the location of the acoustic port (e.g., acoustic port 24), or other configurations. Figure 6B shows an array of sensors 612 (e.g., shown in a set of up / down / left / right directions). The array 612 enables local detection of contact. Multiple such arrays 612 can enable directional identification of contact locations from directional data within the arrays 612. In some embodiments, such directionality can be a substitute for, or in addition to, a location determined from the delay of signals received by sensors having known relative positions (e.g., using a composite system that approximates the contact location from both a delay time circle associated with the time when signals from the same contact are received by different sensors and directional data). Something within the controller, DSP, or processor can combine two sensor (e.g., microphone) signals from two or more microphone sensors, which are configured to detect different directions of arrival from an audio source (e.g., through audio beamforming). Having two or more microphones located on different axes or different sections of a MEMS structure, as shown in Figure 2, enables the detection of different directions of arrival from one or more audio sources. Based on the coordination of the multiple microphone signals in a controller integrated into one or more processors, the direction detection can extend to different areas or sectors mapped to different parts of the surface of a first object (e.g., a door).

[0069] As described above, the machine learning engine determines the type of motion. Therefore, the exemplary embodiment trains the machine learning engine to provide that function. To this end, Figures 7, 8, and 9 show motions that may be used to generate either training data or motion data. Training data is matched with known events to generate machine learning system associations between data patterns and events, while motion data is provided to a classification circuit by sensors during motion, enabling the classification circuit to indicate whether the motion data is associated with a pattern identified by the machine learning engine during training. One process of training is shown. Those skilled in the art may use other techniques to train the machine learning engine. Therefore, this method should be considered a simplified example from a longer process that may be used to train the machine learning engine. Accordingly, the illustrated method in Figures 7, 8, and 9 can be carried out with additional steps, repeated steps, or intervening steps. Therefore, those skilled in the art can modify the process as appropriate.

[0070] Figure 7 illustrates a method associated with a piezoelectric MEMS contact detection system in a device, according to an embodiment described herein. Figure 7 shows an exemplary method 700 for the operation of a transducer system (e.g., a system according to any embodiment described above). In some embodiments, method 700 is implemented by a transducer system such as a system integrated with a device in a computing system or device (e.g., computing device 1100), as described below. In some embodiments, method 700 is implemented as a computer-readable instruction in a storage medium, which, when executed by the processing circuit of the device, causes the device to perform the operation of method 700, as described in the following blocks. Method 700 is shown in one exemplary embodiment according to the details provided herein. According to the embodiments described herein, it will be apparent that other methods are possible, including methods involving intervening operations or repeated operations.

[0071] Method 700 includes block 702 describing storing data from a first analog signal generated by a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, the first piezoelectric MEMS transducer being mechanically coupled to a first surface of an object, and the first piezoelectric MEMS transducer being configured to generate a first analog signal at a first output when the first analog signal is converted by the first piezoelectric MEMS transducer from vibrations propagating through the object.

[0072] Method 700 further includes block 704 describing storing data from a second piezoelectric MEMS converter having a second output in the memory of the device, wherein the second piezoelectric MEMS converter is configured to generate a second analog signal at the second output when the second analog signal is converted by the second piezoelectric MEMS converter from acoustic vibrations incident on a first surface of an object.

[0073] Method 700 further includes block 706, which describes processing data from a first analog signal and data from a second analog signal using a classification circuit coupled to the output of a first piezoelectric MEMS converter and the output of a second piezoelectric MEMS converter to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0074] In order to perform contact detection according to any details provided herein, additional actions, repeated actions, and intervening actions may be performed in addition to the actions of method 700.

[0075] Figure 8A shows another exemplary method 800 in several embodiments. Method 800 illustrates an alternative method for the operation of a contact system in embodiments described herein.

[0076] Method 800 includes block 802, which includes converting contact information into multiple frames for analysis. The size of the frames may be a function of the data and timing. In some embodiments, each frame is 20 milliseconds (ms). The contact information in each frame may be in the form of a waveform with or without one or more zero crossings. Each frame is then processed by the operations of blocks 804, 806, and 808 to generate motion data features (e.g., characteristic data for each frame).

[0077] Method 800 includes block 804, in which the processing circuit squares the amplitude(s) of the waveform of the frame to ensure that the data does not have negative values. These amplitudes may be in digital form, but in some embodiments, analog mixing can be used to square the analog signals from the piezoelectric MEMS transducers of the sensors. After squaring the amplitudes, block 804 further includes summing all the squared amplitudes to produce a single amplitude value. A corresponding analog step may be integrating the squared signals to produce a single analog value. Block 804 includes performing the same step for each signal from each piezoelectric MEMS transducer of the sensors. In some embodiments, such operation may be performed in series for data from a first piezoelectric MEMS sensor and a second piezoelectric MEMS sensor. In other embodiments, the operation of block 804 is performed in parallel for signals from different transducers. In an application with two transducers (e.g., a microphone and a motion detector), block 804 produces two data values: one single amplitude value is produced for each transducer (e.g., a microphone and a motion sensor).

[0078] Correspondingly, block 806 includes processing circuitry to calculate the number of zero crossings in the signals from each piezoelectric MEMS transducer. Similar to block 804, this step also generates two data values: one zero crossing for each piezoelectric MEMS transducer. The zero crossing value reflects the primary frequency component of the energy detected by a given transducer. If the frequency of the frame signal is higher, there will correspondingly be more zero crossings (e.g., within the detection bandwidth of a given transducer).

[0079] Block 808 then includes an operation to determine the ratio of the number of sums of the squares of the amplitudes (e.g., values ​​from Block 804) for different transducers. In implementations having a microphone (e.g., an acoustic transducer) and a motion sensor, Block 808 generates a ratio of signals associated with acoustic vibrations and mechanical vibrations. Such a ratio can enable a characterization system to distinguish between large noise or high-amplitude acoustic vibrations (e.g., which may not be within the audible frequency range) that are not associated with object contact, and high-amplitude vibrations (e.g., which may or may not be within the audible frequency range) that are associated with contact (e.g., a collision). In embodiments with more than two transducer signals, the system design can determine which ratio is most relevant to classifying incidents or contacts associated with matching (e.g., of the same period) data signals.

[0080] Method 800 yields five data points of data that characterize the transformer data for the nature and range of a particular contact associated with a frame. If a known trained classifier is associated with the data, additional operations train the machine learning engine. While the device is operating (e.g., not in training mode), such data can be processed by the machine learning engine to classify any matches with a trained incident or contact type. For example, the system may be trained to identify a specific number of contact types (e.g., 8, 16, 32, etc.). In some embodiments, contacts that do not match a trained data pattern can be processed by a control circuit according to additional rules such as amplitude or energy thresholds.

[0081] The trained sensor can match the trained contact type to data consisting of five illustrated data points per frame and perform additional actions according to rules within the control system.

[0082] Figure 8B shows a generalized classifier 810. In some embodiments, classifier 810 may be used to implement method 800 of Figure 8A. In other embodiments, other implementation forms or methods may be implemented using classifier 810. Classifier 810 has inputs 812 and 814. In some embodiments, input 812 may be a motion sensor input configured to accept analog motion sensor data from a MEMS sensor as described above, or digital data processed from a MEMS sensor (e.g., using a DSP and / or ADC as described above in Figure 1A). In such embodiments, input 814 may be an audio sensor input configured to accept analog signals or digital data generated from a MEMS acoustic sensor (e.g., the MEMS sensor in Figure 2). The similarity input 816 may be used to provide settings, operating modes, or other control inputs for classifier 810. For example, the similarity input 816 may be used in some embodiments to manage training data inputs or to provide classification of training data provided in inputs 812, 814 during training mode. During operation (as opposed to, for example, training mode or low-power sleep mode), the classifier 810 can receive data via inputs 812 and 814, and any additional data via similarity 816, and provide contact classification values ​​(e.g., noise contact, key scraping contact, minor car collision contact, major car collision contact, etc.) at output 818. Output 818 can then be coupled to an additional control circuit, as described herein, to process the contact classification values ​​and automatically initiate further actions based on the output values ​​(e.g., an automatic emergency service call in response to a major car collision contact, activation of a camera, and activation of an alarm in response to a key scraping contact, etc.). The classifier 810 is shown with two data inputs 812 and 814. In other embodiments, any number of data inputs may be implemented in the classifier.

[0083] Such inputs can be associated with multiple sensors, such as a car or other device having any number of sensors or combinations of sensors (e.g., a pair of motion sensors and audio sensors). For example, there may be a linear array of microphones, such as the one in Figure 2 that performs audio beamforming, and in combination with the outputs from the motion sensors and classifiers, the contact type can help detect whether the location of the sound source is outside the car (e.g., audible) or whether it has also made contact with the door (e.g., scratch, bump, soft contact, hard contact, etc.). That is, the location may be on or off the door, and further distinguished by the volume and / or level of contact on the door. Furthermore, one classification type may arise for each microphone and motion sensor pair. There may be multiple classifiers, and there may be one classifier for each motion sensor and microphone pair. The outputs of multiple classifiers may be combined, for example, if both classifiers indicate a contact type that provides higher confidence in the contact type result. If both classifiers do not indicate the same contact type, separate classifications must be repeated until the contact types match. This can happen, for example, because there isn't enough memory or buffer to store the history of past frames or contact types.

[0084] In addition, or alternatively, several frames of motion sensor data and microphone data may be used, and the entirety of multiple frames of a combined contact type, for example, 5 to 10 frames of data (e.g., 50 to 100 ms), may be used to determine the contact type. For example, in frame 1, classifier 1 determines contact type 1, and classifier 2 determines contact type 2, but in the next four frames, frames 2 to 5, both classifier outputs from classifier 1 and classifier 2 are the same, and the confidence of the contact type result is higher. This can occur if there is an additional buffer memory from the previous generation which was only intended to take the classification output of one frame. That is, there may be enough memory or buffer to store the history of past frames or contact types. For example, if there is only one classifier with a microphone / motion sensor pair, there is one contact type. Or, if there are two classifiers, both outputs must be the same in order to determine the final contact type.

[0085] Similarly, a classifier may include multiple outputs, such as a classification output, a directional output (e.g., having a directionality identified from the delay between similar signals received by different sensors having known locations, and a velocity of vibration used to determine the location or direction of contact), or other such outputs (e.g., different alert or contact outputs for different locations on a device associated with different inputs). In other embodiments, a combination of classifiers 810 can be combined into a single system within a single device, rather than a single classifier used for any number of inputs and outputs, and the multiple classifiers 810 provide inputs to a control circuit that can be used to generate data and / or automatic alerts or actions in response to a combination of classifier outputs 818.

[0086] Figure 8C illustrates the operation of the classifier 810 in several implementation configurations. In Figure 8C, the x-axis X1 can represent data from input 812, and the y-axis can represent data from input 814. Lines H1, H2, and H3 can represent inputs of classification thresholds or similarity 816 used to identify different classification groups. The various data points can be combinations of data from sensors over a given period, and the output 818 can provide data about where a group of data falls within a classification threshold, or a classification output associated with a group of data within a classification threshold, as shown. In some embodiments, the classifier 810 can take in motion signals at input 812 and microphone signals at input 814, and generate classification signals at output 818 based on classification groupings generated by analyzing the data compared to classification thresholds H1, H2, and H3.

[0087] The illustrated H1, H2, and H3 lines shown in 2D are merely lines, but if the signals in inputs 812, 814 are multidimensional, or if additional inputs are added, the classification threshold may be a hyperplane in a higher-dimensional space (for example, if X1 in an alternative embodiment is some representation of a combination of microphones and motion sensors, and in some embodiments this may be indicated by a similarity value provided in the input similarity 816, then the number of microphones may be dimensional, or the number of microphone / motion sensor pairs may be dimensional).

[0088] In some embodiments, the higher-dimensional space may be 8-dimensional, or 16-dimensional if only microphone and motion sensor signals are input to classifier 810, or 24-dimensional if eight additional similarity values ​​are input to classifier 816 in the similarity input. In some embodiments, for example, the cantilever beam of the MEMS converter shown in Figure 2 can provide multiple signals, with additional separate signals for the motion sensor and acoustic sensor provided in parallel to classifier 810. The coupled crossover of the outputs shown in output 818, or additional outputs in implementations with multiple outputs, may be used by logic circuits to indicate contact type.

[0089] As described herein, in some embodiments, the machine learning engine (e.g., machine learning engine 7 in Figure 1A) may be a support vector machine or a support vector network. In machine learning, support vector machines (SVMs) are supervised learning models that have associated learning algorithms for analyzing data for classification and regression analysis. SVMs are robust prediction methods based on statistical learning frameworks or VC theory. Given a set of training examples, each marked to belong to one of two categories, the SVM training algorithm constructs a model that assigns new examples to one of the two categories, making it a non-stochastic binary linear classifier (for example, there are methods such as Platt scaling that use SVMs in a stochastic classification setting). The SVM maps the training examples to points in space so as to maximize the width of the gap between the two categories. New examples are then mapped to that same space and predicted to belong to a category based on which side of the gap they fall on.

[0090] In addition to performing linear classification, SVMs can efficiently perform nonlinear classification using what is called the kernel trick, implicitly mapping their inputs to a high-dimensional feature space. When data is unlabeled, supervised learning is impossible, and an unsupervised learning approach is required, which attempts to discover natural clustering of data into groups and then map new data to these formed groups. Support vector clustering algorithms categorize unlabeled data by applying the statistics of support vectors developed in support vector machine algorithms.

[0091] In some embodiments of SVM, data points are treated as p-dimensional vectors (lists of p numbers), and a (p-1)-dimensional hyperplane (e.g., a linear classifier) ​​is used to identify distinct point clusters.

[0092] There are many hyperplanes that can classify data. One reasonable choice as a selected hyperplane (e.g., lines or planes H1, H2, H3, etc.) is the hyperplane that represents the greatest separation or margin between two classes.

[0093] More formally, a support vector machine constructs a hyperplane or set of hyperplanes in a high-dimensional or infinite-dimensional space, which can be used for other tasks such as classification, regression, or outlier detection. In some embodiments, the separation of groups or classifications is achieved by a hyperplane having the maximum distance (e.g., a functional margin) to the nearest data point of any class, because generally, a larger margin results in a lower generalization error for the classifier.

[0094] In some embodiments, a classification group with errors exceeding a certain threshold may be assigned a specific automated response (e.g., a call or communication with an emergency service), while a group with errors or classification separations below a certain threshold may generate a user message or trigger an additional sensor (e.g., a video camera), which requests further analysis from the user to determine whether a specific contact type has occurred based on the sensor data and additional video or alert data.

[0095] Figure 9 shows another exemplary method 900 in several embodiments. The method 900 in Figure 9 exhibits operation similar to that of Figure 8A, with the operation of generating data from the analog piezoelectric MEMS converter signal being performed in parallel.

[0096] Method 900 includes blocks 902 and 904 that receive parallel streams of input data from different piezoelectric MEMS transducers. In the example of block 902, the data is from a motion detector transducer, and in the example of block 904, the data is from a microphone. As described above, in some embodiments, the data streams described in blocks 902 and 904 are generated only when a threshold detection occurs, waking up the system's sensors from low-power mode. In other embodiments, “always-on” operation may be used to collect transducer input data when the power consumed is low compared to the available power or the value when detecting initial vibration data.

[0097] Furthermore, Method 900 demonstrates the collection of two parallel data streams from two transducers in blocks 902 and 904. In other embodiments, any number of data streams may be used. For example, in some embodiments, such as System 600 in Figure 6A, each object may have two transducers whose data is processed independently and then further analyzed after the two signals from each object are characterized (for example, Method 900 is repeated for the transducer data of each object in System 600). In other embodiments, the sensors from each object may be characterized jointly (for example, with input data similar to that of blocks 902 and 904, which are jointly characterized in a similar manner to Method 900).

[0098] Method 900 includes block 906, in which data from blocks 902 and 904 are converted into frame data. Such conversion may include clock timing with identified start and end periods for a time frame, and each data stream from blocks 902 and 904 is segmented into data frames that match the data collected for each corresponding period of the time frame. In various embodiments, the periods used for the data frames can match the expected vibration frequencies and periods that will generate accurate characterization data of the events to be characterized by the classification circuit. In embodiments involving car panels, 20ms may be used. In other embodiments with larger panel objects, such as ship hulls or airplane panels, or where different vibration frequencies exist and may be key to contact characterization, different time frames may be used.

[0099] Blocks 908, 910, and 912 include parallel processing of data in the same manner as described above in the corresponding blocks 804, 806, and 808 (for example, block 804 corresponds to 908, 806 to 910, and 808 to 912). The characterized data is then stored and associated with a set of actions and contact types (e.g., collision, key scratch, etc.) under training, or matched with training data in operating modes so that contacts can be identified by matching the operating data with previously stored training data.

[0100] Block 914 then processes the data from the previous block in order to format the block for a classifier or other machine learning engine, or to process it by a machine learning engine.

[0101] Figures 10A to 10D illustrate embodiments of contact detection and classification using MEMS transducers according to embodiments described herein. The illustrated systems in Figures 10A to 10D may be used, for example, to carry out Method 900 or any similar method described herein. The system in Figure 10A includes, for example, two sensors, indicated as motion sensors 1002 and acoustic sensor 1004. A similarity meter 1006 can process data signals and generate standardized data that, based on the data from sensors 1002 and 1004, can be used by a contact type classifier 1008 to generate a contact type output indicating information about the expected contact (e.g., location, type, severity, etc.). The combination of steps of squaring the amplitude and sum of block 804 in Figure 8A, counting the zero-crossing operations of block 806, and determining the ratio of sensor sum operations of block 808 may also be generally represented as a similarity measurement or incorporated as an alternative to other embodiments described herein. The combination of blocks 802, 804, and 806 in Figure 8A may be incorporated as part of the similarity meter 1006 shown in Figure 10A. The combination of the steps of squaring the amplitude of block 908 in Figure 9, counting the zero crossings of block 910, and determining the ratio of the transducer totals of block 912 may also be generally represented as a similarity measure, or may be incorporated as part of an alternative to other embodiments described herein. The combination of blocks 908, 910, and 912 in Figure 9 may be incorporated as part of the similarity meter 1006 shown in Figure 10A. In some embodiments described herein, similar operations of normalizing or standardizing signals from MEMS sensors may be handled using circuits that perform similar operations that are not the exact operations described in Figures 8 and 9. Such operations may improve the performance, accuracy, and / or standard operation of a classification or machine learning engine circuit.

[0102] Figure 10B shows an example of the system of Figure 10A that can be adapted to Method 900. The similarity meter 1006 in Figure 10B includes a frequency detection 1012 that can perform a frequency detection operation to quantify the frequency characteristics of a signal, such as the operation of block 806 or block 910. A magnitude detection can perform an operation to quantify the amplitude characteristics of a signal, such as the operation of block 804 or block 908. A comparison 1014 can perform an operation such as the operation of block 808 or block 912.

[0103] Figures 10C and 10D show additional examples of systems for processing sensor data. As illustrated, Figure 10C includes sensors 1002 and 1004, with a correlation block 1022 for motion sensor 1002, a correlation block 1024 for acoustic sensor 1004, and a joint correlator 1026 for the combination of sensors. In such embodiments, independent similarity calculations may be used in addition to joint similarity calculations to process the sensor data and provide input to a classifier. Figure 10D shows a normalizer 1032 for both sensors 1002 and 1004, and a correlator 1034 that uses the output of the normalizer 1032. In other embodiments, any number of data operations can be used. Such systems can support correlation calculations between sensor data, autocorrelation calculations of data streams, absolute value calculations, rectification calculations, and other such operations.

[0104] Figures 11 and 12 illustrate embodiments of a system supporting contact detection and classification using a MEMS converter, according to embodiments described herein. Figure 11 shows an audio analyzer 1140 including a wake detector 1126 in addition to a classification application (one or more) 1156. As described herein, in some embodiments, the sensor can operate in a low-power mode of about 20 milliwatts (mW), and preprocessing is used to wake up the sensor to perform higher-power operation. Sensors such as audio sensors can have sufficient sensitivity to detect both minor collision signals and audio inputs such as “wake words” or wake signals used to activate a voice command system. In some embodiments, an audio sensor used in combination with a motion sensor can be used for both collision detection and voice input from a user. In other embodiments, the voice input for collision detection and the audio sensor are implemented as separate systems with separate sensors.

[0105] The device 1100 in Figure 11 may comprise any number of different devices having any number of sensor sources 1202 (e.g., motion and audio MEMS sensors as described herein). Such sensor sources 1202 and microphones 1104 (e.g., microphones 1104A-F) may be integrated with a system having audio transducers 1162A-D (e.g., speakers) that can provide audio output information for system users to identify collision or contact type (e.g., using an output speaker system to inform the driver of collision type, location, severity, etc.).

[0106] Figure 12 illustrates a first example of a vehicle 1250 capable of operating to selectively filter audio data for speech processing, according to several examples of the present disclosure. In Figure 12, the device 1100 or a part thereof is integrated into a vehicle 1250, which is shown as an automobile including several seats 1252A–1252E in the example of Figure 12. Although the vehicle 1250 is shown as an automobile in Figure 12, in other embodiments the vehicle 1250 is a bus, train, aircraft, ship, or another type of vehicle configured to transport one or more passengers (which may optionally include a vehicle operator).

[0107] The vehicle 1250 includes an audio analyzer 1140 and one or more sensor sources 1202. The audio analyzer 1140 and sensor sources (one or more) 1202 (e.g., motion and / or audio MEMS sensors) are connected via analog signal processing 1204. The vehicle 1250 in Figure 12 also includes one or more vehicle systems 1260, some or all of which may be coupled to the audio analyzer 1140 to enable a voice assistant classification application (one or more) 1156 to control various operations of the vehicle systems (one or more) 1260.

[0108] In Figure 12, the vehicle 1250 includes multiple microphones 1104A-104F. For example, in Figure 12, each microphone 1104 is positioned near one of each of the seats 1252A-1252E. In the example in Figure 12, the positioning of the microphones 1104 relative to the seats 1252 allows the audio analyzer 1140 to distinguish the audio zones 1254 of the vehicle 1250. In Figure 12, there is a one-to-one relationship between the audio zones 1254 and the seats 1252. In some other implementations, one or more of the audio zones 1254 include two or more seats 1252. For example, seats 1252C-1252E may be associated with a single "rear seat" audio zone.

[0109] In Figure 12, the vehicle 1250 is shown to include multiple microphones 1104A-104F arranged to detect sound within the vehicle 1250 and allow the audio analyzer 1140 to selectively distinguish which audio zone 1254 contains the sound source. However, in other implementations, the vehicle 1250 includes only a single microphone 1104. In yet another implementation, the vehicle 1250 includes multiple microphones 1104, and the audio analyzer 1140 does not distinguish between audio zones 1254.

[0110] In Figure 12, the audio analyzer 1140 includes an audio preprocessor 1118, a first-stage speech processor 1124, and a second-stage speech processor 1154, each of which operates as described in any of Figures 11 to 5. In the specific example shown in Figure 12, the audio preprocessor 1118 includes one or more speech input filters 1120 that can be configured to act as speaker-specific speech input filters that selectively filter audio data for speech processing.

[0111] The audio preprocessor 1118 in Figure 12 also includes an echo cancellation and noise suppression (ECNS) unit 1206 and an adaptive interference canceller (AIC) 1208. The ECNS unit 1206 and the AIC 1208 can operate independently of the speech input filter(s) 1120 to filter audio data from the microphone(s) 1104. For example, the ECNS unit 1206, the AIC 1208, or both may perform non-speaker specific audio filtering operations. For example, the ECNS unit 1206 can operate to perform echo cancellation operations, noise suppression operations (e.g., adaptive noise filtering), or both. The AIC 1208 is optionally configured to restrict the audio data provided to the first-stage speech processor 1124, the second-stage speech processor 1154, or both, to audio from one or more specific audio zones 1254, in order to distinguish between audio zones 1254. For example, based on the configuration of the audio analyzer 1140, the AIC 1208 may only allow audio from a person in one of the front seats 1252A, 1252B to be provided to the wake detector 1126, to a voice assistant classification application (one or more) 1156, or to both.

[0112] During operation, one or more of the microphones 1104 may detect sounds within the vehicle 1250 and provide audio data representing the sounds to the audio analyzer 1140. When a voice assistant session is not in progress, the ECNS unit 1206, the AIC 1208, or both, processes the audio data to generate filtered audio data and provides the filtered audio data to the wake detector 1126. If the wake detector 1126 detects a wake signal (e.g., motion and / or audio signals from a collision sensor exceeding a threshold magnitude) within the filtered audio data, the wake detector 1126 is configured to identify details associated with the wake signal (e.g., collision identification or identification of the person speaking). Furthermore, the wake detector 1126 activates the second-stage voice processor 1154 to initiate a voice assistant session. In some implementations, the wake detector 1126 may also provide information to the AIC 1208 to indicate in which audio zone 1254 a threshold wake signal occurred, and the AIC 1208 may filter the audio data provided to the audio input filter(s) 1120 based on the audio zone 1254 in which the threshold wake signal occurred.

[0113] Speaker-specific speech input filters are used to filter audio data and provide the filtered audio data to a voice assistant classification application(s) 1156, as described herein. Based on the content of the speech represented in the filtered audio data, the voice assistant classification application(s) 1156 may control the operation of an audio source(s) 1202, control the operation of a vehicle system(s) 1260, or perform other operations such as retrieving information from a remote data source.

[0114] Responses from the voice assistant classification application(s) 1156 (e.g., voice assistant responses) can be reproduced to the occupants of the vehicle 1250 via an audio transducer(s) 1162 that indicates information associated with the output generated in response to the classification of MEMS sensor data associated with the user's voice input or processed according to the embodiments described herein. In the example shown in Figure 12, the audio transducer(s) 1162 is located near or within a specific zone of the audio zone 1254, thereby enabling the voice assistant classification application(s) 1156 to provide responses to a specific occupant(e.g., the occupant who initiated the voice assistant session) or a group of occupants of the vehicle 1250.

[0115] The selective operation of the voice input filter(single or multiple) 1120 as a speaker-specific voice input filter removes noise and irrelevant speech from the audio data provided to the voice assistant classification application(single or multiple) 1156, thereby enabling more accurate speech recognition by the voice assistant classification application(single or multiple) 1156. Furthermore, the selective operation of the voice input filter(single or multiple) 1120 as a speaker-specific voice input filter restricts the ability of other occupants in the vehicle 1250 to enter the voice assistant session. For example, if the driver of the vehicle 1250 initiates a voice assistant session to request driving instructions, the voice assistant session can be associated only with the driver (or, as described above, with one or more other people) so that other occupants of the vehicle 1250 cannot interrupt the voice assistant session.

[0116] Figure 13 shows an implementation form 1300 of device 1100 as an integrated circuit 1302, which includes one or more processors 1190, including one or more components of the audio analyzer 1140. The integrated circuit 1302 also includes input circuits 1304, such as one or more bus interfaces, which enable audio data to be received for processing. The integrated circuit 1302 also includes output circuits 1306, such as bus interfaces, which enable the transmission of output data 1308 from the integrated circuit 1302. For example, the output data 1308 may include voice assistant responses. As another example, the output data 1308 may include commands to other devices (such as media players, vehicle systems, or smart home devices) or queries (such as information retrieval queries sent to remote devices).

[0117] The integrated circuit 1302 enables an implementation that selectively filters audio data for voice processing as a component in a system including a microphone, such as a mobile phone or tablet shown in Figure 14, a wearable electronic device shown in Figure 15, a camera shown in Figure 16, an extended reality (e.g., virtual reality, mixed reality, or augmented reality) headset shown in Figure 17, or a vehicle shown in Figure 12 or Figure 18.

[0118] Figure 14 shows an exemplary, non-limiting implementation of device 1100, which includes a mobile device 1402 such as a telephone or tablet. In a particular implementation, the integrated circuit 1302 is integrated into the mobile device 1402. In Figure 14, the mobile device 1402 includes a microphone(s) 1104, an audio transducer(s) 1162, and a display screen 1404. Components of the processor(s) 1190, including an audio analyzer 1140, are integrated into the mobile device 1402 and are shown using dashed lines to indicate internal components that are generally not visible to the user of the mobile device 1402.

[0119] In certain cases, the audio analyzer 1140 may selectively enable speaker-specific speech input filtering to improve the accuracy of speech recognition by the voice assistant classification application(s) 1156 and to limit the ability of others to interrupt the voice assistant session. During a voice assistant session, responses from the voice assistant application may be provided to the user as output via the audio converter(s) 1162, via the display screen 1404, or both.

[0120] Figure 15 shows an implementation configuration 1500 in which device 1100 includes a wearable electronic device 1502, referred to as a “smartwatch.” In a particular implementation configuration, the integrated circuit 1302 is integrated into the wearable electronic device 1502. In Figure 15, the wearable electronic device 1502 includes one or more microphones 1104, one or more audio transducers 1162, and a display screen 1504.

[0121] Components of the processor(s) 1190, including the audio analyzer 1140, are integrated into the wearable electronic device 1502. In a particular example, the audio analyzer 1140 in Figure 15 operates to selectively enable speaker-specific speech input filtering to improve the accuracy of speech recognition by the voice assistant classification application(s) 1156 and to limit the ability of others to interrupt the voice assistant session. During a voice assistant session, responses from the voice assistant application may be provided as output to the user via the audio transducer(s) 1162, via haptic feedback to the user, via the display screen 1504, or any combination thereof.

[0122] As an example of the operation of the wearable electronic device 1502, during a voice assistant session, the person initiating the voice assistant session may provide voice prompts requesting that messages sent to them (e.g., text messages, emails, etc.) be displayed on the display screen 1504 of the wearable electronic device 1502. In this example, another person near the wearable electronic device 1502 may speak a threshold wake signal associated with the audio analyzer 1140 without interrupting the voice assistant session, because the audio data is filtered during the voice assistant session so as not to emphasize portions of the audio data that do not correspond to the voice of the person who initiated the voice assistant session.

[0123] Figure 16 shows an implementation configuration 1600 in which device 1100 includes a portable electronic device corresponding to camera device 1602. In a particular implementation configuration, integrated circuit 1302 is integrated into camera device 1602. In Figure 16, camera device 1602 includes one or more microphones 1104 and one or more audio converters 1162. Camera device 1602 may have a display screen on a side not shown in Figure 16.

[0124] Components of the processor(s) 1190, including the audio analyzer 1140, are integrated into the camera device 1602. In a particular example, the audio analyzer 1140 in Figure 16 operates to selectively enable speaker-specific speech input filtering to improve the accuracy of speech recognition by the voice assistant classification application(s) 1156 and to limit the ability of others to interrupt the voice assistant session. During a voice assistant session, responses from the voice assistant application may be provided to the user as output via the audio transducer(s) 1162, via the display screen, or both.

[0125] As an example of the operation of camera device 1602, during a voice assistant session, the person initiating the voice assistant session may provide audio requesting camera device 1602 to capture an image. In this example, another person in the vicinity of camera device 1602 may speak a threshold wake signal associated with audio analyzer 1140 without interrupting the voice assistant session, because the audio data is filtered during the voice assistant session so as not to emphasize portions of the audio data that do not correspond to the voice of the person initiating the voice assistant session.

[0126] Figure 17 shows one implementation configuration 1700 in which device 1100 includes a portable electronic device corresponding to an extended reality (e.g., virtual reality, mixed reality, or augmented reality) headset 1702. In a particular implementation configuration, the integrated circuit 1302 is incorporated into the headset 1702. In Figure 17, the headset 1702 includes one or more microphones 1104 and one or more audio transducers 1162. Furthermore, a visual interface device is positioned in front of the user's eyes to enable the display of augmented reality, mixed reality, or virtual reality images or scenes to the user while the headset 1702 is being worn.

[0127] Components of the processor(s) 1190, including the audio analyzer 1140, are integrated into the headset 1702. In a particular example, the audio analyzer 1140 in Figure 17 operates to selectively enable speaker-specific speech input filtering to improve the accuracy of speech recognition by the voice assistant classification application(s) 1156 and to limit the ability of others to interrupt the voice assistant session. During a voice assistant session, responses from the voice assistant application may be provided as output to the user via the audio transducer(s) 1162, via a visual interface device, or both.

[0128] As an example of the operation of headset 1702, during a voice assistant session, the person initiating the session may provide voice prompts requesting that specific media be displayed on the headset 1702's visual interface device. In this example, another person nearby the headset 1702 may speak a threshold wake signal associated with the audio analyzer 1140 without interrupting the voice assistant session, because the audio data is filtered during the voice assistant session to avoid emphasizing portions of the audio data that do not correspond to the voice of the person initiating the session.

[0129] Figure 18 shows an implementation form 1800 in which device 1100 corresponds to or is integrated within a vehicle 1802, which is shown as a manned or unmanned aerial device (e.g., a package delivery drone). In a particular implementation form, the integrated circuit 1302 is integrated within the vehicle 1802. In Figure 18, the vehicle 1802 also includes one or more microphones 1104 and one or more audio transducers 1162.

[0130] Components of the processor(s) 1190, including the audio analyzer 1140, are integrated into the vehicle 1802. In a particular example, the audio analyzer 1140 in Figure 18 operates to selectively enable speaker-specific speech input filtering to improve the accuracy of speech recognition by the voice assistant classification application(s) 1156 and to limit the ability of others to interrupt the voice assistant session. During a voice assistant session, responses from the voice assistant application may be provided as output to the user via the audio transducer(s) 1162.

[0131] As an example of vehicle 1802 operation, during a voice assistant session, the person initiating the voice assistant session may provide voice instructions requesting vehicle 1802 to deliver a package to a specified location. In this example, another person in the vicinity of vehicle 1802 may speak a threshold wake signal associated with the audio analyzer 1140 without interrupting the voice assistant session, because the audio data is filtered during the voice assistant session to avoid highlighting portions of the audio data that do not correspond to the voice of the person who initiated the voice assistant session. As a result, the other person cannot redirect vehicle 1802 to a different delivery location.

[0132] Figure 19 shows an embodiment of a system supporting contact detection and classification using a MEMS transducer, according to the embodiments described herein. Referring to Figure 19, a block diagram of a particular exemplary implementation of the device is shown, designated collectively as 1900. In various implementations, device 1900 may have more or fewer components than those shown in Figure 19. In the exemplary implementation, device 1900 may correspond to device 1100. In the exemplary implementation, device 1900 may perform collision detection operations according to any embodiment described herein.

[0133] In certain implementations, device 1900 includes a processor 1906 (e.g., a central processing unit (CPU)). Device 1900 may include one or more additional processors 1910 (e.g., one or more DSPs). In certain embodiments, the processor(s) 1190 in Figure 11 corresponds to processor 1906, processor 1910, or a combination thereof. The processor(s) 1910 may include a voice and music coder decoder (codec) 1908, including a voice coder ("vocoder") encoder 1936 and a vocoder decoder 1938. In the example shown in Figure 19, the processor(s) 1910 also includes an audio preprocessor 1118, a first-stage voice processor 1124, and optionally, a second-stage voice processor 1154.

[0134] Device 1900 may include memory 1142 and codec 1934. In certain implementations, the analog signal processing 1204 in Figure 11 may be implemented in some embodiments with respect to the codec 1934 in Figure 19. Memory 1142 may include instructions 1956 that can be executed by one or more additional processors 1910 (or processor 1906) to implement the functions described with reference to the audio preprocessor 1118, the first-stage audio processor 1124, the second-stage audio processor 1154, or a combination thereof. In the example shown in Figure 19, memory 1142 also includes registration data 1136.

[0135] Device 1900 may include a display 1928 coupled to a display controller 1926. An audio converter(s) 1162, a microphone(s) 1104, or both may be coupled to a codec 1934. The codec 1934 may include a digital-to-analog converter (DAC) 1902, an analog-to-digital converter (ADC) 1904, or both. In certain implementations, the codec 1934 may receive an analog signal from the microphone(s) 1104, use the analog-to-digital converter 1904 to convert the analog signal to a digital signal (e.g., audio data 1116 in Figure 11), and provide the digital signal to an audio and music codec 1908. The audio and music codec 1908 may process the digital signal, which may be further processed by an audio preprocessor 1118, a first-stage audio processor 1124, a second-stage audio processor 1154, or a combination thereof. In certain implementations, the audio and music codec 1908 can provide a digital signal to the codec 1934. The codec 1934 may convert the digital signal to an analog signal using the digital-to-analog converter 1902, or it may provide the analog signal to one or more audio converters 1162.

[0136] In certain implementations, device 1900 may be contained within a system-in-package or system-on-chip device 1922. In certain implementations, memory 1142, processor 1906, processor 1910, display controller 1926, codec 1934, and modem 1954 are contained within a system-in-package or system-on-chip device 1922. In certain implementations, input device 1930 and power supply 1944 are coupled to the system-in-package or system-on-chip device 1922. Furthermore, in certain implementations, as shown in Figure 19, the display 1928, input device 1930, audio converter(s) 1162, microphone(s) 1104, antenna 1952, and power supply 1944 are located outside the system-in-package or system-on-chip device 1922. In certain implementations, the display 1928, input device 1930, audio converter(s) 1162, microphone(s) 1104, antenna 1952, and power supply 1944 may each be coupled to a component of the system-in-package or system-on-chip device 1922, such as an interface or controller.

[0137] In some implementations, device 1900 includes a modem 1954 coupled to antenna 1952 via a converter 1950. In some such implementations, modem 1954 may be configured to send data associated with the voice from a first person (e.g., at least a portion of the audio data 1116 in Figure 11) to a remote classification assistance server which may be used in a cloud or server-based implementation of a classification application(s) 1156. In such implementations, the voice assistant classification application(s) 1156 runs on the classification assistance server. In such implementations, the second-stage voice processor 1154 may be omitted from device 1900. However, speaker-specific voice input filtering may be performed in device 1900 based on threshold wake signal detection in device 1900.

[0138] Device 1900 may include smart speakers, speaker bars, mobile communication devices, smartphones, cellular phones, laptop computers, computers, tablets, personal digital assistants, display devices, televisions, game consoles, music players, radios, digital video players, digital video disc (DVD) players, tuners, cameras, navigation devices, vehicles, headsets, augmented reality headsets, mixed reality headsets, virtual reality headsets, aerospace vehicles, home automation systems, voice-activated devices, wireless speakers and voice-activated devices, portable electronic devices, cars, computing devices, communication devices, Internet of Things (IoT) devices, virtual reality (VR) devices, base stations, mobile devices, or any combination thereof.

[0139] In relation to the described implementation, the apparatus includes means for acquiring first voice signature data associated with a first person based on the detection of a threshold wake signal in the voice of the first person. For example, the means for acquiring first voice signature data may correspond to device 1100, one or more processors 1190, an audio analyzer 1140, an audio preprocessor 1118, one or more voice input filters 1120, a first-stage voice processor 1124, a processor 1906, one or more processors 1910, one or more other circuits or components configured to acquire voice signature data, sensor threshold data, or any combination thereof.

[0140] The apparatus also includes means for selectively enabling speaker-specific speech input filters based on first speech signature data. For example, the means for selectively enabling speaker-specific speech input filters may correspond to device 1100, one or more processors 1190, an audio analyzer 1140, an audio preprocessor 1118, one or more speech input filters 1120, a first-stage speech processor 1124, a processor 1906, one or more processors 1910, one or more other circuits or components configured to selectively enable speaker-specific speech input filters, or any combination thereof.

[0141] In some implementations, a non-temporary computer-readable medium (e.g., a computer-readable storage device such as memory 1142) is executed by one or more processors (e.g., one or more processors 1190, one or more processors 1910, or processor 1906) and includes instructions (e.g., instruction 1956) that cause one or more processors to retrieve first speech signature data associated with a first person based on the detection of a threshold wake signal in the speech from the first person, and to selectively enable a speaker-specific speech input filter based on the first speech signature data.

[0142] Figure 20 is a functional block diagram of a wireless communication device configured for contact detection according to an embodiment described herein. The device 2000 comprises means 2002 for generating a first analog signal converted from vibrations propagating through an object having a first surface. Means 2002 may be, for example, a transducer 6, or a MEMS motion detector formed from a cantilever beam in Figure 3, or any other such MEMS sensor described herein.

[0143] The apparatus 2000 comprises means 2004 for generating a second analog signal converted from an acoustic signal incident on a first surface of an object. Means 2004 may be, for example, a transducer 5, or a MEMS motion detector formed from the cantilever beam of Figure 3, or any other such MEMS sensor described herein.

[0144] The device 2000 includes means 2006 for processing data from a first analog signal and data from a second analog signal in order to classify combinations of a first analog signal and a second analog signal received in one or more time frames. Means 2006 may include an ML engine 7, or any other ML engine circuit, such as a neural network, decision tree, and / or support vector machine circuit. In some embodiments, means 2006 may further include processing circuits, such as an ASIC chip 16, a control circuit 8, an ADC 54, a DSP 56, a controller 58, or any other such circuit used to generate and process data from the first and second analog signals generated by means 2002 and means 2004.

[0145] As described herein, various embodiments may be implemented using deep networks such as neural networks or multiple neural networks. Figure 21 is an example for illustrating a deep learning neural network 2100 that may be used by the contact classifier described. The input layer 2120 contains input data which may be data generated from the aforementioned speech and motion sensors. As detailed in Figures 10A to 10D, for example, such input to the classifier implemented as the neural network 2100 may include sensor data which is converted from analog data to digital data and processed in various ways. The neural network 2100 includes multiple hidden layers 2122a, 2122b to 2122n. The hidden layers 2122a, 2122b to 2122n contain a number of “n” hidden layers, where “n” is an integer greater than or equal to 1. The number of hidden layers may be the same as the number of layers required for a given application. The neural network 2100 further includes an output layer 2124 that provides an output obtained as a result of processing performed by hidden layers 2122a, 2122b through 2122n. In one exemplary example, the output layer 2124 may provide a classification of contact or vibration type based on sensor data that is processed and used as input to the neural network 2100. The classification may include classes that identify details associated with the input data (e.g., sound input, minor contact, violent collision, etc.).

[0146] The neural network 2100 is a multilayer neural network of interconnected nodes. Each node can represent one piece of information. The information associated with a node is shared between different layers, and each layer holds the information as it is processed. In some cases, the neural network 2100 may include a feedforward network, in which case there are no feedback connections where the output of the network is fed back to the network itself. In some cases, the neural network 2100 may include a recurrent neural network, which may have loops that allow information to be carried between nodes while being read at the input.

[0147] Information can be exchanged between nodes through node-to-node interconnections between various layers. Nodes in input layer 2120 can activate a set of nodes in the first hidden layer 2122a. For example, as shown in the figure, each input node in input layer 2120 is connected to each node in the first hidden layer 2122a. Nodes in hidden layers 2122a, 2122b through 2122n can transform the information of each input node by applying an activation function to the information. The information derived from this transformation can then be passed to the nodes of the next hidden layer 2122b, and those nodes can be activated, allowing them to perform their own specified functions. Illustrative functions include convolution, upsampling, data transformation, and / or any other suitable function. The output of hidden layer 2122b can then activate the nodes of the next hidden layer, and so on. The output of the last hidden layer 2122n can activate one or more nodes in output layer 2124, providing outputs at those nodes. In some cases, a node in the neural network 2100 (e.g., node 2126) is shown as having multiple output lines, but the node actually has a single output, and all lines shown as outputs from the node represent the same output value.

[0148] In some cases, each node, or the interconnections between nodes, may have weights, which are sets of parameters derived from training the neural network 2100. Once the neural network 2100 is trained, it can be called a trained neural network and can be used to classify one or more objects. For example, the interconnections between nodes may represent a piece of information learned about the interconnected nodes. By having tunable numerical weights that can be adjusted (for example, based on the training dataset), the neural network 2100 can be adaptive to inputs and can learn more as more data is processed.

[0149] The neural network 2100 is pre-trained to process features from data in the input layer 2120 using various hidden layers 2122a, 2122b through 2122n to provide an output via the output layer 2124. In one embodiment where the neural network 2100 is used to identify collision or audio types from MEMS sensor inputs, the neural network 2100 can be trained using training data that includes both images and labels. For example, the training data may be input to the network with training that has classification labels indicating a class (e.g., collision or sensor pattern type) associated with the training data.

[0150] In some cases, the neural network 2100 can adjust the node weights using a training process called backpropagation. Backpropagation may include a forward pass, loss function, backward pass, and weight updates. The forward pass, loss function, backward pass, and parameter updates are performed for one training iteration. This process can be repeated for each set of training images over a certain number of iterations until the neural network 2100 is sufficiently trained so that the layer weights are precisely adjusted.

[0151] The neural network 2100 may include any suitable deep network. One example is a convolutional neural network (CNN) which includes an input layer and an output layer, with multiple hidden layers between the input and output layers. The hidden layers of a CNN include a series of convolutional layers, nonlinear layers, pooling layers (for downsampling), and fully connected layers. The neural network 2100 may include any other deep networks besides CNNs, such as autoencoders, deep belief networks (DBNs), and recurrent neural networks (RNNs).

[0152] An exemplary CNN may receive input data (e.g., digital data processed as normalized and / or processed using a similarity measure, or processed using any operation described herein). The input data may include an array of digits representing data received from a MEMS sensor, where each digit in the array is associated with a characteristic of the sensor data received during a time frame. In some embodiments, data from multiple time frames may be processed sequentially or in parallel as part of the CNN classification.

[0153] In some embodiments, training of one or more of the machine learning systems or neural networks described herein (e.g., among the various other machine learning networks described herein with respect to Figures 1A to 21, such as the machine learning (ML) engine 7 in Figure 1A and the ASIC chip 16E in Figure 1E) may be carried out using online training (e.g., possibly on-device training), offline training, and / or various combinations of online and offline training. In some cases, online may refer to a period during which input data (e.g., data from a microphone, audio data, and / or other input data described herein) is processed to detect vibrations associated with an object surface implemented by the systems and technologies described herein (e.g., by a piezoelectric microelectromechanical system (MEMS) vibration sensing device described herein). In some examples, offline may refer to an idle period or a period during which no input data is being processed. Furthermore, offline may be based on one or more time conditions (e.g., after a certain period of time has elapsed, such as a day, a week, or a month), and / or on various other conditions, such as network and / or server availability, among other things. In some embodiments, offline training of a machine learning model (e.g., a neural network model) may be performed by a first device (e.g., a server device) to generate a pre-trained model, and a second device may receive the trained model from the second device. In some cases, the second device (e.g., a mobile device, an XR device, a vehicle or a vehicle system / component, or other device) may perform online (or on-device) training of the pre-trained model to further adapt or synchronize the model's parameters.

[0154] Figure 22 shows an example of a system implementing a particular aspect of the present technology. In particular, Figure 22 shows an example of a computing system 2200 that may include a piezoelectric MEMS sensor system (e.g., at least one piezoelectric MEMS transducer system including at least one piezoelectric MEMS acoustic sensor or microphone and a piezoelectric MEMS acoustic transducer in the feedback conversion configuration described above) according to the aspects described herein. The acoustic transducer (e.g., a piezoelectric MEMS acoustic transducer and associated MEMS transducer system) can be integrated with, for example, an internal computing system, a remote computing system, a camera, or any computing device that constitutes any component thereof, and the components of the system communicate with each other using connection 2205. Connection 2205 may be a physical connection using a bus, or a direct connection to a processor 2210, such as in a chipset architecture. Connection 2205 may also be a virtual connection, a networked connection, or a logical connection.

[0155] An exemplary system 2200 includes at least one processing unit (CPU or processor) 2210 and connections 2205 that communicatively connect various system components, including system memory 2215 such as read-only memory (ROM) 2220 and random access memory (RAM) 2225, to the processor 2210. The computing system 2200 may include a high-speed memory cache 2212 that is directly connected to the processor 2210, connected in close proximity to the processor 2210, or integrated as part of the processor 2210.

[0156] The processor 2210 may include an arbitrary general-purpose processor, hardware or software services such as services 2232, 2234, and 2236 stored in a memory device 2230 configured to control the processor 2210, and a dedicated processor in which software instructions are incorporated into the actual processor design. The processor 2210 can essentially be a completely self-contained computing system including multiple cores or processors, buses, memory controllers, caches, etc. The multicore processor can be symmetric or asymmetric.

[0157] To enable user interaction, the computing system 2200 includes an input device 2245 that can represent any number of input mechanisms, such as a microphone for voice or audio detection (e.g., a piezoelectric MEMS converter or MEMS converter system in the manner described above), along with other input devices 2245 such as a touch-sensitive screen, keyboard, mouse, motion input, and speech for gesture or graphical input. The computing system 2200 may also include an output device 2235 that may be one or more of several output mechanisms. In some cases, a multimodal system may allow the user to provide multiple types of input / output to communicate with the computing system 2200.

[0158] The computing system 2200 may include a communication interface 2240 that can comprehensively control and manage user input and system output. Communication interfaces include audio jacks / plugs, microphone jacks / plugs, universal serial bus (USB) ports / plugs, Apple® Lightning® ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, proprietary wired ports / plugs, 3G, 4G, 5G, and / or other cellular data network wireless signal transmission, Bluetooth® wireless signal transmission, Bluetooth® Low Energy (BLE) wireless signal transmission, IBEACON® wireless signal transmission, radio-frequency identification (RFID) wireless signal transmission, near-field communications (NFC) wireless signal transmission, dedicated short-range communication (DSRC) wireless signal transmission, 802.11 Wi-Fi wireless signal transmission, wireless local area network (WLAN) signal transmission, Visible Light Communication (VLC), and Worldwide Interoperability for Microwave. Reception and / or transmission of wired or wireless communications using wired and / or wireless converters, including but not limited to Access (WiMAX), infrared (IR) wireless signal transmission, Public Switched Telephone Network (PSTN) signal transmission, Integrated Services Digital Network (ISDN) signal transmission, ad hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or any combination thereof, may be performed or facilitated.The communication interface 2240 may also include one or more GNSS receivers or transducers used to determine the location of the computing system 2200 based on the reception of one or more signals from one or more satellites associated with one or more Global Navigation Satellite System (GNSS) systems. GNSS systems include, but are not limited to, the United States-based Global Positioning System (GPS), the Russia-based Global Navigation Satellite System (GLONASS), the China-based Beidou Navigation Satellite System (BDS), and the Europe-based Galileo GNSS. There are no restrictions on operation on any particular hardware configuration, and therefore, the fundamental features described herein can be easily superseded as improved hardware or firmware configurations are developed.

[0159] The storage device 2230 may be a non-volatile memory device and / or a non-temporary memory device and / or a computer-readable memory device, such as a magnetic cassette, flash memory card, solid memory device, digital multipurpose disk, cartridge, floppy disk, flexible disk, hard disk, magnetic tape, magnetic strip / stripe, any other magnetic storage medium, flash memory, memristor memory, any other solid memory, compact disc read-only memory (CD-ROM) optical disc, rewritable compact disc (CD) optical disc, digital video disc (DVD) optical disc, Blu-ray disc (BDD) optical disc, holographic optical disc, another optical medium, secure digital (SD) card, micro secure digital (microSD) card, Memory Stick® card, smart card chip, EMV chip, subscriber identification module (SIM) card, mini / micro / nano / pico SIM card, or another integrated circuit (integrated Circuits, ICs, chips / cards, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash EPROM, cache memory (e.g., Level 1 (L1) cache, Level 2 (L2) cache, Level 3 (L3) cache, Level 4 (L4) cache, Level 5 (L5) cache, or other (L#) caches), resistive random-access memoryThis may be a hard disk or other type of computer-readable medium capable of storing computer-accessible data, such as memory (RRAM / ReRAM), phase-change memory (PCM), spin-transfer torque RAM (STT-RAM), other memory chips or cartridges, and / or any combination thereof.

[0160] The storage device 2230 may include software services, servers, and other services that cause the processor 2210 to perform functions in the system when code defining such software is executed by the processor 2210. In some embodiments, a hardware service that performs a particular function may include software components stored in a computer-readable medium in relation to hardware components necessary to perform that function, such as the processor 2210, connection 2205, and output device 2235. The term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instructions and / or data. The computer-readable medium may include non-transient media capable of storing data, and such non-transient media does not include carrier waves and / or transient electronic signals propagating wirelessly or via wired connections. Examples of non-temporary media include, but are not limited to, optical storage media such as magnetic disks or magnetic tapes, compact discs (CDs) or digital versatile disks (DVDs), flash memory, memory, or memory devices. Computer-readable media can store code and / or machine-executable instructions that may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments can be coupled to other code segments or hardware circuits by passing and / or receiving information, data, arguments, parameters, or memory content. Information, arguments, parameters, data, etc., can be passed, transferred, or transmitted via any preferred means, including memory sharing, message passing, token passing, or network transmission.

[0161] Certain details are provided in the above description to provide a complete understanding of the embodiments and examples provided herein; however, those skilled in the art will understand that this application is not limited thereto. Therefore, while the exemplary embodiments of this application have been described in detail herein, it should be understood that the concepts of the present invention can be embodied and employed in various other ways, except as limited by the prior art, and that the appended claims are intended to be interpreted to include such variations. The various features and aspects of this application described above may be used individually or in combination. Furthermore, embodiments may be used in any number of environments and applications other than those described herein without departing from the broader scope of this specification. Thus, this specification and the drawings should be considered illustrative and not limiting. For illustrative purposes, the methods are described in a particular order. It should be understood that in alternative embodiments, those methods may be carried out in an order different from that described.

[0162] For clarity of explanation, in some cases the technology may be presented as including individual functional blocks, which include a device, device components, and steps or routines of a method embodied in software or a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may also be used. For example, circuits, systems, networks, processes, and other components may be shown as components in the form of block diagrams, so as not to obscure the embodiments with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details, in order to avoid obscuring the embodiments.

[0163] Furthermore, those skilled in the art will understand that various exemplary logic blocks, modules, circuits, and algorithmic steps described in relation to the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware-software compatibility, various exemplary components, blocks, modules, circuits, and steps have been outlined above in relation to their functionality. Whether such functionality is implemented as hardware or executed as software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for specific applications, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure.

[0164] Individual embodiments may be described above as processes or methods shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations may be performed in parallel or simultaneously. Furthermore, the order of operations can be rearranged. A process terminates when its operations are complete, but it may have additional steps not shown in the diagram. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to a calling function or main function.

[0165] The processes and methods described above may be implemented using computer-executable instructions stored in or otherwise available from computer-readable media. Such instructions may include, for example, instructions and data that cause a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or set of functions, or otherwise configure a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or set of functions. The portion of computer resources used may be accessible via a network. Computer-executable instructions may be, for example, binaries, or intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that can be used to store the instructions, information used, and / or information created during the execution of the methods described may include magnetic disks or optical disks, flash memory, USB devices with non-volatile memory, and networked storage devices.

[0166] In some embodiments, computer-readable storage devices, media, and memory may include cables or wireless signals, such as bitstreams. However, non-temporary computer-readable storage media, as referred to, explicitly exclude media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0167] Those skilled in the art will understand that information and signals can be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips, which may be mentioned throughout the above description, can in some cases be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof, depending in part on the specific application, desired design, corresponding technology, etc.

[0168] The various exemplary logic blocks, modules, and circuits described in relation to the embodiments disclosed herein may be implemented or executed using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take on any of the various form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segment (e.g., a computer program product) that performs the required tasks may be stored in a computer-readable or machine-readable medium. One or more processors may perform the required tasks. Examples of form factors include laptops, smartphones, mobile phones, tablet devices or other space-saving personal computers, personal digital assistants, rack-mount devices, and standalone devices. The functions described herein may also be embodied in peripheral devices or add-in cards. Such functions may also, as a further example, be implemented on circuit boards of different chips or on different processes running in a single device.

[0169] Instructions, a medium for transmitting such instructions, computing resources for executing those instructions, and other structures supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0170] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as general-purpose computers, wireless communication device handsets, or integrated circuit devices with multiple applications, including applications in wireless communication device handsets and other devices. Any feature described as a module or component may be implemented as a whole in an integrated logic device, or separately as individual but interoperable logic devices. When implemented in software, these techniques may be implemented at least in part by a computer-readable data storage medium having program code containing instructions, which, when executed, perform one or more of the methods, algorithms, and / or operations described above. The computer-readable data storage medium may also form part of a computer program product, which may include packaging materials. Computer-readable media may include memory or data storage media such as random access memory (RAM) including synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electro-erasable programmable read-only memory (EEPROM), flash memory, and magnetic or optical data storage media. The technique may be at least partially implemented by computer-readable communication media, such as propagated signals or waves, which carry or communicate program code in the form of instructions or data structures and can be accessed, read, and / or executed by a computer.

[0171] The program code can be executed by a processor, which may include one or more processors such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor may be configured to implement any of the techniques described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, as used herein, the term “processor” may refer to any of the structures described herein, any combination thereof, or any other structure or device suitable for implementing the techniques described herein.

[0172] When components are described as "configured to perform certain operations," such configurations can be achieved, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform the operations, or by any combination thereof.

[0173] The phrases “connected to ~” or “communicatively connected to ~” refer to any component that is physically connected to another component, either directly or indirectly, and / or any component that communicates with another component, either directly or indirectly (for example, connected to another component via a wired or wireless connection and / or other suitable communication interface).

[0174] Any claim wording that includes "at least one of" and / or "one or more" of a set indicates that one member of that set, or multiple members of that set (in any combination), satisfy the claim. For example, any claim wording that includes "at least one of A and B" or "at least one of A or B" means A, B, or A and B. In another example, any claim wording that includes "at least one of A, B, and C" or "at least one of A, B, or C" means A, B, C, or A and B, or A and C, or B and C, or A and B and C, or any overlapping information or data (e.g., A and A, B and B, C and C, A and A and B, etc.), or any other order, overlap, or combination of A, B, and C. The phrases "at least one of" and / or "one or more" of a set do not limit the set to items listed within it. For example, the claim wording "at least one of A and B" or "at least one of A or B" could mean A, B, or A and B, and may also include items not listed within the set of A and B. The phrases "at least one" and "one or more" are used interchangeably herein.

[0175] The language of a claim that includes phrases such as “at least one processor configured to do so,” “one or more processors configured to do so,” or “one or more processors configured to do so,” indicates that one or more processors (in any combination) can perform the associated operations (one or more). For example, the wording of a claim that includes “at least one processor configured to do X, Y, and Z” means that a single processor can perform operations X, Y, and Z, or that each of several processors can task a subset of operations X, Y, and Z, or that a group of several processors can work together to perform operations X, Y, and Z, so that several processors perform X, Y, and Z together. In another example, the wording of a claim that includes “at least one processor configured to do X, Y, and Z” may mean that any single processor can perform only one subset of operations X, Y, and Z.

[0176] When one or more elements are mentioned that perform a function (e.g., steps of a method), one element may perform all of the functions, or two or more elements may collectively perform the functions. When two or more elements collectively perform a function, each function does not have to be performed by each of those elements (e.g., different functions may be performed by different elements), and / or each function does not have to be performed entirely by only one element (e.g., different elements may perform different subfunctions of a function). Similarly, when one or more elements are mentioned that are configured to cause another element (e.g., a device) to perform a function, one element may be configured to cause the other elements to perform all of the functions, or two or more elements may collectively be configured to cause the other elements to perform the functions.

[0177] Where referring to an entity (e.g., any entity or device described herein) that performs a function or is configured to perform a function (e.g., a step of a method), the entity may be configured to have one or more elements perform the function (individually or collectively). One or more components of the entity may include at least one memory, at least one processor, at least one communication interface, another component configured to perform one or more (or all) of the functions, and / or any combination thereof. Where referring to an entity that performs a function, the entity may be configured to have one component perform all the functions, or to have two or more components collectively perform the functions. When an entity is configured to have two or more components collectively perform the functions, each function does not have to be performed by each of those components (e.g., different functions may be performed by different components), and / or each function as a whole does not have to be performed by only one component (e.g., different components may perform different subfunctions of the function).

[0178] Several implementations have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Other embodiments are within the scope of the claims.

[0179] Exemplary embodiments of this disclosure include the following:

[0180] Embodiment 1. A system comprising a motion sensor, a microphone, a machine learning engine, and at least one package including the motion sensor, microphone, and machine learning engine, the package having a base for fixing the motion sensor and microphone to a surface, wherein the machine learning engine is configured to be trained to distinguish different types of contact on a surface.

[0181] Embodiment 2. The system according to Embodiment 1, wherein the base has solder pads for connecting the package to a printed circuit board located within the housing, and the housing is bonded to the surface.

[0182] Embodiment 3. An apparatus comprising: a motion sensor; a microphone; a machine learning engine; and at least one package comprising the motion sensor; a microphone; and a machine learning engine, the package having a base for fixing the motion sensor and microphone to a surface, and trained to distinguish different types of contact on the surface.

[0183] Embodiment 4. The apparatus according to Embodiment 3, wherein the base has solder pads for connecting the package to a printed circuit board located within the housing, and the housing is bonded to the surface.

[0184] Embodiment 5. The apparatus according to Embodiment 3 or 4, wherein a motion sensor, a microphone, and a machine learning engine are located in a single package.

[0185] Embodiment 6. The apparatus according to any one of Embodiments 3 to 5, wherein the motion sensor and microphone are located in a first package, and the machine learning engine is located in a second package and electrically coupled to the first package.

[0186] Embodiment 7. The apparatus according to any one of Embodiments 3 to 6, wherein the motion sensor and microphone are located on a first die, the machine learning engine is located on a second die, and the first and second dies are located in the same package.

[0187] Embodiment 8. The apparatus according to any one of Embodiments 3 to 7, wherein the motion sensor, microphone, and machine learning engine are formed on a single die.

[0188] Embodiment 9. The apparatus according to any one of Embodiments 3 to 8, wherein the microphone includes a piezoelectric MEMS microphone.

[0189] Embodiment 10. The apparatus according to any one of Embodiments 3 to 9, wherein the motion detector comprises an accelerometer or a piezoelectric MEMS microphone with its aperture closed.

[0190] Embodiment 11. The apparatus according to any one of Embodiments 3 to 10, wherein the motion sensor has a bandwidth between 3 kilohertz and 8 kilohertz.

[0191] Embodiment 12. The apparatus according to any one of Embodiments 3 to 11, wherein the motion sensor has a noise floor between 100 μg / sqrt(Hz) and 0.5 μg / sqrt(Hz) at 1 kHz, and between 50 μg / sqrt(Hz) and 5 μg / sqrt(Hz) at 1 kHz.

[0192] Embodiment 13. The apparatus according to any one of Embodiments 3 to 12, wherein the motion sensor has a noise floor between 100 μg / sqrt(Hz) and 0.05 μg / sqrt(Hz) at device resonance and between 5 μg / sqrt(Hz) and 0.05 μg / sqrt(Hz) at resonance.

[0193] Embodiment 14. The apparatus according to any one of Embodiments 3 to 13, wherein different types of contact include no contact, touch, damage, and / or hard touch.

[0194] Embodiment 15. The apparatus according to any one of Embodiments 3 to 14, wherein the second motion sensor and the second microphone are located in a second package set, the second package set is configured to be coupled to a surface, and the system is further configured to determine the location and / or direction of contact on the surface.

[0195] Embodiment 16. The apparatus according to any one of Embodiments 3 to 15, wherein the surface functions as a touch surface / sensor.

[0196] Embodiment 17. A system comprising: a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, the first piezoelectric MEMS transducer being mechanically coupled to the surface of an object and configured to generate a first analog signal at the first output when the first analog signal is converted by the first piezoelectric MEMS transducer from vibrations propagating through the object; a second piezoelectric MEMS transducer having a second output, the second piezoelectric MEMS transducer being configured to generate a second analog signal at the second output when the second analog signal is converted by the second piezoelectric MEMS transducer from acoustic vibrations at the location of the object; and a classification circuit coupled to the output of the first piezoelectric MEMS transducer and the output of the second piezoelectric MEMS transducer, the classification circuit being configured to process data from the first analog signal and data from the second analog signal, and to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0197] Embodiment 18. The system according to Embodiment 17, wherein the first piezoelectric MEMS converter has a noise floor that defines noise at a given frequency related to the signal output in gravitational units (g), and the noise floor is between 100 parts per million of gravitational units (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 μg / sqrt(Hz).

[0198] Embodiment 19. The system according to Embodiment 17 or 18, wherein the first piezoelectric MEMS converter has a conversion bandwidth for detecting vibrations propagating through an object at frequencies between 1 kilohertz (kHz) and 8 kHz.

[0199] Embodiment 20. The system according to any one of Embodiments 17 to 19, wherein data from a first analog signal includes frequency data of vibrations propagating through an object and magnitude data of vibrations propagating through an object, the magnitude data being associated with the severity of contact with the object.

[0200] Embodiment 21. A system according to any one of Embodiments 17 to 20, wherein one or more time frames include a plurality of 20 millisecond (ms) frames.

[0201] Embodiment 22. The system according to any one of Embodiments 17 to 21, further comprising a first sensor package, the first sensor package comprising a substrate base and a lid, and a first piezoelectric MEMS converter, a second piezoelectric MEMS converter, and an application-specific integrated circuit (ASIC) mounted on the substrate base.

[0202] Embodiment 23. The system of Embodiment 22, wherein the ASIC comprises an analog-to-digital converter (ADC), a digital signal processor (DSP), and a controller, the output of a first piezoelectric MEMS converter is coupled to the input of the ADC via a wire bond, the output of the ADC is coupled to the input of the controller via the digital signal processor, and the output of the controller is coupled to a classification circuit.

[0203] Embodiment 24. The system according to any one of embodiments 21 to 23, further comprising a second sensor package comprising a third MEMS transducer and a fourth MEMS transducer, wherein the first sensor package is positioned at a first location on the surface of an object, and the second sensor package is positioned at a second location on the surface of an object at a predetermined distance from the first location.

[0204] Embodiment 25. The system according to Embodiment 24, wherein the classification circuit is further configured to detect the location of an impact on the surface of an object based on the time delay or magnitude difference between the vibration detected by the first sensor package and the vibration detected by the second sensor package.

[0205] Embodiment 26. The classification circuit is coupled to the output of a first piezoelectric MEMS converter and the output of a second piezoelectric MEMS converter via an application-specific integrated circuit (ASIC), the ASIC converts a first analog signal into a first plurality of data frames associated with one or more time frames, converts a second analog signal into a second plurality of data frames associated with one or more time frames, calculates the sum of the squares of the amplitude values ​​for each data frame of the first plurality of data frames in order to generate amplitude values ​​for the first piezoelectric MEMS converter for each of the one or more time frames, and calculates the sum of the square of the amplitude value of each data frame of the generated second plurality of data frames and the amplitude value of the second piezoelectric MEMS converter for each of the one or more time frames. The system is configured to generate data from a first analog signal and a second analog signal by: calculating the number of zero crossings for each data frame of a first plurality of data frames in order to generate a zero crossing value for a first piezoelectric MEMS converter for each of one or more time frames; calculating the number of zero crossings for each data frame of a second plurality of data frames in order to generate a zero crossing value for a second piezoelectric MEMS converter for each of one or more time frames; and calculating a ratio value for each of one or more time frames, wherein the ratio value is the ratio between the sum of the squares of the amplitudes of each data frame of the first plurality of data frames and the sum of the squares of the amplitudes of each data frame of the second plurality of data frames, according to embodiment 24 or 25.

[0206] Embodiment 27. The system according to any one of Embodiments 17 to 26, wherein the classification circuit is further configured to receive data from a first analog signal and data from a second analog signal as training data in training mode, and to match the data from the first analog signal and data from the second analog signal with a provided training classification value.

[0207] Embodiment 28. The system according to any one of Embodiments 17 to 27, wherein the object is a bumper and the surface is the outward-facing surface of the bumper.

[0208] Embodiment 29. The system according to Embodiment 27 or 28, wherein the training classification values ​​provided are collision classification values.

[0209] Embodiment 30. The system according to any one of embodiments 17 to 29, further comprising a control circuit coupled to a classification circuit, wherein the control circuit is configured to automatically generate an alert in response to receiving a collision classification output from the classification circuit during an operating mode.

[0210] Embodiment 31. The system according to Embodiment 27 or 28, wherein the training classification value provided is a door closing value, and a control circuit coupled to the classification circuit is configured to generate a record of the timing of the door closing value during the operating mode.

[0211] Embodiment 32. The system according to Embodiment 27 or 28, wherein the training classification value provided is a key scratch value, and a control circuit coupled to the classification circuit is configured to initiate video recording of a region surrounding a surface in response to the key scratch value during the operating mode.

[0212] Embodiment 33. The system according to any one of Embodiments 17 to 32, wherein the object is an element of a robotic arm, a wall of a storage container, a wall of a building, a hull panel of a ship, or a fuselage panel of an airplane.

[0213] Embodiment 34. The system according to any one of Embodiments 17 to 33, wherein the classification circuit includes one or more of a decision tree circuit, a support vector machine, or a neural network.

[0214] Embodiment 35. Storing data from a first analog signal generated by a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, wherein the first piezoelectric MEMS transducer is mechanically coupled to a first surface of an object, and the first piezoelectric MEMS transducer is configured to generate the first analog signal at a first output when the first analog signal is converted from vibrations propagating through the object by the first piezoelectric MEMS transducer; and storing data from a second piezoelectric MEMS transducer having a second output, A method comprising: storing a second analog signal, the second piezoelectric MEMS converter configured to generate a second analog signal at the second output when the second analog signal is converted by the second piezoelectric MEMS converter from acoustic vibrations incident on a first surface of an object; and processing data from the first analog signal and data from the second analog signal using a classification circuit coupled to the output of the first piezoelectric MEMS converter and the output of the second piezoelectric MEMS converter in order to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0215] Embodiment 36. The method according to Embodiment 35, further comprising processing the first analog signal and the second analog signal using a digital signal processor (DSP) and an analog-to-digital converter (ADC) to generate data from the first analog signal and data from the second analog signal as digital data.

[0216] Embodiment 37. A system comprising: means for generating a first analog signal converted from vibrations propagating through an object having a first surface; means for generating a second analog signal converted from an acoustic signal incident on the first surface of the object; and means for processing data from the first analog signal and data from the second analog signal in order to classify combinations of the first analog signal and the second analog signal received in one or more time frames.

[0217] Embodiment 38. The system according to Embodiment 37, wherein the means for generating a first analog signal has a noise floor that defines noise at a given frequency related to the signal output in gravitational units (g), and the noise floor is between 100 parts per million of a gravitational unit (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 ug / sqrt(Hz).

[0218] Embodiment 39. A system comprising a motion sensor, a microphone, a machine learning engine, and at least one package including the motion sensor, microphone, and machine learning engine, the package having a base for fixing the motion sensor and microphone to a surface, wherein the machine learning engine is configured to be trained to distinguish different types of contact on a surface.

[0219] Embodiment 40. The system according to Embodiment 39, wherein the base has solder pads for connecting at least one package to a printed circuit board located within the housing, and the housing is bonded to the surface.

[0220] Embodiment 41. The system according to Embodiment 39 or 40, wherein the motion sensor, microphone, and machine learning engine are located in a single package.

[0221] Embodiment 42. The system according to Embodiment 39 or 40, wherein the motion sensor and microphone are located on a first die, the machine learning engine is located on a second die, and the first and second dies are located in a single package.

[0222] Embodiment 43. The system according to Embodiment 39 or 40, wherein the motion sensor and microphone are located in a first package, and the machine learning engine is located in a second package and electrically coupled to the first package.

[0223] Embodiment 44. A system according to any one of Embodiments 39 to 41, wherein a motion sensor, a microphone, and a machine learning engine are formed on a single die.

[0224] Embodiment 45. The system according to any one of Embodiments 39 to 44, wherein the microphone includes a piezoelectric MEMS microphone.

[0225] Embodiment 46. The system according to any one of Embodiments 39 to 45, wherein the motion sensor comprises an accelerometer or a piezoelectric MEMS microphone having a closed aperture.

[0226] Embodiment 47. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors are configured to acquire audio signals, which are generated based on sound detection by a microphone; acquire motion signals, which are generated based on motion detection by a motion sensor mounted on the surface of an object; perform similarity measurements based on the audio signals and motion signals; and determine the context of the contact type on the surface of the object based on the similarity measurements.

[0227] Embodiment 48. The device according to Embodiment 1, wherein one or more processors are configured to perform a similarity measurement based on a first comparison between an audio signal representation and a motion signal representation.

[0228] Embodiment 49. The device according to Embodiment 2, wherein the first comparison is the difference between the representation of an audio signal and the representation of a motion signal.

[0229] Embodiment 50. The device according to Embodiment 2, wherein the first comparison is the ratio of the representation of the audio signal to the representation of the motion signal.

[0230] Embodiment 51. The device according to Embodiment 2, wherein the representation of the audio signal is a first correlation and the representation of the motion signal is a second correlation.

[0231] Embodiment 52. The device according to Embodiment 2, wherein the representation of the audio signal is based on the rectification of the audio signal, such that it is acquired by one or more processors.

[0232] Embodiment 53. The device according to Embodiment 2, wherein the first comparison between the representation of an audio signal and the representation of a motion signal is based on a second comparison between the representation of an audio signal and an audio threshold, and a third comparison between the representation of a motion signal and a motion threshold.

[0233] Embodiment 54. The device according to Embodiment 2, wherein determining the context of the contact type on the surface of an object includes classifying the contact type based on a combination of the representation of an audio signal and the representation of a motion signal.

[0234] Embodiment 55. The device according to Embodiment 54, wherein determining the context of the contact type on the surface of an object includes classifying the contact type based on the magnitude of the contact.

[0235] Embodiment 56. The device according to Embodiment 55, wherein the context of the type of contact of the surface of the object includes at least one of scratch, dent, touch, non-contact touch, damage, and hard touch.

[0236] Embodiment 57. The device according to Embodiment 56, wherein determining the context of the contact type on the surface of an object includes comparing the output of a machine learning engine with the past context type of the contact determined by the machine learning engine.

[0237] Embodiment 58. The device according to Embodiment 57, wherein the machine learning engine is one of a decision tree, a support vector machine, or a neural network.

[0238] Embodiment 59. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors are configured to acquire audio signals based on sound detection by a microphone, acquire motion signals based on motion detection by a motion sensor mounted on the surface of an object, quantify the frequency characteristics of the audio signals and motion signals, quantify the amplitude characteristics of the audio signals and motion signals, perform one or more comparisons between the audio signals and motion signals to generate comparison data, and classify the contact type associated with contact on the surface of an object based on the comparison data.

[0239] Embodiment 60. The device according to Embodiment 59, further comprising storing relative position information of a microphone and a motion sensor in memory, wherein one or more comparisons of an audio signal and a motion signal use the relative position information to generate comparison data.

[0240] Embodiment 61. The device according to Embodiment 60, wherein the memory is further configured to store multiple audio signals from multiple microphones, including a microphone, and relative position information further includes the relative positions of the multiple microphones, and comparison data is further generated using the multiple audio signals and the relative position information of the multiple microphones.

[0241] Embodiment 62. The device according to Embodiment 61, wherein one or more processors are configured to implement a machine learning engine trained to select a contact type from a plurality of contact types using comparison data.

[0242] Embodiment 63. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors are configured to acquire audio signals based on sound detection by a microphone, acquire motion signals based on motion detection by a motion sensor mounted on the surface of an object, generate digital correlation data for the audio signals, generate digital correlation data for the motion signals, generate combined correlation data for the audio signals and motion signals, and select a classification based on the combined correlation data.

[0243] Apparatus 64. The device according to Apparatus 63, further based on the magnitude of the audio signal and the magnitude of the motion signal.

[0244] Embodiment 65. The device according to Embodiment 63, wherein the classification is selected from a first set of classifications, including scratch classification, dent classification, touch classification, and contactless classification.

[0245] Embodiment 66. The device according to Embodiment 65, wherein the classification includes a first value from a first classification set and a second value from a second classification set, the second classification set including damaged classifications and undamaged classifications.

[0246] Embodiment 67. A method comprising the operation of storing an audio signal and a motion signal, acquiring an audio signal which is generated based on sound detection by a microphone, acquiring a motion signal which is generated based on motion detection by a motion sensor mounted on the surface of an object, performing a similarity measurement based on the audio signal and the motion signal, and determining the contact type context of the surface of the object based on the similarity measurement.

[0247] Embodiment 68. The method according to Embodiment 67, wherein one or more processors are configured to perform a similarity measurement based on a first comparison between an audio signal representation and a motion signal representation.

[0248] Apparatus 69. The method according to Apparatus 68, wherein the first comparison is the difference between the representation of an audio signal and the representation of a motion signal.

[0249] Apparatus 70. The method according to Apparatus 68, wherein the first comparison is the ratio of the representation of the audio signal to the representation of the motion signal.

[0250] Embodiment 71. The method according to Embodiment 68, wherein the representation of the audio signal is a first correlation and the representation of the motion signal is a second correlation.

[0251] Embodiment 72. The method of Embodiment 68, wherein the representation of the audio signal is based on the rectification of the audio signal, such that the representation is obtained by one or more processors.

[0252] Embodiment 73. The method according to Embodiment 68, wherein the first comparison between the representation of the audio signal and the representation of the motion signal is based on a second comparison between the representation of the audio signal and an audio threshold, and a third comparison between the representation of the motion signal and a motion threshold.

[0253] Embodiment 74. The method of Embodiment 68, wherein determining the context of the contact type on the surface of an object includes classifying the contact type based on a combination of the representation of an audio signal and the representation of a motion signal.

[0254] Embodiment 75. The method according to Embodiment 74, wherein determining the context of the type of contact on the surface of an object includes classifying the type of contact based on the magnitude of the contact.

[0255] Embodiment 76. The method according to Embodiment 75, wherein the context of the type of contact of the surface of the object includes at least one of scratch, dent, touch, non-contact touch, damage, and hard touch.

[0256] Embodiment 77. The method of Embodiment 68, wherein determining the context of the contact type on the surface of an object includes comparing the output of a machine learning engine with the past context type of the contact determined by the machine learning engine.

[0257] Embodiment 78. The method according to Embodiment 77, wherein the machine learning engine is one of a decision tree, a support vector machine, or a neural network.

[0258] Embodiment 79. A method comprising the operation of storing an audio signal and a motion signal, acquiring an audio signal based on sound detection by a microphone, acquiring a motion signal based on motion detection by a motion sensor mounted on the surface of an object, quantifying the frequency characteristics of the audio signal and the motion signal, quantifying the amplitude characteristics of the audio signal and the motion signal, performing one or more comparisons between the audio signal and the motion signal to generate comparison data, and classifying the contact type associated with contact on the surface of an object based on the comparison data.

[0259] Embodiment 80. The method according to Embodiment 79, further comprising the step of storing relative position information of a microphone and a motion sensor in memory, wherein one or more comparisons of an audio signal and a motion signal use the relative position information to generate comparison data.

[0260] Embodiment 81. The method according to Embodiment 80, wherein the memory is further configured to store multiple audio signals from multiple microphones, including a microphone, and relative position information further includes the relative positions of the multiple microphones, and comparison data is further generated using the multiple audio signals and the relative position information of the multiple microphones.

[0261] Embodiment 82. The method according to Embodiment 79, wherein one or more processors are configured to implement a machine learning engine that is trained to select a contact type from a plurality of contact types using comparison data.

[0262] Embodiment 83. A method comprising the operation of storing an audio signal and a motion signal, acquiring an audio signal based on sound detection by a microphone, acquiring a motion signal based on motion detection by a motion sensor mounted on the surface of an object, generating digital correlation data for the audio signal, generating digital correlation data for the motion signal, generating combined correlation data for the audio signal and the motion signal, and selecting a classification based on the combined correlation data.

[0263] Embodiment 84. The method according to Embodiment 83, wherein the classification is further based on the magnitude of the audio signal and the magnitude of the motion signal.

[0264] Embodiment 85. The method according to Embodiment 83, wherein the classification is selected from a first set of classifications including scratch classification, indentation classification, touch classification, and non-contact classification.

[0265] Embodiment 86. The method according to Embodiment 85, wherein the classification comprises a first value from a first classification set and a second value from a second classification set, the second classification set comprising damaged classifications and undamaged classifications.

[0266] Embodiment 84. A micro-electromechanical (MEMS) converter comprising means for providing an output signal according to any of the embodiments described above.

[0267] Appendix 85. A method for operating any MEMS converter described herein.

[0268] Embodiment 86. A storage medium that, when executed by one or more processors of a system, includes instructions that cause the system to perform any of the operations described herein.

[0269] Additional aspects are exemplified by the following clauses:

[0270] Clause 1. A device comprising a memory configured to store an audio signal and a motion signal, and one or more processors, wherein the one or more processors obtain an audio signal based on sound detection by two or more microphones, obtain a motion signal based on motion detection by two or more motion sensors mounted on the surface of a first object, perform one or more comparisons between the audio signal and the motion signal to generate comparison data, determine a context of a contact type on the surface of the first object based on the comparison data, and determine the location of a second object within a threshold distance to the first object based on the context determined by the one or more processors.

[0271] Clause 2. The device according to Clause 1, wherein a plurality of cantilever beams are configured as a film surrounding the sensor area.

[0272] Clause 3. The device according to Clause 1 or 2, wherein the first object is a car door.

[0273] Clause 4. The device according to any one of Clauses 1 to 3, wherein the second object is a person, a key, or a balloon.

[0274] Clause 5. The device according to any one of Clauses 1 to 4, wherein the contact type includes an area associated with the first object.

[0275] Clause 6. The device according to Clause 5, wherein the first object is a door, and the area associated with the first object is one of the upper right part of the door, the lower right part of the door, the upper left part of the door, the lower left part of the door, or the center of the door.

[0276] Clause 7. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein the one or more processors obtain an audio signal based on detection of sound by a microphone, obtain a motion signal based on detection of motion by a motion sensor mounted on a surface of an object, normalize the audio signal and the motion signal to generate a normalized audio signal and a normalized motion signal, generate correlation data from the normalized audio signal and the normalized motion signal, and determine a contact classification using the correlation data.

[0277] Clause 8. The device according to Clause 7, wherein the one or more processors are configured as a classification circuit that determines a contact classification using the correlation data.

[0278] Clause 9. The device according to Clause 8, wherein the classification circuit is further configured to receive, in a training mode, an audio signal and a motion signal as training data and match the training data to a provided training classification value.

[0279] Clause 10. The device according to Clause 9, wherein the object is an exterior surface of a motor vehicle, and the provided training classification value is a set of collision classification values having known force and damage type characteristics.

[0280] Clause 11. Memory, a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, the first piezoelectric MEMS transducer being mechanically coupled to the surface of an object, and configured to generate a first analog signal at the first output when the first analog signal is converted from vibrations propagating through the object by the first piezoelectric MEMS transducer, and a second piezoelectric MEMS transducer having a second output, the second piezoelectric MEMS transducer being configured to generate a second analog signal when the second analog signal is converted from a second piezoelectric MEMS transducer. A device comprising: a second piezoelectric MEMS converter configured to generate a second output when converted from acoustic vibrations at the location of an object by an EMS converter; and a classification circuit coupled to a memory, the output of the first piezoelectric MEMS converter, and the output of the second piezoelectric MEMS converter, the classification circuit being configured to process data from a first analog signal and data from a second analog signal, and to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0281] The device described in Clause 11, further comprising storing categorization data from a classification circuit into memory.

[0282] Clause 13. The device according to Clause 11 or 12, wherein the first piezoelectric MEMS transducer has a noise floor that defines noise at a given frequency relating to the signal output in gravitational units (g), and the noise floor is between 100 parts per million of gravitational units (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 μg / sqrt(Hz).

[0283] Clause 14. The first piezoelectric MEMS converter is the device described in Clause 13, having a conversion bandwidth for detecting vibrations propagating through an object at frequencies between 0.5 kilohertz (kHz) and 15 kHz.

[0284] Clause 15. A device as described in any of Clauses 11 to 14, wherein the data from the first analog signal includes frequency data of vibrations propagating through an object and magnitude data of vibrations propagating through an object, the magnitude data being associated with the severity of contact with the object.

[0285] Clause 16. One or more time frames are captured at a rate greater than 60 frames per second by any of the devices described in Clauses 11 through 15.

[0286] The device according to any one of the clauses 11 to 16, further comprising a first sensor package, the first sensor package comprising a substrate base and a lid, and a first piezoelectric MEMS converter, a second piezoelectric MEMS converter, and an application-specific integrated circuit (ASIC) mounted on the substrate base.

[0287] Clause 18. The ASIC is the device described in Clause 17, comprising an analog-to-digital converter (ADC), a digital signal processor (DSP), and a controller, wherein the output of the ADC is coupled to the input of the controller via the digital signal processor.

[0288] Clause 19. The device according to Clause 18, further comprising a second sensor package comprising a third MEMS transducer and a fourth MEMS transducer, wherein the first sensor package is positioned at a first location on the surface of an object, and the second sensor package is positioned at a second location on the surface of an object at a predetermined distance from the first location.

[0289] Clause 20. The device according to Clause 19, wherein the classification circuit is further configured to detect the location of an impact on the surface of an object based on a time delay or magnitude difference between vibrations detected in a first sensor package and vibrations detected in a second sensor package in the same time frame. Clause 1. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein one or more processors are configured to acquire audio signals based on the detection of sound by two or more microphones, acquire motion signals based on the detection of motion by two or more motion sensors mounted on the surface of a first object, perform one or more comparisons of the audio signals and motion signals to generate comparison data, determine the context of the contact type on the surface of the first object based on the comparison data, and determine the location of a second object within a threshold distance to the first object based on the context determined by one or more processors.

[0290] Clause 21. A device comprising a memory configured to store audio signals and motion signals, and one or more processors, wherein one or more processors are configured to acquire audio signals based on sound detection by two or more microphones, acquire motion signals based on motion detection by two or more motion sensors mounted on the surface of a first object, perform one or more comparisons of the audio signals and motion signals to generate comparison data, determine the contact type context of the surface of the first object based on the comparison data, and determine the location of a second object within a threshold distance to the first object based on the context determined by one or more processors.

[0291] Clause 22. The device according to Clause 21, wherein multiple cantilever beams are configured as a membrane surrounding the sensor area.

[0292] Clause 23. The first object is a car door, the device described in Clause 21 or 22.

[0293] Clause 24. The second object is a person, a key, or a balloon, and is the device described in any of Clauses 21 to 23.

[0294] Clause 25. The contact type is the device described in any of Clauses 21 to 24, including the area associated with the first object.

[0295] Clause 26. The first object is a door, and the area associated with the first object is one of the upper right part of the door, the lower right part of the door, the upper left part of the door, the lower left part of the door, or the center of the door, and is the device described in Clause 25.

[0296] Clause 27. A device comprising a memory configured to store an audio signal and a motion signal, and one or more processors, wherein the one or more processors obtain an audio signal based on detection of sound by a microphone, obtain a motion signal based on detection of motion by a motion sensor mounted on the surface of an object, normalize the audio signal and the motion signal to generate a normalized audio signal and a normalized motion signal, generate correlation data from the normalized audio signal and the normalized motion signal, and use the correlation data to determine a contact classification.

[0297] Clause 28. The device described in Clause 27, wherein one or more processors are configured as a classification circuit that uses correlation data to determine a contact classification.

[0298] Clause 29. The device described in Clause 28, wherein the classification circuit is further configured to receive an audio signal and a motion signal as training data in a training mode and match the training data with a provided training classification value.

[0299] Clause 30. The object is the vehicle surface, the surface is the outward-facing surface of the vehicle surface, and the provided training classification values ​​are a set of collision classification values ​​having known force and damage type characteristics, as described in Clause 29.

[0300] Clause 31. Memory, a first piezoelectric microelectromechanical system (MEMS) transducer having a first output, the first piezoelectric MEMS transducer being mechanically coupled to the surface of an object, and configured to generate a first analog signal at the first output when the first analog signal is converted from vibrations propagating through the object by the first piezoelectric MEMS transducer, and a second piezoelectric MEMS transducer having a second output, the second piezoelectric MEMS transducer being configured to generate a second analog signal when the second analog signal is converted from a second piezoelectric MEMS transducer. A device comprising: a second piezoelectric MEMS converter configured to generate a second output when converted from acoustic vibrations at the location of an object by an EMS converter; and a classification circuit coupled to a memory, the output of the first piezoelectric MEMS converter, and the output of the second piezoelectric MEMS converter, the classification circuit being configured to process data from a first analog signal and data from a second analog signal, and to categorize combinations of the first analog signal and the second analog signal received in one or more time frames.

[0301] Clause 32. The device described in Clause 31, further comprising storing categorization data from a classification circuit into memory.

[0302] Clause 33. The device according to Clause 31 or 32, wherein the first piezoelectric MEMS transducer has a noise floor that defines noise at a given frequency related to the signal output in gravitational units (g), and the noise floor is between 100 parts per million of gravitational units (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 μg / sqrt(Hz).

[0303] Clause 34. The first piezoelectric MEMS converter is the device described in Clause 33, having a conversion bandwidth for detecting vibrations propagating through an object at frequencies between 0.5 kilohertz (kHz) and 15 kHz.

[0304] Clause 35. A device as described in any of Clauses 31 to 34, wherein the data from the first analog signal includes frequency data of vibrations propagating through an object and magnitude data of vibrations propagating through an object, the magnitude data being associated with the severity of contact with the object.

[0305] Clause 36. One or more time frames are captured at a rate greater than 60 frames per second by any of the devices described in Clauses 31 to 35.

[0306] Clause 37. A device according to any one of Clauses 31 to 36, further comprising a first sensor package, the first sensor package comprising a substrate base and a lid, and a first piezoelectric MEMS converter, a second piezoelectric MEMS converter, and an application-specific integrated circuit (ASIC) mounted on the substrate base.

[0307] Clause 38. The ASIC is the device described in Clause 37, comprising an analog-to-digital converter (ADC), a digital signal processor (DSP), and a controller, wherein the output of the ADC is coupled to the input of the controller via the digital signal processor.

[0308] Clause 39. The device according to Clause 38, further comprising a second sensor package comprising a third MEMS transducer and a fourth MEMS transducer, wherein the first sensor package is positioned at a first location on the surface of an object, and the second sensor package is positioned at a second location on the surface of an object at a predetermined distance from the first location.

[0309] Clause 40. The device according to Clause 39, wherein the classification circuit is further configured to detect the location of an impact on the surface of an object based on the time delay or magnitude difference between vibrations detected in a first sensor package and vibrations detected in a second sensor package in the same time frame.

[0310] Clause 41. A micro-electromechanical (MEMS) converter comprising means for providing an output signal in any of the above-described manner.

[0311] Clause 42. A method for operating any MEMS converter described herein.

[0312] Clause 43. A storage medium containing instructions that, when executed by one or more processors of the system, cause the system to perform any of the operations described herein.

Claims

1. A memory configured to store audio signals and motion signals, A device comprising one or more processors, wherein the one or more processors are The audio signal is acquired based on the detection of sound by two or more microphones. The motion signal is acquired based on the detection of motion by two or more motion sensors mounted on the surface of the first object. To generate comparison data, one or more comparisons are performed between the audio signal and the motion signal. Based on the comparison data, the context of the contact type of the surface of the first object is determined. Based on the context determined by the one or more processors, the location of the second object within a threshold distance to the first object is determined. A device configured in such a way.

2. The device according to claim 1, wherein multiple cantilever beams are configured as a membrane surrounding the sensor area.

3. The device according to claim 1, wherein the first object is a car door.

4. The device according to claim 1, wherein the second object is a person, a key, or a balloon.

5. The device according to claim 1, wherein the contact type includes an area associated with the first object.

6. The first object is a door, and the area associated with the first object is The device according to claim 5, wherein the device is one of the following: the upper right portion of the door, the lower right portion of the door, the upper left portion of the door, the lower left portion of the door, or the center of the door.

7. It is a device, A memory configured to store audio signals and motion signals, A device comprising one or more processors, wherein the one or more processors are Based on sound detection by the microphone, the audio signal is acquired. The motion signal is acquired based on the detection of motion by a motion sensor mounted on the surface of the object. To generate normalized audio signals and normalized motion signals, the audio signals and motion signals are normalized. Correlation data is generated from the normalized audio signal and the normalized motion signal. A device configured to determine contact classification using the aforementioned correlation data.

8. The device according to claim 7, wherein the one or more processors are configured as a classification circuit that determines the contact classification using the correlation data.

9. The device according to claim 8, wherein the classification circuit is further configured to receive an audio signal and the motion signal as training data in training mode, and to match the training data with a provided training classification value.

10. The device according to claim 9, wherein the object is an automobile surface, the surface is an outward-facing surface of the automobile surface, and the provided training classification values ​​are a set of collision classification values ​​having known force and damage type characteristics.

11. It is a device, A memory configured to store converter data, A first piezoelectric micro-electromechanical system (MEMS) transducer having a first output, wherein the first piezoelectric MEMS transducer is mechanically coupled to the surface of an object, and the first piezoelectric MEMS transducer is configured to generate a first analog signal at the first output when the first analog signal is converted from vibrations propagating through the object by the first piezoelectric MEMS transducer, A second piezoelectric MEMS converter having a second output, wherein the second analog signal is generated at the second output when the second analog signal is converted by the second piezoelectric MEMS converter from acoustic vibrations at the location of the object, A classification circuit coupled to the memory, the output of the first piezoelectric MEMS converter, and the output of the second piezoelectric MEMS converter, wherein the classification circuit is configured to process data from the first analog signal and data from the second analog signal, and to categorize combinations of the first analog signal and the second analog signal received in one or more time frames, A device equipped with the following features.

12. The device according to claim 11, wherein the memory is further configured to store categorization data from the classification circuit in the memory.

13. The device according to claim 11, wherein the first piezoelectric MEMS converter has a noise floor that defines noise at a given frequency related to the signal output in gravitational units (g), and the noise floor is between 100 parts per million of gravitational units (μg) per square root of frequency in Hertz (μg / sqrt(Hz)) and 0.5 μg / sqrt(Hz).

14. The device according to claim 13, wherein the first piezoelectric MEMS converter has a conversion bandwidth for detecting vibrations propagating through the object at frequencies between 0.5 kilohertz (kHz) and 15 kHz.

15. The device according to claim 11, wherein the data from the first analog signal includes frequency data of the vibration propagating through the object and magnitude data of the vibration propagating through the object, the magnitude data being associated with the severity of contact with the object.

16. The device according to claim 11, wherein the one or more time frames are captured at a rate greater than 60 frames per second.

17. The device according to claim 11, further comprising a first sensor package, the first sensor package comprising a substrate base and a lid, the first piezoelectric MEMS converter, the second piezoelectric MEMS converter, and the application-specific integrated circuit (ASIC) mounted on the substrate base.

18. The ASIC comprises an analog-to-digital converter (ADC), a digital signal processor (DSP), and a controller, wherein the output of the ADC is coupled to the input of the controller via the digital signal processor, the device according to claim 17.

19. The device according to claim 18, further comprising a second sensor package comprising a third MEMS converter and a fourth MEMS converter, wherein the first sensor package is positioned at a first position on the surface of the object, and the second sensor package is positioned at a second position on the surface of the object at a predetermined distance from the first position.

20. The device according to claim 19, wherein the classification circuit is further configured to detect the location of an impact on the surface of the object based on the time delay or magnitude difference between the vibration detected by the first sensor package and the vibration detected by the second sensor package in the same time frame.

Images