Tactile feedback implementation method and device based on MEMS ultrasonic sensor array

Abstract

The invention discloses a tactile feedback implementation method and device based on an MEMS ultrasonic sensor array, and the method comprises the steps: obtaining a focus point coordinate array, and obtaining the array coordinates of all sensors; calculating the sound wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjusting the transmission delay value of each sensor according to the transmission time; the FPGA generates PWM driving signals of corresponding phases of the sensors according to the delay values of the sensors; after the PWM driving signal is boosted, ultrasonic array driving is carried out, focus point strong sound pressure is formed, and focus point tactile feedback is achieved. Energy focusing is carried out on the ultrasonic array through the FPGA, ultrasonic tactile focus points can be continuously generated in the space, tactile sense of a specific graph is generated, an active air tactile sense mode is achieved, the focus points vibrate at the frequency of 160 Hz macroscopically, the focusing sound pressure energy is not smaller than 160 dB, the human palm can obviously feel the tactile sense, the effect is obvious, and the application scene is wide.

Description

Technical Field

[0001] The present invention relates to the technical field of tactile feedback using a MEMS ultrasonic sensor array, and in particular to a method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array. Background Art

[0002] With the development of human-computer interaction technology, more and more efficient and natural human-computer interaction methods are emerging. The diversification of interaction methods urgently requires tactile perception feedback other than vision and hearing to improve the user experience of the interaction process. The interaction method based on ultrasonic tactile feedback can provide users with a more natural and intuitive interaction that is different from the traditional interaction method, conveying some information that other senses cannot convey, such as the texture and texture of objects, injecting new elements into the human-computer interaction process. By storing the coordinates of the spatial focal point through FPGA, ultrasonic tactile focal points can be continuously generated in space, which can produce tactile sensations of specific graphics. It can be widely used in non-contact human-computer interaction scenarios, such as VR, virtual buttons, etc., to provide a force feedback for control, and has a large application scenario in future non-contact applications.

[0003] The invention application with application number 202310001024.8 discloses a touch-sensing sound-gathering screen based on ultrasonic echo feedback. Each sound sensor receives the ultrasonic echo and sends it to the main control module. The main control module calculates the contact point position based on the time difference principle, realizing the precise touch sensing of the sound-gathering screen. It has a simple structure, good pronunciation effect, and loud volume, which is conducive to the thin design of the product. However, the device still has the problem of not effectively utilizing the FPGA controller, not being easy to obtain higher sound pressure energy, and poor tactile feedback effect felt by the palm of the hand. Summary of the invention

[0004] The object of the present invention is to provide a method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array, which can obtain higher sound pressure energy at a focal point and achieve better tactile feedback effect.

[0005] The object of the present invention can be achieved by the following technical solution: A method for realizing tactile feedback based on a MEMS ultrasonic sensor array, comprising:

[0006] S1. Obtain the focus point coordinate array and the array coordinates of each sensor;

[0007] S2, calculating the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjusting the transmission delay value of each sensor according to the transmission time;

[0008] S3, FPGA generates a PWM drive signal corresponding to the phase of each sensor according to the delay value of each sensor;

[0009] S4, after boosting the PWM driving signal, the ultrasonic array is driven to form a strong sound pressure at the focus point, thereby achieving tactile feedback at the focus point.

[0010] As a further solution of the present invention, the calculation of the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates in S2 includes the steps of:

[0011] S21. Get the focus point coordinate array, the formula is:

[0012] f[n]=(Xfn,Yfn,Zfn)

[0013] S22, obtaining the array coordinates of each sensor, the formula is expressed as:

[0014] P[n]=(Xpm,Ypm)

[0015] S23, calculate the distance from each sensor to the focus point, the formula is expressed as:

[0016]

[0017] S24, convert the distance from each sensor to the focal point into the sound wave transmission time, the formula is expressed as:

[0018] t[n]=d[n] / V 空气

[0019] Among them, V 空气 is the speed of sound waves in air.

[0020] As a further solution of the present invention, the step of adjusting the transmission delay value of each sensor according to the transmission time in S2 includes the following steps:

[0021] S25, obtaining the modulation wave frequency Fc, the driving signal frequency fc and the FPGA system clock frequency Mc;

[0022] S26, obtaining the driving signal cycle time tc and the modulation wave cycle time Tc, the formula is expressed as:

[0023] tc=Mc / (2*fc)

[0024] Tc=Mc / (2*Fc)

[0025] S27. Calculate the transmission delay value of each sensor according to the sound wave transmission time t[n], and the formula is expressed as:

[0026] S[n]=M(t)*s(tc+t[n])

[0027]

[0028] Among them, Tc is the modulation wave cycle time, and tc is the drive signal cycle time.

[0029] As a further solution of the present invention, the driving signal frequency fc in S25 is not less than 160 Hz.

[0030] As a further solution of the present invention, the FPGA in S3 uses a unified crystal oscillator to generate a system clock frequency Mc.

[0031] As a further solution of the present invention, a strong sound pressure at a focal point is formed in S4, and the sound pressure energy is not less than 160dB.

[0032] As a further solution of the present invention, when implementing the focus point tactile feedback in S4, a time division multiplexing method or an array blocking method is adopted.

[0033] The device running based on the above method includes:

[0034] The PC host computer is used to obtain the transmission delay value of each sensor and control the drive signal frequency;

[0035] The FPGA module obtains the PWM drive signal of each sensor according to the acoustic delay value and the drive signal frequency of each sensor;

[0036] The ultrasonic array drives each sensor according to the PWM driving signal to form a strong sound pressure at the focus point and realize tactile feedback at the focus point.

[0037] As a further solution of the present invention, it also includes:

[0038] The boost module boosts each PWM drive signal and sends it to the sensor end to drive the sensor.

[0039] As a further solution of the present invention, the sensor adopts a 2.25*2.25mm 80kHz frequency band MEMS ultrasonic sensor.

[0040] Beneficial effects of the present invention:

[0041] 1. The present invention adopts a 2.25*2.25mm MEMS ultrasonic sensor array in the 80kHz frequency band, and focuses the energy of the ultrasonic array through FPGA, which can continuously generate ultrasonic tactile focal points in space, produce tactile sensations of specific patterns, and realize an active aerial tactile mode. The focal point vibrates at a frequency of 160Hz on a macro scale, and the focused sound pressure energy is not less than 160dB. The human palm can clearly feel the tactile sensation, with obvious effects and wide application scenarios.

[0042] 2. The FPGA of the present invention adopts a unified crystal oscillator, and the FPGA operating frequency is consistent, which effectively reduces the frequency deviation of each sensor around the focusing point, improves the energy convergence on the focusing point, and has a better application effect.

[0043] 3. The present invention can realize multi-point synchronous tactile feedback by adopting time division multiplexing method or array blocking method, which has obvious effect and wide application range. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of a method for realizing tactile feedback based on a MEMS ultrasonic sensor array of the present invention;

[0045] Figure 2 It is a flow chart of a method for realizing tactile feedback based on a MEMS ultrasonic sensor array of the present invention;

[0046] Figure 3 It is a structural schematic diagram of a tactile feedback implementation device based on a MEMS ultrasonic sensor array of the present invention;

[0047] Figure 4 This is a schematic diagram of energy convergence of the ultrasound array of the present invention;

[0048] Figure 5 This is a schematic diagram of the principle structure of multi-point tactile feedback of the present invention;

[0049] Figure 6 This is a schematic diagram of the multi-FPGA same-frequency structure of the present invention. DETAILED DESCRIPTION

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0051] It is difficult for existing tactile feedback devices to obtain high sound pressure energy at the convergence point.

[0052] In view of the above problems, the present invention discloses a method for realizing tactile feedback based on a MEMS ultrasonic sensor array, such as Figure 1 and 2 As shown, including:

[0053] S1. Obtain the focus point coordinate array and the array coordinates of each sensor.

[0054] Tactile feedback requires the formation of a large sound wave radiation pressure at one point, and it is necessary to construct a sound field where energy is concentrated or called a focal point. This can be achieved by using the principle of ultrasonic phased focusing. Figure 4The figure shows the schematic diagram of ultrasonic phase-controlled focusing deflection and focusing. By controlling the emission phase of each ultrasonic sensor, the propagation direction of the sound beam formed by the ultrasonic array and the energy focus point can be controlled.

[0055] The distances from each sensor to the focal point are different, and the time difference is reasonably controlled so that all sound waves reach the focal point at the same time, forming a phase superposition, thereby generating a point where ultrasonic energy is focused. The sensor of the present invention adopts a 2.25*2.25mm MEMS ultrasonic sensor in the 80kHz frequency band to form a sensor array to achieve strong sound pressure at the focal point. The sound pressure energy is not less than 160dB, and the human palm can clearly feel the touch, with obvious effect.

[0056] According to the setting of the ultrasonic array, the ultrasonic array plane is taken as the XY plane, a three-dimensional coordinate system is established, and the array coordinates of each sensor are obtained. The formula is expressed as:

[0057] P[n]=(Xpm,Ypm)

[0058] Based on the three-dimensional coordinate system of the ultrasound array, the PC host computer acquires the focus point coordinate array through external equipment, and the formula is expressed as:

[0059] f[n]=(Xfn,Yfn,Zfn)

[0060] In order to make the ultrasonic waves emitted by each sensor in the ultrasonic array reach the focal point in phase, the distance between each sensor in the array and the focal point is calculated based on the spatial relationship between the focal point and each sensor. The formula is expressed as:

[0061]

[0062] The distance from each sensor to the focal point is converted into the sound wave transmission time, and the formula is expressed as:

[0063] t[n]=d[n] / V 空气

[0064] Where V 空气 is the propagation speed of sound waves in the air. At room temperature of 25°C, the speed of sound in the air is 345 m / s. (Xfn, Yfn, Zfn) are the coordinates of the focal point in space. (Xpm, Ypm) are the plane coordinates of each sensor, where Zpm=0 for each sensor. The corresponding sound wave transmission time is further calculated based on the distance from each sensor to the focal point.

[0065] S2. Calculate the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjust the transmission delay value of each sensor according to the transmission time.

[0066] like Figure 2As shown, the transmission delay value of each sensor is adjusted according to the transmission time. First, the modulation wave frequency Fc, the driving signal frequency fc and the FPGA system clock frequency Mc are obtained.

[0067] Then, the driving signal cycle time tc is obtained according to the driving signal frequency fc. The modulation wave cycle time Tc is obtained according to the modulation wave frequency Fc, and the formula is expressed as:

[0068] tc=Mc / (2*fc)

[0069] Tc=Mc / (2*Fc)

[0070] Then, according to the sound wave transmission time t[n], the transmission delay value of each sensor is calculated, and the formula is expressed as:

[0071] S[n]=M(t)*s(tc+t[n])

[0072]

[0073] Wherein, Tc is the modulation wave cycle time, tc is the drive signal cycle time. The drive signal frequency fc of the present invention is not less than 160Hz, so that the focal point vibrates at a frequency of 160Hz in the macroscopic sense, and the palm of a person can feel the touch, achieving a better effect.

[0074] S3, FPGA generates a PWM drive signal corresponding to the phase of the sensor according to the delay value of each sensor;

[0075] The FPGA controller is responsible for controlling the phase of each sensor and outputting a PWM drive signal of the corresponding resonant frequency. The boost module further amplifies the PWM drive signal output by the FPGA and ultimately drives the sensor to work.

[0076] The sound waves of each sensor of the ultrasonic array are mixed and superimposed. For a point in space, even if the frequencies of the two waves differ by only a very small value, this error will continue to accumulate over time, causing the originally in-phase signal to gradually change to anti-phase and then slowly change back to the same phase and continue to cycle. Especially when the array is large, it is the superposition of hundreds of sensor signals. If there are tiny errors between signals, there will be no way to maintain a stable sound field. Therefore, it is necessary to ensure that the operating frequencies of all sensors in the array are consistent.

[0077] The FPGA controller uses a unified crystal oscillator to control the phase of each sensor. However, as the scale of the ultrasound array expands, the IO of only one FPGA is often not enough, and multiple FPGA applications may be involved. The biggest problem with using multiple FPGAs for control is that the clock frequency of each FPGA comes from its own crystal oscillator. There are more or less frequency deviations between these crystal oscillators, and this error tends to accumulate over time, resulting in phase drift.

[0078] In view of the above situation, if Figure 6 As shown, the present invention regards one of the multiple FPGAs as the master FPGA and the rest as slave FPGAs. The slave FPGAs all use the crystal oscillator of the master FPGA to form a unified clock signal, so that the operating frequencies of all FPGAs are consistent, and then the output signal frequencies of each are also consistent.

[0079] S4, after boosting the PWM driving signal, the ultrasonic array is driven to form a strong sound pressure at the focus point, thereby achieving tactile feedback at the focus point.

[0080] Furthermore, in order to realize multi-point synchronous tactile feedback, a time division multiplexing method or an array blocking method may be adopted.

[0081] Among them, time division multiplexing method is as follows Figure 5 As shown in A, P1 and P2 in the figure represent two focal points where tactile feedback is desired to be obtained synchronously. First, according to the above method of the present invention, the transmission delay value of each sensor for realizing single-point focusing at each focal point is solved, and then the ultrasonic array is controlled to perform high-frequency switching between two groups of corresponding phase controls in turn. As long as the switching frequency is high enough, it can deceive the human brain and make people feel that the tactile feedback of the two points exists at the same time.

[0082] You can also use array partitioning, such as Figure 5 As shown in B, the ultrasonic array is processed in blocks by dividing the sensor space area corresponding to the focal point, and each small block is only responsible for the tactile feedback of a point above it. The advantage of this method is that there is no need to switch the phase, and each focal point can be controlled independently.

[0083] The present invention also discloses a tactile feedback device based on an ultrasonic array controlled by an FPGA, such as Figure 3 As shown, it includes a PC host computer, a boost module, an ultrasound array, and an FPGA module.

[0084] The ultrasonic array is composed of a 2.25*2.25mm 80kHz frequency band MEMS ultrasonic sensor array. The PC host computer can collect the spatial coordinates of the focal point by connecting to external devices, and then obtain the transmission delay value of each sensor in combination with the array coordinates of the ultrasonic array. The PC host computer can also modulate and control the frequency of the drive signal; the boost module boosts each PWM drive signal and sends it to the sensor end to drive the sensor. The ultrasonic array drives each sensor according to the PWM drive signal to form a strong sound pressure at the focal point and realize tactile feedback at the focal point; the FPGA module obtains the PWM drive signal of each sensor according to the sound delay value and drive signal frequency of each sensor.

[0085] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0086] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

Claims

1. A method for realizing tactile feedback based on a MEMS ultrasonic sensor array, characterized in that: include: S1. Obtain the focus point coordinate array and the array coordinates of each sensor; S2, calculating the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjusting the transmission delay value of each sensor according to the transmission time; S3, FPGA generates a PWM drive signal corresponding to the phase of each sensor according to the delay value of each sensor; S4, after boosting the PWM driving signal, the ultrasonic array is driven to form a strong sound pressure at the focus point, thereby realizing tactile feedback at the focus point.

2. The method according to claim 1, characterized in that The step S2 calculates the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates, including the following steps: S21. Get the focus point coordinate array, the formula is: f[n]=(Xfn,Yfn,Zfn) S22, obtaining the array coordinates of each sensor, the formula is expressed as: P[n]=(Xpm,Ypm) S23, calculate the distance from each sensor to the focus point, the formula is expressed as: S24, convert the distance from each sensor to the focal point into the sound wave transmission time, the formula is expressed as: t[n]=d[n] / V 空气 Among them, V 空气 is the speed of sound waves in air.

3. The method according to claim 1, characterized in that The step S2 adjusts the transmission delay value of each sensor according to the transmission time, including the steps of: S25, obtaining the modulation wave frequency Fc, the driving signal frequency fc and the FPGA system clock frequency Mc; S26, obtaining the driving signal cycle time tc and the modulation wave cycle time Tc, the formula is expressed as: tc=Mc / (2*fc) Tc=Mc / (2*Fc) S27. Calculate the transmission delay value of each sensor according to the sound wave transmission time t[n], and the formula is expressed as: S[n]=M(t)*s(tc+t[n]) Among them, Tc is the modulation wave cycle time, and tc is the drive signal cycle time.

4. The method according to claim 3, characterized in that: The driving signal frequency fc in S25 is not less than 160 Hz.

5. The method according to claim 1, characterized in that The FPGA in S3 uses a unified crystal oscillator to generate a system clock frequency Mc.

6. The method according to claim 1, characterized in that A strong sound pressure at the focal point is formed in S4, and the sound pressure energy is not less than 160dB.

7. The method according to claim 1, characterized in that When implementing the focus point tactile feedback in S4, a time division multiplexing method or an array blocking method is adopted.

8. A device operating according to any one of claims 1 to 7, characterized in that: include: The PC host computer is used to obtain the transmission delay value of each sensor and control the drive signal frequency; The FPGA module obtains the PWM drive signal of each sensor according to the acoustic delay value and the drive signal frequency of each sensor; The ultrasonic array drives each sensor according to the PWM driving signal to form a strong sound pressure at the focus point and realize tactile feedback at the focus point.

9. The device according to claim 8, characterized in that Also includes: The boost module boosts each PWM drive signal and sends it to the sensor end to drive the sensor.

10. The device according to claim 8, characterized in that The sensor adopts a 2.25*2.25mm 80kHz frequency band MEMS ultrasonic sensor.

Images