A method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array

Abstract

The present invention discloses a method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array. The method includes: obtaining a focal point coordinate array and the array coordinates of each sensor; calculating the acoustic wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjusting the transmission delay value of each sensor according to the transmission time; the FPGA generates PWM drive signals corresponding to the phases of each sensor according to the delay values of each sensor; after boosting the PWM drive signals, ultrasonic array drive is performed to form a strong sound pressure at the focal point, so as to realize tactile feedback at the focal point. By performing energy focusing on the ultrasonic array through the FPGA, the present invention can continuously generate ultrasonic tactile focal points in space, produce tactile sensations of specific patterns, and realize an active aerial tactile method. The focal point vibrates at a frequency of 160 Hz macroscopically, the focused sound pressure energy is not less than 160 dB, and the human palm can clearly feel the tactile sensation, with obvious effects and wide application scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of tactile feedback using a MEMS ultrasonic sensor array, and particularly to a method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array. Background Art

[0002] With the development of human-computer interaction technology, more and more efficient and natural human-computer interaction methods have emerged. The diversification of interaction methods urgently requires tactile perception feedback in addition to vision and hearing to improve the user experience during the interaction process. The interaction method based on ultrasonic tactile feedback can provide a more natural and intuitive interaction different from traditional interaction methods for users, convey some information such as the texture and grain of objects that cannot be conveyed by other senses, inject new elements into the human-computer interaction process, and continuously generate ultrasonic tactile focus points in space by focusing the coordinates of the storage space of the FPGA, and can generate tactile sensations of specific patterns. It can be widely applied in non-contact human-computer interaction scenarios, such as VR, virtual buttons, etc., to provide a force feedback for manipulation, and has a large application scenario in future non-contact applications.

[0003] The invention application with the application number 202310001024.8 discloses a touch-sensing sound-collecting screen based on ultrasonic echo feedback. Each sound sensor receives the ultrasonic echo and sends it to the main control module. The main control module calculates the contact position based on the time difference principle, realizes the precise touch sensing of the sound-collecting screen, and has a simple structure, good sound-emitting effect, and high volume, which is conducive to the thin design of the product. However, this device still has problems such as not effectively using the FPGA controller, being difficult to obtain high sound pressure energy, and poor tactile feedback effect felt by the human palm. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and device for realizing tactile feedback based on a MEMS ultrasonic sensor array, which can obtain high sound pressure energy at the focus point and achieve a better tactile feedback effect.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A method for realizing tactile feedback based on a MEMS ultrasonic sensor array includes:

[0006] S1. Obtain an array of focus point coordinates and obtain the array coordinates of each sensor;

[0007] S2. Calculate the sound wave transmission time of each sensor according to the coordinate array and the array coordinates, and adjust the transmission delay value of each sensor according to the transmission time;

[0008] S3. The FPGA generates PWM drive signals corresponding to the phases of each sensor according to the delay values of each sensor;

[0009] S4. After boosting the PWM drive signal, perform ultrasonic array drive to form a strong sound pressure at the focal point and achieve tactile feedback at the focal point.

[0010] As a further aspect of the present invention, calculating the sound wave transmission time of each sensor according to the coordinate array and the array coordinates in S2 includes the steps of:

[0011] S21. Obtain the coordinate array of the focal point, which is expressed by the formula:

[0012] f[n] = (Xfn, Yfn, Zfn)

[0013] S22. Obtain the array coordinates of each sensor, which is expressed by the formula:

[0014] P[n] = (Xpm, Ypm)

[0015] S23. Calculate the distance from each sensor to the focal point, which is expressed by the formula:

[0016]

[0017] S24. Convert the distance from each sensor to the focal point into the sound wave transmission time, which is expressed by the formula:

[0018] t[n] = d[n] / V 空气

[0019] where V 空气 is the speed of sound in air.

[0020] As a further aspect of the present invention, adjusting the transmission delay value of each sensor according to the transmission time in S2 includes the steps of:

[0021] S25. Obtain the modulation wave frequency Fc, the drive signal frequency fc, and the FPGA system clock frequency Mc;

[0022] S26. Obtain the drive signal cycle time tc and the modulation wave cycle time Tc, which are expressed by the formulas:

[0023] tc = Mc / (2*fc)

[0024] Tc = Mc / (2*Fc)

[0025] S27. Calculate the transmission delay value of each sensor according to the sound wave transmission time t[n], which is expressed by the formula:

[0026] S[n] = M(t)*s(tc + t[n])

[0027]

[0028] Among them, Tc is the modulation wave period time, and tc is the driving signal period time.

[0029] As a further solution of the present invention, in S25, the driving signal frequency fc is not less than 160 Hz.

[0030] As a further solution of the present invention, in S3, the FPGA uses a unified crystal oscillator to generate the system clock frequency Mc.

[0031] As a further solution of the present invention, in S4, a strong sound pressure is formed at the focus point, and the sound pressure energy is not less than 160 dB.

[0032] As a further solution of the present invention, when implementing the tactile feedback at the focus point in S4, the time-division multiplexing method or the array block method is adopted.

[0033] The device operating based on the above method includes:

[0034] A PC host computer, which is used to obtain the transmission delay values of each sensor and control the driving signal frequency;

[0035] An FPGA module, which obtains the PWM driving signals of each sensor according to the sound delay values and the driving signal frequency of each sensor;

[0036] An ultrasonic array, which drives each sensor according to the PWM driving signal, forms a strong sound pressure at the focus point, and realizes the tactile feedback at the focus point.

[0037] As a further solution of the present invention, it further includes:

[0038] A boost module, which boosts each PWM driving signal and sends it to the sensor end to drive the sensor.

[0039] As a further solution of the present invention, the sensor uses a 2.25 * 2.25 mm MEMS ultrasonic sensor in the 80 kHz frequency band.

[0040] The beneficial effects of the present invention:

[0041] 1. The present invention adopts a MEMS ultrasonic sensor array in the 80 kHz frequency band with a size of 2.25 * 2.25 mm. Through the FPGA, energy focusing is performed on the ultrasonic array, and ultrasonic tactile focus points can be continuously generated in space, generating specific graphics of touch, realizing an active air tactile method. The focus point vibrates at a frequency of 160 Hz macroscopically, and the focused sound pressure energy is not less than 160 dB. The human palm can clearly feel the touch, with obvious effects and wide application scenarios.

[0042] 2. The FPGA of the present invention uses a unified crystal oscillator, and the working frequency of the FPGA is consistent, effectively reducing the frequency deviation of each sensor around the focal point, improving the energy convergence on the focal point, and having a good application effect.

[0043] 3. The present invention can achieve multi-point synchronous tactile feedback by using the time-division multiplexing method or the array block method, with obvious effects and a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the principle of the method for realizing tactile feedback based on the MEMS ultrasonic sensor array of the present invention;

[0045] Figure 2 It is a schematic flow diagram of the method for realizing tactile feedback based on the MEMS ultrasonic sensor array of the present invention;

[0046] Figure 3 It is a schematic structural diagram of the device for realizing tactile feedback based on the MEMS ultrasonic sensor array of the present invention;

[0047] Figure 4 It is a schematic diagram of the energy convergence of the ultrasonic array of the present invention;

[0048] Figure 5 It is a schematic structural diagram of the multi-point tactile feedback principle of the present invention;

[0049] Figure 6 It is a schematic diagram of the same-frequency structure of multiple FPGAs of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar symbols represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0051] Existing tactile feedback devices are not easy to obtain high sound pressure energy at the convergence point.

[0052] In view of the above problems, the present invention discloses a method for realizing tactile feedback based on a MEMS ultrasonic sensor array, as shown in Figure 1 and 2 , including:

[0053] S1. Obtain the focal point coordinate array and the array coordinates of each sensor.

[0054] Tactile feedback requires a large acoustic radiation pressure to be formed at a point, and a sound field with energy aggregation or a focal point at a point needs to be constructed, which can be realized by using the principle of ultrasonic phased focusing. Figure 4The following is a schematic diagram of ultrasonic phased focusing deflection and focusing. By controlling the emission phases of each ultrasonic sensor, the propagation direction of the sound beam formed by the ultrasonic array and the energy focus point can be controlled.

[0055] The distances from each sensor to the focus point are different. By reasonably controlling the time difference, all sound waves can reach the focus point simultaneously, forming in-phase superposition, generating a point of ultrasonic energy focus. The sensors of the present invention use MEMS ultrasonic sensors with a frequency band of 80 kHz and a size of 2.25 * 2.25 mm to form a sensor array to achieve a strong sound pressure at the focus point. The sound pressure energy is not less than 160 dB, and the human palm can clearly feel the tactile sensation, with obvious effects.

[0056] According to the setting of the ultrasonic array, taking the ultrasonic array plane as the XY plane, a three-dimensional coordinate system is established, and the array coordinates of each sensor are obtained. The formula is expressed as:

[0057] P[n] = (Xpm, Ypm)

[0058] According to the three-dimensional coordinate system of the ultrasonic array, the PC host computer acquires the focus point coordinate array through an external device. The formula is expressed as:

[0059] f[n] = (Xfn, Yfn, Zfn)

[0060] In order to make the ultrasonic waves emitted by each sensor in the ultrasonic array reach the focus point in the same phase, according to the spatial relationship between the focus point and each sensor, the distances between each sensor in the array and the focus point are obtained. The formula is expressed as:

[0061]

[0062] The distances from each sensor to the focus point are converted into the sound wave transmission time. The formula is expressed as:

[0063] t[n] = d[n] / V 空气

[0064] In the formula, V 空气 is the propagation speed of sound waves in the air. Under the room temperature condition of 25 °C, the speed of sound in the air is 345 m / s. (Xfn, Yfn, Zfn) are the coordinates of the focus point in space, and (Xpm, Ypm) are the plane coordinates of each sensor, where Zpm = 0 for each sensor. According to the distances from each sensor to the focus point, the corresponding sound wave transmission times are further calculated.

[0065] S2. Calculate the sound wave transmission times of each sensor based on the coordinate array and the array coordinates, and adjust the transmission delay values of each sensor according to the transmission times.

[0066] Such as Figure 2As shown, adjust the transmission delay values of each sensor according to the transmission time. First, obtain the modulation wave frequency Fc, the driving signal frequency fc, and the FPGA system clock frequency Mc.

[0067] Then, according to the driving signal frequency fc, obtain the driving signal cycle time tc. Obtain the modulation wave cycle time Tc according to the modulation wave frequency Fc. The formula is expressed as:

[0068] tc = Mc / (2 * fc)

[0069] Tc = Mc / (2 * Fc)

[0070] Then, according to the acoustic wave transmission time t[n], calculate the transmission delay values of each sensor. The formula is expressed as:

[0071] S[n] = M(t) * s(tc + t[n])

[0072]

[0073] Among them, Tc is the modulation wave cycle time, and tc is the driving signal cycle time. In the present invention, the driving signal frequency fc is not less than 160 Hz, so that the focal point vibrates at a frequency of 160 Hz macroscopically, and the human palm can feel the touch, achieving a better effect.

[0074] S3. The FPGA generates a PWM driving signal corresponding to the phase of the sensor according to the delay values of each sensor;

[0075] The FPGA controller is responsible for controlling the phases of each sensor and outputting a PWM driving signal corresponding to the resonance frequency. The boost module further amplifies the PWM driving signal output by the FPGA, and finally drives the sensor to work.

[0076] For a point in space, the acoustic waves of each sensor in the ultrasonic array are mixed and superimposed. Even if the frequencies of two waves only differ by a very small value, as time goes by, this error will accumulate continuously, causing the originally in-phase signals to gradually become out-of-phase and then slowly return to in-phase and cycle continuously. Especially when the array scale is relatively large, it is the superposition of hundreds of sensor signals. If there are small errors between signals, it will be impossible to maintain a stable sound field. Therefore, it is necessary to ensure that the working frequencies of all sensors in the array are the same.

[0077] When the FPGA controller controls the phases of each sensor, a unified crystal oscillator is adopted. However, as the scale of the ultrasonic array expands, the I / O of only one FPGA is often insufficient, and multiple FPGAs may be involved. When multiple FPGAs are used for control, the biggest problem is that the clock frequencies of each FPGA come from their respective crystal oscillators, and there are more or less frequency deviations between these crystal oscillators. Moreover, this error often accumulates over time, resulting in phase drift.

[0078] In view of the above situation, as Figure 6 shown, in the present invention, one of the multiple FPGAs is regarded as the master FPGA, and the rest are regarded as slave FPGAs. The slave FPGAs all use the crystal oscillator of the master FPGA to form a unified clock signal, so that the operating frequencies of all FPGAs reach consistency, and then the output signal frequencies of each are also consistent.

[0079] S4. After boosting the PWM drive signal, perform ultrasonic array drive to form a strong sound pressure at the focal point and achieve tactile feedback at the focal point.

[0080] Furthermore, in order to achieve multi-point synchronous tactile feedback, time-division multiplexing or array partitioning methods can be adopted.

[0081] Among them, the time-division multiplexing method is as Figure 5 shown in A. In the figure, P1 and P2 represent two focal points where tactile feedback is desired to be obtained synchronously. First, according to the above method of the present invention, solve the transmission delay values of each sensor for single-point focusing at each focal point, and then control the ultrasonic array to perform high-frequency switching between two groups of corresponding phase controls in turn. As long as the switching frequency is high enough, it can deceive the human brain and make people feel that the tactile feedback of the two points exists simultaneously.

[0082] The array partitioning method can also be adopted, as Figure 5 shown in B. By dividing the sensor spatial area corresponding to the focal point and processing the ultrasonic array in blocks, and each small block is only responsible for the tactile feedback of one point above it. The advantage of this method is that there is no need to switch the phase, and each focal point can be independently controlled.

[0083] The present invention also discloses a tactile feedback device for an ultrasonic array based on FPGA control, as Figure 3 shown, including a PC host computer, a boosting module, an ultrasonic array, an FPGA module, etc.

[0084] Among them, the ultrasonic array consists of a MEMS ultrasonic sensor array with a frequency band of 80 kHz and a size of 2.25 * 2.25 mm. The PC host computer is connected to external devices and can collect the spatial coordinates of the focal point. Then, by combining the array coordinates of the ultrasonic array, the transmission delay values of each sensor can be obtained. The PC host computer can also modulate and control the frequency of the driving signal. The boost module boosts each PWM driving signal and sends it to the sensor end to drive the sensors. The ultrasonic array drives each sensor according to the PWM driving signal to form a strong sound pressure at the focal point and realize tactile feedback at the focal point. The FPGA module obtains the PWM driving signals of each sensor based on the sound delay values and driving signal frequencies of each sensor.

[0085] As mentioned above, the above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes equivalent replacements or changes, and all should be covered within the protection scope of the present invention.

[0086] In the present invention, unless otherwise clearly specified and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "below" and "beneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

Claims

1. A method for realizing tactile feedback based on a MEMS ultrasonic sensor array, characterized in that, Including: S1. Obtain the focal point coordinate array and the array coordinates of each sensor; S2. Calculate the acoustic wave transmission time of each sensor based on the coordinate array and the array coordinates, and adjust the transmission delay value of each sensor according to the transmission time; S3. The FPGA generates the PWM drive signal of the corresponding phase of each sensor according to the delay value of each sensor; S4. After boosting the PWM drive signal, perform ultrasonic array drive to form a strong sound pressure at the focal point and realize tactile feedback at the focal point; Among them, the sensor uses a 2.25*2.25mm MEMS ultrasonic sensor in the 80kHz frequency band; In the S2, calculating the acoustic wave transmission time of each sensor based on the coordinate array and the array coordinates includes the steps: S21. Obtain the focal point coordinate array, which is expressed by the formula: f[n]=(Xfn,Yfn,Zfn) Where Xfn, Yfn, and Zfn are the coordinates of the focal point in space; S22. Obtain the array coordinates of each sensor, which is expressed by the formula: P[n]=(Xpm,Ypm) Where Xpm and Ypm are the plane coordinates of each sensor; S23. Calculate the distance from each sensor to the focal point, which is expressed by the formula: S24. Convert the distance from each sensor to the focal point into the acoustic wave transmission time, which is expressed by the formula: t[n] = d[n] / V 空气 where V 空气 is the speed of sound waves in air; In the S2, adjusting the transmission delay value of each sensor according to the transmission time includes the steps: S25. Obtain the modulation wave frequency Fc, the drive signal frequency fc, and the FPGA system clock frequency Mc; S26. Obtain the drive signal period time tc and the modulation wave period time Tc, which are expressed by the formula: tc=Mc / (2*fc) Tc=Mc / (2*Fc) S27. Calculate the transmission delay value of each sensor according to the acoustic wave transmission time t[n], which is expressed by the formula: S[n]=M(t)*s(tc+t[n]) Where Tc is the modulation wave period time and tc is the drive signal period time.

2. The method according to claim 1, characterized in that In the S25, the drive signal frequency fc is not less than 160Hz.

3. The method according to claim 1, wherein In the S3, the FPGA uses a unified crystal oscillator to generate the system clock frequency Mc.

4. The method according to claim 1, wherein In the S4, when forming a strong sound pressure at the focal point, the sound pressure energy is not less than 160dB.

5. The method according to claim 1, wherein When realizing tactile feedback at the focal point in the S4, the time-division multiplexing method or the array block method is adopted.

6. An apparatus operating according to the method of any one of claims 1-5, characterized in that, Including: A PC host computer, which is used to obtain the transmission delay value of each sensor and control the drive signal frequency; An FPGA module, which obtains the PWM drive signal of each sensor according to the acoustic delay value of each sensor and the drive signal frequency; An ultrasonic array, which drives each sensor according to the PWM drive signal to form a strong sound pressure at the focal point and realize tactile feedback at the focal point.

7. The device according to claim 6, characterized in that, Also including: A boosting module, which boosts each PWM drive signal and sends it to the sensor end to drive the sensor.

Images