Non-touch ultrasonic gesture control module with force feedback as well as use method and application of non-touch ultrasonic gesture control module

Abstract

The invention discloses a touch-free ultrasonic gesture control module with force feedback and a use method and application thereof, and belongs to the technical field of intelligent wearable equipment. The module has a volley tactile feedback function and mainly comprises a piezoelectric micromechanical ultrasonic transducer array chip and an ASIC chip used for generating control signals, and the piezoelectric micromechanical ultrasonic transducer array chip can generate ultrasonic waves and position finger positions and recognize finger control actions in a mode of detecting reflected sound waves. And meanwhile, ultrasonic waves can be focused and acted on fingertips, so that a human hand can feel force feedback like touching an entity key in a non-touch control process, and the experience feeling of non-touch control is enhanced. The module can be integrated into various intelligent wearable devices such as intelligent AR / VR glasses, intelligent bracelets, intelligent rings, intelligent earphones and the like, richer operation options and a more flexible man-machine interaction mode are provided, and therefore the user experience is remarkably improved.

Description

Technical Field

[0001] The present application relates to a touchless ultrasonic gesture control module with force feedback and a method for using and application thereof, and belongs to the technical field of smart wearable devices. Background Art

[0002] Smart wearable devices (such as AR / VR glasses, smart bracelets, smart rings, smart headphones, etc.) have had a huge impact in the fields of health testing and medical care, AR / VR technology, smart clothing, etc. These devices not only enhance people's daily technological experience, provide new human-computer interaction methods, but also facilitate people's daily learning and work. With the continuous maturity and improvement of technology, smart wearable devices will continue to expand their application scope and bring more convenience. Nowadays, there are many human-computer interaction technologies, and common methods include touch screen operation, voice control, knob adjustment, etc. The hardware relied on by such interactive control methods usually occupies a large physical space in smart wearable devices, affecting the miniaturization and portability of the device. For example, in smart wearable devices such as smart glasses and smart watches, rotary buttons are a more common design, which can control page sliding by rotation, or realize diversified functions by different number of key presses. However, the operation methods of rotary buttons mainly include pressing and rotating, and the human-computer interaction experience needs to be improved. There are certain limitations in appearance, which occupies additional space, which is not conducive to the aesthetic design and functional expansion of the device.

[0003] In addition, with the advancement of technology, human-computer interaction technology can also use gesture sensing, facial expression recognition and other technologies to achieve more natural interaction. Existing gesture sensing technology often separates the functions of gesture detection and tactile feedback. Gesture detection relies on optical or other sensors, while tactile feedback is generated by ultrasonic arrays. This separation design increases the complexity and power consumption of the system, which is not conducive to its application in wearable devices with limited size and power consumption. Summary of the invention

[0004] In order to solve the problem that gesture detection and tactile feedback in the existing human-computer interaction technology are difficult to integrate efficiently and with low power consumption, the present application proposes a technical solution of a touchless ultrasonic gesture control module with force feedback based on a piezoelectric micromechanical ultrasonic transducer (pMUT) array. The solution has a mid-air tactile feedback function, which can generate force feedback through ultrasonic focusing while providing touchless control of the device, thereby achieving a combination of efficient tactile feedback and low power consumption, as well as a more natural and rich interaction method.

[0005] This application adopts the following technical solutions:

[0006] According to a first aspect of the present application, there is provided a touchless ultrasonic gesture control module with force feedback, comprising: a pMUT array composed of a plurality of pMUT units;

[0007] The pMUT unit includes a piezoelectric layer formed of a piezoelectric material with a high piezoelectric coefficient;

[0008] The piezoelectric material with a high piezoelectric coefficient is selected from at least one of lithium niobate, lithium tantalate, PZT, and PMN-PT.

[0009] In one embodiment, the pMUT unit further includes an electrode layer, a structural layer and a substrate;

[0010] The piezoelectric layer has an upper side and a lower side arranged opposite to each other;

[0011] The structural layer and the substrate are sequentially arranged on the lower side of the piezoelectric layer;

[0012] A cavity is formed on the substrate, and the cavity extends to a side of the structural layer away from the piezoelectric layer;

[0013] In one embodiment, the pMUT unit satisfies at least one of the following conditions:

[0014] (1) The electrode layer is an upper and lower electrode structure, including an upper electrode layer and a lower electrode layer, wherein the upper electrode layer is arranged on the upper side of the piezoelectric layer, and the lower electrode layer is arranged between the piezoelectric layer and the structural layer;

[0015] (2) The electrode layer is a coplanar electrode structure, and the coplanar electrode layer is arranged on the upper side of the piezoelectric layer;

[0016] (3) The thickness of the piezoelectric layer is 0.1 μm to 25 μm;

[0017] (4) The piezoelectric layer includes at least one piezoelectric material layer with a high piezoelectric coefficient.

[0018] In one embodiment, the thickness of the electrode layer is 0.01 μm to 5 μm.

[0019] In one embodiment, the material of the electrode layer is selected from at least one of aluminum, platinum, gold, and chromium.

[0020] In one embodiment, the thickness of the structural layer is 0.1 μm to 50 μm.

[0021] In one embodiment, the thickness of the substrate is 10 μm to 200 μm.

[0022] In one embodiment, the shape of the pMUT unit is circular or rectangular.

[0023] In one embodiment, the arrangement of the plurality of pMUT units in the pMUT array is selected from a rectangular, circular, annular, and polygonal shape.

[0024] In one embodiment, the pMUT array is composed of m*n pMUT units, where m≥2 and n≥2.

[0025] In one embodiment, the touchless ultrasonic gesture control module with force feedback further includes an ASIC chip for generating a control signal to control the operation of the pMUT array.

[0026] In one embodiment, the ASIC chip includes a detection module and a driving module.

[0027] In one embodiment, the pMUT array is integrated with an ASIC chip.

[0028] According to a second aspect of the present application, a touchless ultrasonic gesture control system with force feedback is provided, including a processing unit of a main device and the touchless ultrasonic gesture control module with force feedback.

[0029] According to a third aspect of the present application, a method for using the above-mentioned touchless ultrasonic gesture control system with force feedback is provided, comprising the following process:

[0030] performing a gesture contactlessly within a detection range of the pMUT array;

[0031] The pMUT array transmits ultrasonic waves and receives echo signals reflected by the finger to obtain a real-time position signal of the finger;

[0032] Forming a gesture action signal by continuously obtaining a real-time position signal of the finger;

[0033] The pMUT array is used to locate and focus the transmitted ultrasonic wave toward the finger surface, thereby forming tactile stimulation feedback on the finger surface.

[0034] In one embodiment, the method of use comprises:

[0035] All pMUT units in the pMUT array simultaneously transmit ultrasonic waves and receive echo signals reflected by the finger, and position and focus the transmitted ultrasonic waves toward the surface of the finger; or,

[0036] Some of the pMUT units in the pMUT array transmit ultrasonic waves and receive echo signals reflected by the finger, while another part of the pMUT units locate and focus on the surface of the finger to transmit ultrasonic waves.

[0037] In one embodiment, the touchless ultrasonic gesture control module with force feedback further includes an ASIC chip for generating a control signal to control the operation of the pMUT array, and the ASIC chip includes a detection module and a driving module;

[0038] The detection module in the ASIC chip detects the echo signal received by the pMUT unit in the pMUT array, and transmits the real-time position signal and gesture action signal of the finger to the processing unit of the main device for signal processing to obtain finger positioning and gesture recognition information. Then, the processing unit of the main device controls the pMUT unit in the pMUT array through the driving module, and emits ultrasonic waves of different phases to locate and focus the finger surface according to the finger positioning and gesture recognition information.

[0039] In one implementation, the detection module and the driving module are integrated into an ASIC chip.

[0040] In one embodiment, the method of use includes the following interactive process:

[0041] Finger position perception process, tactile stimulation feedback process, and air gesture control process.

[0042] In one of the embodiments, during the finger position sensing process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the real-time position signal of the finger to the processing unit of the main device for signal processing, and converts the real-time position signal of the finger into the finger position information;

[0043] In the tactile stimulation feedback process, based on the finger position information, the processing unit of the main device generates a delay signal through the driving module to control the pMUT unit in the pMUT array, and according to the finger position information, ultrasonic waves of different phases are emitted to locate and focus the finger surface to form tactile stimulation feedback;

[0044] During the air gesture control process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the gesture action signal to the processing unit of the main device for signal processing, converts the gesture action signal into gesture action information, analyzes and identifies the gesture action, completes the gesture control, and at the same time, based on the tactile stimulation feedback process, enables the ultrasonic wave to continuously locate and track the focus on the finger surface, and continuously form tactile stimulation feedback on the finger surface.

[0045] According to the fourth aspect of the present application, there is provided an application of the above-mentioned touchless ultrasonic gesture control module with force feedback, the above-mentioned touchless ultrasonic gesture control system with force feedback, or the method of using the above-mentioned touchless ultrasonic gesture control module with force feedback in a smart wearable device.

[0046] The beneficial effects of this application include:

[0047] (1) The touchless ultrasonic gesture control module with force feedback provided in this application realizes accurate perception of finger position through ultrasonic emission and reception of the pMUT array, combined with the signal processing capability of the ASIC chip, and can detect the three-dimensional spatial movement of the finger, with significant improvements in spatial resolution and response speed.

[0048] (2) The touchless ultrasonic gesture control module with force feedback provided in this application can generate physical tactile feedback at the finger position through the ultrasonic focusing function of the pMUT array, realizing real-time perception of the user during airborne control. Compared with traditional tactile feedback technology, the tactile sensation provided by this solution is more delicate.

[0049] (3) The touchless ultrasonic gesture control module with force feedback provided by this application realizes high system integration and low power consumption operation through the integration of pMUT array and ASIC chip, and is suitable for use in portable devices such as smart glasses and bracelets. Compared with existing mechanical vibration feedback devices, the system is smaller in size and consumes less energy, which can meet the needs of more miniaturized devices.

[0050] (4) Through the touchless ultrasonic gesture control system with force feedback provided by this application, users can perform precise gesture control in the air, and the operation experience is natural and smooth, which is suitable for scenes that require touchless operation, such as AR / VR and smart home. This system supports a wider range of gesture control and expands the dimension of human-computer interaction.

[0051] (5) This application uses high-voltage electrical coefficient materials as the basic materials of the pMUT array, which significantly improves the conversion efficiency between electrical signals and mechanical vibrations. This allows the pMUT array to maintain stable performance in the transmission and reception of ultrasonic signals, ensuring that the gesture recognition system can capture complex gesture movements in real time and efficiently.

[0052] (6) This application uses phased array technology to control the ultrasonic emission of the pMUT array, achieving precise ultrasonic focusing, and can form a high-precision tactile feedback area on the finger surface, thereby improving the user's operating experience. Compared with traditional ultrasonic emission methods, the phased array can dynamically control the emission direction and focal position of the ultrasonic wave. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of the pMUT array in the touchless ultrasonic gesture control module with force feedback of this application;

[0054] Figure 2 Schematic diagram of the pMUT unit structure in the touchless ultrasonic gesture control module with force feedback of this application, wherein (a) adopts a coplanar electrode structure, and (b) adopts an upper and lower electrode structure;

[0055] Figure 3This is a schematic diagram of the interaction process of the touchless ultrasonic gesture control system with force feedback of this application;

[0056] Figure 4 A schematic diagram of the specific workflow of the interaction process of the touchless ultrasonic gesture control system with force feedback of this application;

[0057] Figure 5 A schematic diagram of the gesture control of smart glasses containing a touchless ultrasonic gesture control system with force feedback is provided in this application.

[0058] Figure ID

[0059] 1. Finger position perception process; 2. Tactile stimulation feedback process; 3. Air gesture control process; 4. Electrode layer; 5. Piezoelectric layer; 6. Structural layer; 7. Substrate; 8. Cavity. DETAILED DESCRIPTION

[0060] The present application is described in detail below with reference to embodiments, but the present application is not limited to these embodiments.

[0061] Unless otherwise specified, the raw materials in the examples of this application were purchased through commercial channels.

[0062] Unless otherwise specified, conventional methods were used for testing and instrument settings were those recommended by the manufacturer.

[0063] In order to solve the problems in the existing human-computer interaction technology that gesture detection and tactile feedback are difficult to achieve efficient and low-power integration and the modules in the existing technology occupy a large space, a touchless ultrasonic gesture control module with force feedback is provided. The space required for the control module on the device is greatly reduced compared with traditional control buttons, making the overall design of the device more simple and beautiful, and also improving the dust and water resistance of the device.

[0064] In one embodiment, a touchless ultrasonic gesture control module with force feedback includes: a pMUT array composed of a plurality of pMUT units;

[0065] The pMUT unit includes a piezoelectric layer formed of a piezoelectric material with a high piezoelectric coefficient;

[0066] The piezoelectric material with a high piezoelectric coefficient is selected from at least one of lithium niobate, lithium tantalate, PZT, and PMN-PT.

[0067] The pMUT array has high-efficiency energy conversion capability and can provide sufficient sound pressure and stable operating frequency in a small device size. The pMUT array using the piezoelectric layer of the above-mentioned high-voltage electric coefficient material is responsible for the recognition of finger positions, gestures and tactile stimulation feedback. The system can provide more accurate gesture recognition and stronger tactile feedback.

[0068] The core technical solution of this application is based on a piezoelectric micromechanical ultrasonic transducer (pMUT) array and its efficient driving and detection mechanism, wherein the operating frequency of the core component, the piezoelectric micromechanical ultrasonic transducer (pMUT) array, can be designed to be between tens of kHz and hundreds of kHz.

[0069] The selection of piezoelectric materials for the pMUT unit in this application has better piezoelectric performance, that is, a larger piezoelectric constant and electromechanical coupling coefficient, while taking into account lower dielectric loss, so that when applied to the ultrasonic gesture control module, the same piezoelectric material layer can be used to realize the emission and reception of ultrasonic waves to detect the finger position and form force feedback, thereby realizing the recognition of gesture movements. In the pMUT array, the pMUT unit can be selected to be partitioned or simultaneously complete the functions of "generating a larger sound pressure level at the finger focal point to generate tactile stimulation to form good force feedback, as well as sensitive recognition and rapid response to finger position and gesture movements".

[0070] This kind of touchless ultrasonic gesture control module with force feedback has great advantages and development potential: on the one hand, the miniaturization of pMUT allows it to be combined with a variety of wearable devices, making up for the lack of human-computer interaction functions of small wearable devices. At the same time, pMUT is easy to process and can achieve low-cost mass production; on the other hand, the ultrasonic transducer made of piezoelectric material does not require DC bias, and the piezoelectric material with high piezoelectric coefficient can produce a greater sound pressure level at the finger position, achieving more obvious tactile stimulation, which helps to have a longer detection distance and a larger detection angle, and provides a larger gesture control space. Therefore, this touchless ultrasonic gesture control module with force feedback based on pMUT array has broad application prospects and development space.

[0071] In one embodiment, the pMUT unit further includes an electrode layer, a structural layer and a substrate;

[0072] The piezoelectric layer has an upper side and a lower side arranged opposite to each other;

[0073] The structural layer and the substrate are sequentially arranged on the lower side of the piezoelectric layer;

[0074] A cavity is formed on the substrate, and the cavity extends to a side of the structural layer away from the piezoelectric layer;

[0075] In one embodiment, the electrode layer is an upper and lower electrode structure, including an upper electrode layer and a lower electrode layer, the upper electrode layer is arranged on the upper side of the piezoelectric layer, and the lower electrode layer is arranged between the piezoelectric layer and the structural layer; or, the electrode layer is a coplanar electrode structure, and the coplanar electrode layer is arranged on the upper side of the piezoelectric layer. Using multiple excitation modes of upper and lower electrodes or coplanar electrodes, the independent control capability of each unit enables it to flexibly adjust the phase of the ultrasonic wave, realize the focusing of the ultrasonic wave on the finger surface, and form a perceptible force feedback.

[0076] In one embodiment, the thickness of the electrode layer is 0.01 μm to 5 μm, and can be designed to be in the shape of an interdigital, circular or rectangular shape to optimize the emission sound pressure and uniformity of the ultrasonic wave.

[0077] In one embodiment, the material of the electrode layer is selected from at least one of aluminum, platinum, gold, and chromium, and has good electrical conductivity and processing adaptability.

[0078] In one embodiment, the piezoelectric layer has a thickness of 0.1 μm to 25 μm, and can produce a significant piezoelectric effect when a voltage is applied, and the resonant frequency and bandwidth of the device can be optimized as required.

[0079] In one embodiment, the piezoelectric layer includes at least one layer of piezoelectric material with a high piezoelectric coefficient. The single-layer or multi-layer structure design of the layer can adapt to different application requirements and ensure more precise ultrasonic control and tactile feedback effects. The ultrasonic working frequency generated by the superposition of a single layer or multiple layers of the piezoelectric layer ranges from tens of kHz to hundreds of kHz.

[0080] In one embodiment, the thickness of the structural layer is 0.1 μm to 50 μm, which realizes the out-of-plane vibration of the diaphragm and enhances the stability and durability of the pMUT.

[0081] In one embodiment, the thickness of the substrate is 10 μm to 200 μm, and combined with the cavity etching technology, the degree of freedom of the vibration membrane is effectively improved, thereby improving the sound pressure level.

[0082] In one embodiment, the shape of the pMUT unit is circular or rectangular.

[0083] In one embodiment, the arrangement of the plurality of pMUT units in the pMUT array is selected from a rectangular, circular, annular, and polygonal shape.

[0084] In one embodiment, the pMUT array is composed of m*n pMUT units, wherein m≥2, n≥2, and those skilled in the art may make specific selections according to needs.

[0085] In one embodiment, the touchless ultrasonic gesture control module with force feedback further includes an ASIC chip for generating a control signal to control the operation of the pMUT array.

[0086] In one embodiment, the ASIC chip includes a detection module and a driving module.

[0087] In one embodiment, the pMUT array is integrated with an ASIC chip. The pMUT array can track the gesture position in real time through the detection module in the ASIC chip, receive and process the echo signal, and accurately obtain the finger position information; the driving module adjusts the emission phase of the pMUT unit through phased array technology, so that the ultrasound is focused on the finger surface to form tactile stimulation, thereby providing the user with a real-time force feedback experience.

[0088] In this application, the specific structures of other devices including the driving module and the detection module other than the pMUT array in the touchless ultrasonic gesture control module with force feedback are not strictly limited, and those skilled in the art can select them according to their needs. Similarly, the main device is not strictly limited, and those skilled in the art can make specific selections based on the smart device to which it is applied.

[0089] The touchless ultrasonic gesture control module with force feedback in this application mainly includes a piezoelectric micromechanical ultrasonic transducer (pMUT) array chip and an ASIC chip. Among them, the pMUT array chip has the following two functions at the same time: (1) it can generate ultrasonic waves and locate the finger position and identify the finger control action by detecting the reflected sound waves, and (2) it can also focus the ultrasonic waves on the fingertips, so that the human hand can feel the force feedback like touching a physical button during the touchless control process, enhancing the touchless control experience. The above two functions can be realized by all the pMUT units of the pMUT array at the same time, or a part of the pMUT units in the pMUT array realizes function (1) and another part of the pMUT units realizes function (2); and the ASIC chip is used to generate a control signal to drive the ultrasonic transducer array chip to emit ultrasonic waves, and detect and process the ultrasonic echo signal received by the ultrasonic transducer array chip and upload it to the processor unit of the main device, wherein the detection module in the ASIC chip is responsible for detecting and collecting the ultrasonic echo received by the pMUT unit, and the drive module is responsible for generating a control signal according to the instruction of the processor unit of the device to drive the pMUT unit to emit ultrasonic waves. The processor unit of the main device described in this application is not strictly limited, and it mainly depends on the selection of the main device that integrates the module of this application. For example, the gesture control module can be integrated into a variety of smart wearable devices such as smart AR / VR glasses, smart bracelets, smart rings, and smart headphones. The design and use method of the touchless ultrasonic gesture control module with force feedback based on this application can provide richer operation options and more flexible human-computer interaction methods, thereby significantly improving the user experience.

[0090] The material selection, structural design and interlayer thickness of the technical solution of this application overcome the problems of low sound pressure level and unstable structure in traditional technology. In particular, through the flexible configuration and partitioning of several pMUT units, some units can be used for ultrasonic positioning, while other units are responsible for generating tactile feedback, providing multifunctional gesture control capabilities. This control module can not only use ultrasound to identify human hand control actions and achieve touchless control of the device, but also generate force feedback through ultrasonic focusing to avoid mechanical wear of the device due to frequent contact and pressing by users, while allowing users to experience a more natural and rich interaction method.

[0091] In one embodiment, a touchless ultrasonic gesture control system with force feedback includes a processing unit of a main device and the touchless ultrasonic gesture control module with force feedback.

[0092] In one embodiment, a method for using a touchless ultrasonic gesture control system with force feedback includes the following process:

[0093] S1. Performing a gesture in a non-contact manner within the detection range of the pMUT array;

[0094] S2, transmitting ultrasonic waves through the pMUT array and receiving echo signals reflected by the finger to obtain a real-time position signal of the finger;

[0095] S3, forming a gesture action signal by continuously obtaining the real-time position signal of the finger;

[0096] S4. The pMUT array is used to locate and focus the transmitted ultrasonic wave toward the finger surface, thereby forming tactile stimulation feedback on the finger surface.

[0097] The tactile stimulation feedback formed by the present application is similar to the force feedback of a physical button, which can enhance the experience of touchless control.

[0098] In one embodiment, the finger surface is preferably a fingertip surface.

[0099] In one embodiment, the method of use comprises:

[0100] All pMUT units in the pMUT array simultaneously transmit ultrasonic waves and receive echo signals reflected by the finger, and position and focus the transmitted ultrasonic waves toward the surface of the finger; or,

[0101] Some of the pMUT units in the pMUT array transmit ultrasonic waves and receive echo signals reflected by the finger, while another part of the pMUT units locate and focus on the surface of the finger to transmit ultrasonic waves.

[0102] That is, in terms of functional area division, the present application involves the division of pMUT working modes, including a mode in which multiple pMUT units are responsible for simultaneously detecting signals and emitting tactile stimulation signals, and also includes a mode in which some pMUT units complete the task of detecting signals while other pMUT units are independently responsible for emitting tactile stimulation signals.

[0103] In one embodiment, the touchless ultrasonic gesture control module with force feedback also includes an ASIC chip for generating control signals to control the operation of the pMUT array, and the ASIC chip includes a detection module and a driving module; the detection module in the ASIC chip detects the echo signal received by the pMUT unit in the pMUT array, and transmits the real-time position signal and gesture action signal of the finger to the processing unit of the main device for signal processing to obtain finger positioning and gesture recognition information, and then the processing unit of the main device controls the pMUT unit in the pMUT array through the driving module, and emits ultrasonic waves of different phases to locate and focus the finger surface according to the finger positioning and gesture recognition information.

[0104] In one embodiment, the driving module can control different units of the pMUT array to generate ultrasonic waves with different phases, and the ultrasonic focusing can be achieved on the specified position through phased array control, thereby achieving tactile stimulation feedback.

[0105] In one embodiment, in the process of controlling the operation of the pMUT array through the driving module, a single pMUT unit can be controlled to work independently, or several pMUT units can be controlled to work uniformly as a group, thereby achieving flexible and efficient response to different areas in the pMUT array.

[0106] In one implementation, the detection module and the driving module are integrated into an ASIC chip.

[0107] In one embodiment, the method of use includes the following interactive process:

[0108] Finger position perception process, tactile stimulation feedback process, and air gesture control process.

[0109] In the finger position sensing process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the real-time position signal of the finger to the processing unit of the main device for signal processing, and converts the real-time position signal of the finger into finger position information. The main function of the finger position sensing process is to accurately detect and locate the finger. The pMUT unit transmits high-frequency ultrasonic waves and receives the reflected echo of the finger. The echo signal reaches the ASIC chip for further processing, extracts the finger position signal, and is processed by the main device processing unit to obtain the coordinate information of the finger, so as to accurately track and record the finger position to realize gesture recognition.

[0110] In the tactile stimulation feedback process, based on the finger position information, the processing unit of the main device generates a delay signal through the driving module to control the pMUT unit in the pMUT array, and according to the finger position information, sends ultrasonic waves of different phases to locate and focus the finger surface to form tactile stimulation feedback. This process is responsible for generating ultrasonic focusing on the finger position to achieve tactile stimulation feedback. When the finger is within the detection range, the ASIC chip generates a delay signal to control each feedback unit of the pMUT to focus at the specified position at the same time, generating a perceptible sound pressure at the finger position. This ultrasonic focusing technology can simulate tactile stimulation, allowing users to feel real tactile feedback during operation. This feedback not only enhances the user's operating experience, but also provides corresponding tactile responses for different operations, thereby improving the accuracy and comfort of the interaction.

[0111] During the air gesture control process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the gesture action signal to the processing unit of the main device for signal processing, converts the gesture action signal into gesture action information, analyzes and identifies the gesture action, completes the gesture control, and at the same time, based on the tactile stimulation feedback process, the ultrasonic wave continuously locates and tracks the finger surface, and continuously forms tactile stimulation feedback on the finger surface. This process relies on the functions of the finger position perception process and the tactile stimulation feedback process to realize the control of the smart wearable device. The user operates the device by performing a series of gestures. During this process, the system continuously detects the position of the finger and transmits the real-time signal to the ASIC chip. The processing unit of the main device generates a delay signal based on the data and transmits it to the ASIC, which is then controlled by the ASIC to generate a delay signal. The pMUT is controlled by the driving module, so that the ultrasonic wave generates the corresponding sound pressure at the finger position to achieve continuous and tracked tactile stimulation feedback. After the gesture is completed, the system identifies different gestures by analyzing the displacement and movement changes of the finger, thereby executing the corresponding instruction requirements.

[0112] In the whole system, the cooperation between modules ensures real-time and accuracy. The precise position data provided by the finger position sensing module, the real-time feedback provided by the tactile stimulation feedback module, and the intelligent response of the air gesture control module together realize the efficient control of smart wearable devices and high-quality user experience. In addition, the system can dynamically adjust the intensity and mode of tactile feedback according to different user gestures, so as to adapt to various operation requirements and application scenarios, which not only improves the operational flexibility of the device, but also enhances the user's interactive experience through the innovative tactile feedback mechanism.

[0113] In one embodiment, when a finger approaches the system, the workflow first enters the finger position sensing process. In this stage, the pMUT array transmits ultrasound and receives echoes. Ultrasonic waves are emitted from the pMUT array and reflected back to the pMUT after encountering the finger, forming an echo signal. The received echo signal is transmitted to the ASIC chip for further processing to determine the precise coordinate information of the finger. This information reflects the position of the finger in three-dimensional space and provides basic data for subsequent operations.

[0114] Next, the workflow turns to the tactile stimulation feedback process. In this process, since the position of each unit in the pMUT array is different and the distance from the finger is different, the time it takes for the ultrasound to reach the touch area will be different. In order to achieve precise focusing of the ultrasound, that is, all ultrasounds converge at the finger position to form a focal point, the ASIC chip driver module is required to generate an appropriate delay signal for each pMUT unit to achieve phased array modulation. This delay adjustment is to synchronize the ultrasound emitted by all units so that it forms a sound pressure focal point on the finger, thereby generating tactile stimulation. The driver module generates different drive signals according to the position of the pMUT unit, and modulates the phase and frequency of its ultrasound respectively to achieve the concentration of sound energy and accurate transmission of tactile feedback.

[0115] Finally, the workflow enters the process of gesture control in the air. During this process, the system continuously monitors the displacement changes of the fingers. As the user performs different gestures, the position changes of the fingers will be captured and processed by the detection circuit through the echo signal. These displacement data are transmitted to the ASIC chip in real time to identify the user's gesture instructions. According to the recognized gesture instructions, the system adjusts the pMUT array through the driving circuit, continues to generate ultrasonic focus at the finger position, and makes corresponding adjustments according to the dynamic displacement of the finger, thereby achieving continuous tactile stimulation feedback and precise gesture control.

[0116] Overall, the workflow, from finger approach, ultrasonic focused tactile feedback, to gesture recognition and operation control, constitutes a complete interactive process. Users can get instant tactile feedback during the operation, which enhances the realism and intuitiveness of the operation. Each step works closely together to ensure the accuracy of gesture recognition and the effectiveness of tactile feedback.

[0117] This application effectively solves the problems of insufficient sound pressure level, unclear tactile feedback and low detection accuracy in the prior art through the combination of material selection and structural design scheme, and significantly improves the human-computer interaction performance and user experience of the system.

[0118] In one embodiment, a touchless ultrasonic gesture control module with force feedback or a method for using a touchless ultrasonic gesture control module with force feedback is applied in a smart wearable device.

[0119] Based on the technical solution of the touchless ultrasonic gesture control module with force feedback in this application, users can feel force feedback while performing non-contact control of the device through air gestures, which effectively saves the physical space of the device and makes the control content more diversified. The application of force feedback technology greatly enhances the user's control experience and makes the interaction process more intuitive and immersive.

[0120] The touchless ultrasonic gesture control module with force feedback of the present application can be applied to a variety of smart wearable devices, such as smart glasses, bracelets, rings, headphones and other smart wearable devices, which usually have strict requirements on volume and appearance. The touchless operation module can better meet this demand. The touchless ultrasonic gesture control module with force feedback of the present application can achieve miniaturized integration and diversified human-computer interaction, significantly improving operational flexibility and user experience.

[0121] In order to further enhance the user experience, the touchless ultrasonic gesture control module with force feedback provided by the present invention can also adjust the operating frequency and power through different designs to adapt to different usage scenarios. For example, in AR / VR applications, the system can be configured near the human ear to provide more delicate tactile feedback and enhance the realism of device control; in the field of medical health, the system can be used for rehabilitation training to help users restore hand function through tactile feedback. In addition, the touchless ultrasonic gesture control module with force feedback of the present invention can also be integrated with other sensors and devices to further expand its functions and application scope. For example, it can be combined with a visual sensor to provide more accurate gesture recognition; combined with a temperature sensor, it can provide dual feedback of touch and temperature, etc. Utilizing the high performance characteristics of pMUT, the touchless ultrasonic gesture control module with force feedback proposed by the present invention provides an efficient and flexible human-computer interaction method, which not only improves the user experience, but also expands the application scenarios of smart devices.

[0122] Embodiment 1 Touchless ultrasonic gesture control module with force feedback

[0123] The touchless ultrasonic gesture control module with force feedback includes: a pMUT array composed of a plurality of pMUT units, such as Figure 1 As shown, the pMUT array is composed of 3*3 pMUT units, and the shape of the pMUT unit is circular.

[0124] like Figure 2 As shown, the pMUT unit includes, from top to bottom, an electrode layer 4, a piezoelectric layer 5 formed of a piezoelectric material with a high piezoelectric coefficient, a structural layer 6 and a substrate 7. A cavity 8 is provided on the substrate, and the cavity 8 extends to the side of the structural layer 6 away from the piezoelectric layer 5. The electrode layer 4 is an upper and lower electrode structure ( Figure 2 (a)) or coplanar electrode structure ( Figure 2(b)), wherein the upper and lower electrode structures include an upper electrode layer and a lower electrode layer, the upper electrode layer is arranged on the upper side of the piezoelectric layer 5, and the lower electrode layer is arranged between the piezoelectric layer 5 and the structural layer 6; in the coplanar electrode structure, the coplanar electrode layer is arranged on the upper side of the piezoelectric layer 5.

[0125] The piezoelectric material with a high piezoelectric coefficient is selected from at least one of PZT, PMN-PT, lithium niobate, and lithium tantalate.

[0126] The thickness of the electrode layer 4 is 0.01 μm to 5 μm.

[0127] The material of the electrode layer 4 is selected from at least one of aluminum, platinum, gold and chromium.

[0128] The thickness of the piezoelectric layer 5 is 0.1 μm to 25 μm.

[0129] The piezoelectric layer 5 includes at least one piezoelectric material layer with a high piezoelectric coefficient.

[0130] The thickness of the structural layer 6 is 0.1 μm to 50 μm.

[0131] The thickness of the substrate 7 is 10 μm to 200 μm.

[0132] The touchless ultrasonic gesture control module with force feedback is integrated with an ASIC chip for generating control signals to control the operation of the pMUT array. The ASIC chip includes a detection module and a drive module.

[0133] Example 2: Method for using the touchless ultrasonic gesture control module with force feedback

[0134] The method for using the touchless ultrasonic gesture control module with force feedback in Example 1 includes: performing gesture actions contactlessly within the detection range of the pMUT array.

[0135] All pMUT units in the pMUT array simultaneously transmit ultrasonic waves and receive echo signals reflected by the finger, and position and focus the transmitted ultrasonic waves on the finger surface; or, some pMUT units in the pMUT array transmit ultrasonic waves and receive echo signals reflected by the finger, and the other part of the pMUT units position and focus the transmitted ultrasonic waves on the finger surface. The real-time position signal of the finger is obtained through the echo signal, and the gesture action signal is formed by continuously obtaining the real-time position signal of the finger. The ultrasonic waves are positioned and focused to form tactile stimulation feedback on the finger surface.

[0136] The detection module in the ASIC chip detects the echo signal received by the pMUT unit in the pMUT array, and transmits the real-time position signal and gesture action signal of the finger to the processing unit of the main device for signal processing to obtain finger positioning and gesture recognition information. Then, the processing unit of the main device controls the pMUT unit in the pMUT array through the driving module, and emits ultrasonic waves of different phases to locate and focus the finger surface according to the finger positioning and gesture recognition information.

[0137] like Figure 3 As shown, the usage method includes the following interactive process:

[0138] Finger position perception process, tactile stimulation feedback process, and air gesture control process.

[0139] like Figure 4 As shown, in the finger position sensing process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the real-time position signal of the finger to the processing unit of the main device for signal processing, and converts the real-time position signal of the finger into finger position information;

[0140] In the tactile stimulation feedback process, based on the finger position information, the processing unit of the main device generates a delay signal through the driving module to control the pMUT unit in the pMUT array, and according to the finger position information, ultrasonic waves of different phases are emitted to locate and focus the finger surface to form tactile stimulation feedback;

[0141] During the air gesture control process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the gesture action signal to the processing unit of the main device for signal processing, converts the gesture action signal into gesture action information, analyzes and identifies the gesture action, completes the gesture control, and at the same time, based on the tactile stimulation feedback process, enables the ultrasonic wave to continuously locate and track the focus on the finger surface, and continuously form tactile stimulation feedback on the finger surface.

[0142] During operation, each unit of the pMUT array achieves synchronous focusing of the ultrasound at the finger position through phased array modulation. According to the different distances from each pMUT unit to the finger, a delayed control signal is generated, that is, the ASIC chip driver module generates different drive signals for different pMUT units, thereby realizing phased array modulation and achieving synchronous focusing of the ultrasound at the finger position, ensuring that the ultrasound is focused and forming tactile stimulation. The independence of each unit allows flexible task allocation, some pMUT units can be responsible for detection, and others are responsible for tactile feedback. The system continuously monitors changes in finger position and analyzes and recognizes user gestures in real time. The recognized gesture information is transmitted to the ASIC chip, and the chip generates corresponding control signals to adjust the driving mode of the pMUT array to achieve gesture control and corresponding tactile feedback. The system can adjust the operating frequency and power according to different scenarios. For example, it can provide delicate tactile feedback in AR / VR applications, and can help users restore hand function in medical rehabilitation. Other sensors such as visual or temperature sensors can also be integrated to expand gesture recognition and feedback functions.

[0143] Example 3 Application of touchless ultrasonic gesture control module with force feedback in smart glasses

[0144] The schematic diagram of the application in smart glasses is as follows Figure 5 As shown in the figure, the schematic diagram describes a use scenario of a touchless ultrasonic gesture control module with force feedback. In the schematic diagram, the control module is installed on the legs of the glasses. When the smart glasses device is turned on, the pMUT array starts to work normally, and ultrasonic detection is performed on the human hand that enters its detection range. The echo signal enters the ASIC chip detection module to obtain the position information of the human hand; the ASIC chip driver module controls the pMUT array to emit ultrasonic waves, and focuses at the specified position to generate sound pressure to achieve tactile stimulation. When the user makes various gestures, the spatial position of the finger changes. At this time, the pMUT array can track the position change in real time. On the one hand, gesture recognition can be performed through the law of position change, and gesture control can be achieved; on the other hand, the ASIC chip further processes the detected signal, thereby outputting an appropriate delayed driving signal to control the pMUT array. The ultrasonic transmitting unit emits ultrasonic waves to the tactile feedback area, moves while focusing on the specified position, tracks the moving object, generates sound pressure on its surface, and achieves continuous tracking tactile feedback.

[0145] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. A touchless ultrasonic gesture control module with force feedback, characterized in that: include: A pMUT array composed of a plurality of pMUT units; The pMUT unit includes a piezoelectric layer formed of a piezoelectric material with a high piezoelectric coefficient; The piezoelectric material with a high piezoelectric coefficient is selected from at least one of lithium niobate, lithium tantalate, PZT, and PMN-PT.

2. The touchless ultrasonic gesture control module with force feedback according to claim 1, characterized in that: The pMUT unit also includes an electrode layer, a structural layer and a substrate; The piezoelectric layer has an upper side and a lower side arranged opposite to each other; The structural layer and the substrate are sequentially arranged on the lower side of the piezoelectric layer; A cavity is formed on the substrate, and the cavity extends to a side of the structural layer away from the piezoelectric layer; The pMUT unit satisfies at least one of the following conditions: (1) The electrode layer is an upper and lower electrode structure, including an upper electrode layer and a lower electrode layer, wherein the upper electrode layer is arranged on the upper side of the piezoelectric layer, and the lower electrode layer is arranged between the piezoelectric layer and the structural layer; (2) The electrode layer is a coplanar electrode structure, and the coplanar electrode layer is arranged on the upper side of the piezoelectric layer; (3) The thickness of the piezoelectric layer is 0.1 μm to 25 μm; (4) The piezoelectric layer includes at least one layer of piezoelectric material with a high piezoelectric coefficient; Preferably, the arrangement of the plurality of pMUT units in the pMUT array is selected from one of a rectangular, a circular, a ring, and a polygon; Preferably, the pMUT array consists of m*n pMUT units, wherein m≥2 and n≥2.

3. The touchless ultrasonic gesture control module with force feedback according to claim 1, characterized in that: The touchless ultrasonic gesture control module with force feedback also includes an ASIC chip for generating a control signal to control the operation of the pMUT array; Preferably, the ASIC chip includes a detection module and a driving module; Preferably, the pMUT array is integrated with an ASIC chip.

4. A touchless ultrasonic gesture control system with force feedback, characterized in that: It comprises a processing unit of a main device and a touchless ultrasonic gesture control module with force feedback as described in any one of claims 1 to 3.

5. The method for using the touchless ultrasonic gesture control system with force feedback as claimed in claim 4, characterized in that: The process includes the following: performing a gesture contactlessly within a detection range of the pMUT array; The pMUT array transmits ultrasonic waves and receives echo signals reflected by the finger to obtain a real-time position signal of the finger; Forming a gesture action signal by continuously obtaining a real-time position signal of the finger; The pMUT array is used to locate and focus the transmitted ultrasonic wave toward the finger surface, thereby forming tactile stimulation feedback on the finger surface.

6. The method for using the touchless ultrasonic gesture control system with force feedback according to claim 5, characterized in that: The method of use includes: All pMUT units in the pMUT array simultaneously transmit ultrasonic waves and receive echo signals reflected by the finger, and position and focus the transmitted ultrasonic waves toward the surface of the finger; or, Some of the pMUT units in the pMUT array transmit ultrasonic waves and receive echo signals reflected by the finger, while another part of the pMUT units locate and focus on the surface of the finger to transmit ultrasonic waves.

7. The method for using the touchless ultrasonic gesture control system with force feedback according to claim 5, characterized in that: The touchless ultrasonic gesture control module with force feedback also includes an ASIC chip for generating a control signal to control the operation of the pMUT array, and the ASIC chip includes a detection module and a driving module; The detection module in the ASIC chip detects the echo signal received by the pMUT unit in the pMUT array, and transmits the real-time position signal and gesture action signal of the finger to the processing unit of the main device for signal processing to obtain finger positioning and gesture recognition information. Then, the processing unit of the main device controls the pMUT unit in the pMUT array through the driving module, and emits ultrasonic waves of different phases to locate and focus the finger surface according to the finger positioning and gesture recognition information.

8. The method for using the touchless ultrasonic gesture control system with force feedback according to claim 5, characterized in that: The usage method includes the following interactive process: Finger position perception process, tactile stimulation feedback process, and air gesture control process.

9. The method for using the touchless ultrasonic gesture control system with force feedback according to claim 8, characterized in that: In the finger position sensing process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the real-time position signal of the finger to the processing unit of the main device for signal processing, and converts the real-time position signal of the finger into finger position information; In the tactile stimulation feedback process, based on the finger position information, the processing unit of the main device generates a delay signal through the driving module to control the pMUT unit in the pMUT array, and according to the finger position information, ultrasonic waves of different phases are emitted to locate and focus the finger surface to form tactile stimulation feedback; During the air gesture control process, the detection module detects the echo signal received by the pMUT unit in the pMUT array, transmits the gesture action signal to the processing unit of the main device for signal processing, converts the gesture action signal into gesture action information, analyzes and identifies the gesture action, completes the gesture control, and at the same time, based on the tactile stimulation feedback process, enables the ultrasonic wave to continuously locate and track the focus on the finger surface, and continuously form tactile stimulation feedback on the finger surface.

10. Application of the touchless ultrasonic gesture control module with force feedback as described in any one of claims 1 to 3, the touchless ultrasonic gesture control system with force feedback as described in claim 4, and the method for using the touchless ultrasonic gesture control system with force feedback as described in any one of claims 5 to 9 in smart wearable devices.

Images