Smart terminal virtual sliding button haptic reproduction rendering method and device

Abstract

The application provides a kind of smart terminal virtual sliding button tactile reproduction rendering method and device, belong to touch screen technology and man-machine interaction field. Including the position coordinate of finger on touch screen when user carries out interactive operation, displacement and speed, judge touch click or drag, simulate similar tactile feedback of click, increase tactile braking effect through tactile pawl, according to the size of displacement and movement speed, real-time control the friction coefficient when sliding, produce tactile feedback, realize the tactile reproduction rendering of virtual sliding button. The application produces corresponding tactile feedback in the process of tactile perception through different interactive operations of virtual sliding button, improves the interaction efficiency and accuracy, increases the experience of feeling things with fingertip when using touch screen, reduces the cognitive load of vision and hearing, improves the interaction efficiency and immersion.

Description

Technical Field

[0001] This invention belongs to the fields of touch screen technology and human-computer interaction, and particularly relates to a method and device for rendering the tactile feedback of a virtual sliding button on a smart terminal. Background Technology

[0002] With the advancement of technology, touchscreens have become an indispensable part of mobile phones, tablets, ATMs, vending machines, car dashboards, and smart home appliances. On these interactive screens, physical controls are gradually being replaced by virtual controls to increase the touch area. However, virtual controls lack sophisticated haptic feedback, leading to a decline in user experience and task performance, increased cognitive load, and making interaction uncomfortable and slow. Furthermore, without haptic cues, users must rely on visual and auditory cues to obtain information, which may interfere with other tasks. For visually impaired individuals and other special groups, the lack of haptic information transmission channels when using smart terminal devices causes further inconvenience in daily life and learning.

[0003] In today's widespread smart terminals, using virtual sliding buttons for human-computer interaction is a common practice. Virtual sliding buttons can be used to control volume and audio / video playback, unlock devices via a slide-to-unlock mechanism, verify login credentials for apps, and control smart home appliances. Tasks are completed by clicking and dragging the button, typically providing visual (e.g., changes in appearance or color) or auditory (e.g., sounds of clicking / dragging, or notification sounds upon task completion) feedback, or a combination of both. Compared to visual and auditory feedback, tactile feedback is less frequently used for sensory feedback in virtual sliding button interactions. Adding tactile feedback to the interface can enhance the realism and immersion of the interaction, improving efficiency and accuracy.

[0004] There are already some methods for applying virtual sliding buttons to touchscreen interfaces.

[0005] The 2014 paper “Virtual touch screen “VIRTOS” implementing virtual touchbuttons and virtual sliders using a projector and camera” proposed a large interactive display called VIRTOS with virtual touch buttons and touch sliders. It detects touch based on the shadow area projected by the user’s hand and developed a groundbreaking game with touch sliders.

[0006] The 2015 paper "GelTouch: Localized Tactile Feedback Through Thin, Programmable Gel" uses a gel layer on a tablet computer that can selectively switch between soft and hard textures. It designs a haptic slider that can guide scrolling motion when browsing presented images, establishes a mapping model between scrolling speed and the frequency of haptic cues perceived during scrolling, and provides simple haptic feedback.

[0007] In 2018, Chinese patent CN110764421A obtained control commands by acquiring the user's sliding operation of the virtual slider in the virtual slide rail, thereby controlling the operation of the controlled equipment. This provides the user with a wider range of adjustment, allowing the user to control the controlled equipment more smoothly, and better achieves interactive control between the user and the fan, thus improving the user experience.

[0008] An analysis of the current state of research on virtual sliding buttons for smart terminals reveals that most existing virtual sliding buttons applied to touchscreens still transmit information through vision and hearing, lacking tactile feedback. Users experience a lack of guiding cues, which reduces the interactive experience. The few virtual sliding buttons that incorporate tactile feedback use a gel whose hardness changes with the temperature rise due to current, coated on a resistive touchscreen. This method offers only a limited tactile experience and cannot be used to create programmable tactile rendering through friction modulation, resulting in poor tactile realism. Summary of the Invention

[0009] This invention provides a method and apparatus for rendering the tactile feedback of a virtual sliding button on a smart terminal. This provides tactile information, reducing the user's reliance on visual and auditory cues during interaction and enriching the interactive experience. When the slider is touched, an ultrasonic vibration drive signal is applied to simulate the click sensation, indicating to the user that the virtual sliding button has been detected. As the virtual sliding button is dragged across different areas of the slider, a mapping relationship is established between displacement / velocity and drive signal parameters, guiding and confirming the user's interactive operations, thereby improving the accuracy of the interaction.

[0010] The technical solution adopted by this invention includes the following steps:

[0011] (1) Obtain the coordinates of the user's finger position on the touch screen when the user performs an interactive operation;

[0012] (2) Obtain the displacement and speed of the user's finger at different times during the interactive operation;

[0013] (3) Determine whether the user's gesture during the interaction is a touch click or a drag;

[0014] (4) Simulate tactile feedback similar to clicking when the user touches and presses the virtual sliding button;

[0015] (5) Increase tactile braking effect with tactile pawls at the start of sliding and when approaching the target;

[0016] (6) During the sliding process, displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage, and velocity-electrostatic force driving signal frequency mapping models are established in different regions respectively. The finger displacement distance and movement speed are used as inputs, and the driving signal voltage and frequency are used as outputs. The friction coefficient during sliding is controlled in real time according to the magnitude of displacement and movement speed to generate tactile feedback and realize the tactile reproduction rendering of the virtual sliding button.

[0017] The virtual sliding button described in this invention is applied to a slider CAPTCHA for user registration and login verification and secure payment verification. The slider CAPTCHA is drawn and run on the Unity2017.2.4f1 platform.

[0018] The implementation method of step (1) of the present invention is as follows: obtaining the position of the user's finger on the touch screen when interacting includes real-time tracking and measurement of the finger position information by using a touch display technology that combines the haptic reproduction screen and the touch positioning screen into one.

[0019] The implementation method of step (2) of the present invention is as follows: A two-dimensional coordinate system is established with the location of the virtual sliding button as the origin. The X-axis represents the horizontal direction, and the Y-axis represents the vertical direction. The distance x or y between the finger and the origin at different times is calculated, which is the displacement at the current time. The time taken for the finger to slide horizontally or vertically from the previous position to the current position is calculated. The horizontal or vertical sliding distance is or Then the current speed of the finger movement is or .

[0020] The implementation method of step (3) of the present invention is as follows: calculate the position coordinate difference within 1 second when the finger starts to touch the virtual sliding button. If the difference is equal to 0, it is a touch click; otherwise, it is a drag.

[0021] The implementation method of step (4) of the present invention is as follows:

[0022] 1) Obtain the pressure applied when the user presses the virtual slider button. This includes installing a force sensor on the terminal housing to measure and record force in real time;

[0023] 2) Set the pressure threshold = 0.5N, when the user's interaction with the touchscreen is judged as a static gesture touch click and At that moment, an ultrasonic vibration drive signal is immediately applied, and the click sensation is simulated by the instantaneous change in friction and the increase in perceived roughness.

[0024] The implementation method of step (5) of the present invention is as follows: when the sliding begins and when the target is about to be reached, a square pawl of about 4mm is designed to provide tactile braking effect and prompt the user's operation.

[0025] The implementation method of step (6) of the present invention is as follows:

[0026] (1) Divide the sliding track of the virtual sliding button to be simulated into two parts according to the distance from the target point: a low-friction region and a high-friction region. The maximum displacement of each region is as follows: and ;

[0027] (2) Establish displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage and velocity-electrostatic force driving signal frequency mapping models in different regions to realize the effect of tactile feedback changing with the movement displacement and velocity.

[0028] 1) Ultrasonic vibration tactile reproduction technology

[0029] Under the influence of a driving signal, the piezoelectric ceramic drives the touchscreen to vibrate at high frequency, forming an air film between the finger and the touchscreen. This reduces the actual contact area, thereby decreasing the friction between the two. When the finger contacts the touchscreen, the air film creates a squeezing force between them.

[0030]

[0031]

[0032]

[0033] in, For extrusion pressure, The offset ratio determines the magnitude of the extrusion pressure, and its range is... S represents the contact area between the finger and the touchscreen. Atmospheric pressure, The average amplitude of the fingerprint. For the surface roughness of the touchscreen, The vibration amplitude of the touchscreen is measured when the piezoelectric ceramic is operating at its resonant frequency.

[0034]

[0035] in, For the mechanical quality factor of the piezoelectric ceramic sheet, Where is the piezoelectric constant, and V is the driving signal voltage. For the resonator thickness, The length of the piezoelectric ceramic sheet, , The compressive force is the length of the plate under compression and stretching along the Z-axis. The presence of this reduces the normal force on the fingers, resulting in tangential friction:

[0036]

[0037] in, This refers to the tangential frictional force between the finger and the screen. Apply pressure to the fingers, The coefficient of friction, Given the amplitude of the ultrasonic vibration driving signal voltage, it can be deduced from the formula that the larger the amplitude of the driving signal voltage, the greater the vibration amplitude of the touchscreen. The larger the offset ratio The larger size leads to increased squeezing force, which in turn reduces the tangential friction between the finger and the touchscreen.

[0038] The mapping relationship between tactile roughness and ultrasonic vibration driving signal frequency was obtained through subjective perception scoring experiments. A velocity-ultrasonic vibration driving signal frequency mapping model was then established. The scoring standard was: "the stronger the perceived roughness, the higher the subjective score; the weaker the perceived roughness, the lower the subjective score." The fitting relationship was obtained through experiments.

[0039]

[0040] in, The frequency of the ultrasonic vibration driving signal. For tactile perception of roughness;

[0041] 2) Electrostatic tactile reproduction technology

[0042] When a driving signal is applied to the transparent conductive layer of the touchscreen, an electrostatic attraction is generated between the finger and the touchscreen due to the difference in the polarity of the charge induction:

[0043]

[0044] in, For electrostatic attraction, The free space permittivity, and Here, A represents the relative permittivity of the skin and the insulating film, respectively, and A is the contact area between the finger and the touchscreen. and These are the thicknesses of the stratum corneum and the insulating layer of the skin, respectively. To drive the signal voltage, the electrostatic attraction can be seen from the formula. With drive signal voltage The tangential friction force generated at this time is directly proportional to the square of the number of tangential friction forces.

[0045]

[0046] in, For tangential friction, The coefficient of sliding friction between the finger and the touchscreen. Apply pressure to the fingers, The amplitude of the electrostatic driving signal voltage is determined by the electrostatic force. The larger the driving voltage amplitude, the greater the electrostatic attraction, which in turn increases the tangential friction between the finger and the touch screen.

[0047] A tactile roughness-electrostatic force driven signal frequency mapping model was established in the same manner, and the fitting relationship was obtained through experiments:

[0048]

[0049] in, The frequency of the signal is driven by electrostatic force. For tactile perception of roughness;

[0050] 3) Establish a displacement-driving signal voltage mapping model

[0051] The maximum frictional force occurs when the finger's displacement from the starting point is 0. The finger's maximum displacement is from the starting point. At that time, corresponding to the minimum friction force The displacement of the finger from the starting point is mapped proportionally to the frictional force experienced by the finger during the movement:

[0052]

[0053] in, This represents the displacement of the finger from the starting point. displacement The friction force experienced by the fingers The fitting coefficients for the displacement-friction mapping relationship are... For the intercept, compare it with... By combining these, the mapping relationship between displacement and ultrasonic vibration driving signal voltage amplitude can be obtained:

[0054]

[0055] in, The fitting coefficients for the displacement-friction mapping relationship in ultrasonic vibration tactile reproduction technology are given. This is the intercept. That is, the greater the displacement of the finger from the starting point, the smaller the tangential frictional force on the finger (i.e., the greater the squeezing force), and the greater the amplitude of the ultrasonic vibration driving signal voltage. Similarly, the mapping relationship between displacement and electrostatic force driving signal voltage amplitude is as follows:

[0056]

[0057] in, The fitting coefficients for the displacement-friction mapping relationship in electrostatic tactile reproduction technology are given. The intercept is... The displacement of the finger from the target point is the smaller the tangential friction force on the finger, i.e., the smaller the electrostatic attraction force, and the smaller the amplitude of the electrostatic force driving signal voltage.

[0058] 4) Establish a speed-drive signal frequency mapping model

[0059] Based on the speed at which a person swipes the touchscreen under different conditions, a movement speed range of 0-100 mm / s is selected. As the movement speed increases, the tactile roughness is enhanced, that is:

[0060]

[0061] in, The speed of finger movement. In order to speed Tactile roughness at that time To perceive the maximum roughness, The fitting coefficients for the speed-tactile perception roughness mapping relationship are then compared with... Combining these, we can obtain the mapping relationship between velocity and the frequency of the ultrasonic vibration driving signal:

[0062]

[0063] in, These are the fitting coefficients for the velocity-tactile roughness mapping relationship in ultrasonic vibration tactile reproduction technology. That is, the greater the finger movement speed, the stronger the tactile roughness, and the lower the frequency of the ultrasonic vibration driving signal. Similarly, the velocity-electrostatic force driving signal frequency mapping relationship is as follows:

[0064]

[0065] in, The fitting coefficients for the velocity-tactile roughness mapping relationship in electrostatic tactile reproduction technology;

[0066] (3) Ultrasonic vibration tactile reproduction technology is used in low-friction areas. The specific applied signal is: a sine wave with a carrier frequency of 25.4 kHz; based on the mapping relationship between displacement and ultrasonic vibration driving signal voltage. As the displacement distance between the finger and the starting point increases, the driving signal voltage gradually increases between 30-120Vpp, causing the frictional resistance experienced by the finger to gradually decrease; based on the mapping relationship between velocity and ultrasonic vibration driving signal frequency... As the finger sliding speed gradually increases, the frequency of the amplitude modulation signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness.

[0067] (4) Electrostatic tactile reproduction technology is used in high-friction areas. The specific signal applied is a sine wave; based on the mapping relationship between displacement and electrostatic driving signal voltage. As the distance between the finger and the target point gradually decreases, the driving signal voltage gradually increases between 40-150Vpp, thus increasing the frictional resistance experienced by the finger. This is based on the mapping relationship between speed and electrostatic force driving signal frequency. As the finger slides faster, the frequency of the adjustment drive signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness. The drive signal is turned off the moment the finger is about to reach the target position.

[0068] The present invention provides a tactile rendering device for a virtual sliding button on a smart terminal, comprising:

[0069] (1) Tactile presentation unit: including electrostatic touch screen and piezoelectric ceramic, presenting human-computer interaction interface, receiving drive signals transmitted by tactile driving unit, and providing tangential force tactile feedback on touch screen;

[0070] (2) Tactile driving unit: including signal generator, digital-to-analog converter, amplifier and power supply, used to synthesize the driving signals required for electrostatic force and ultrasonic vibration, and transmit them to the tactile presentation unit to drive and generate tactile sensation at the corresponding position;

[0071] (3) Finger positioning unit: includes a device with positioning technology, which can acquire the position coordinate information of the finger when it moves in real time and transmit it to the tactile processing unit for processing and calculation;

[0072] (4) Tactile processing unit: includes various smart terminals with touch screens, used to output visual and auditory information, can receive finger position information transmitted from the finger positioning unit and process and calculate, and at the same time calculate the parameters of the required driving signal according to the mapping model and transmit them to the tactile driving unit.

[0073] The present invention has the following advantages:

[0074] 1. A tactile reproduction device for a virtual sliding button on an intelligent terminal based on variable friction control does not require any additional vibration actuators. It is small in size and simple in structure, increases tactile feedback, reduces visual burden, and improves interaction efficiency and accuracy.

[0075] 2. The haptic feedback provided when the user touches or clicks is not limited by area and can be implemented at any location on the touchscreen.

[0076] 3. The simulation effect depends only on the parameters of the drive signal applied at different times, so it can be applied to any application interface that includes a virtual slider button.

[0077] 4. Based on the combined tactile feedback of electrostatic force and ultrasonic vibration, the range of force transmitted to the fingertips can be maximized during active touch, which increases the experience of feeling things with the fingertips when using the touch screen, reduces the cognitive load of vision and hearing, improves interaction efficiency and immersion, and provides a wider range of friction modulation. Attached Figure Description

[0078] Figure 1 This is a structural diagram of the tactile reproduction device of the present invention;

[0079] Figure 2 This is a structural diagram of the integrated tactile and touch control system of the present invention;

[0080] Figure 3 This is a schematic diagram of the actuator and force sensor of the present invention;

[0081] Figure 4 This is a flowchart of the present invention;

[0082] Figure 5 This is a diagram showing the friction pattern during touch clicking according to the present invention;

[0083] Figure 6 This is a diagram showing the position on the slide rail that generates tactile braking according to the present invention;

[0084] Figure 7 This is a flowchart illustrating the process of establishing the displacement / velocity-drive signal parameter mapping relationship according to the present invention;

[0085] Figure 8 This is a diagram of the friction patterns generated by applying a driving signal to the entire virtual slider CAPTCHA.

[0086] Figure 9(a) is a schematic diagram of the user interface when the virtual slider verification code is about to be dragged.

[0087] Figure 9(b) is a schematic diagram of the user interface when the virtual slider verification code is about to reach the target position;

[0088] Figure 9(c) is a schematic diagram of the user interface when the virtual slider verification code reaches the target position. Detailed Implementation

[0089] This invention proposes a method and apparatus for rendering the tactile feedback of a virtual sliding button on a smart terminal, providing tactile information, reducing the user's reliance on visual and auditory cues during interaction, and enriching the interactive experience. To make the objectives, features, and advantages of this invention more apparent and understandable, a detailed description will be provided below with reference to accompanying drawings and examples.

[0090] The structure of the virtual sliding button tactile reproduction device based on variable friction control, which is based on either electrostatic force or ultrasonic vibration, or a combination of both, as described in this invention, is as follows: Figure 1 As shown, it includes:

[0091] (1) Tactile presentation unit 11: includes a tactile and touch integrated touch screen and piezoelectric ceramic, presents a human-computer interaction interface, receives the driving signal transmitted by the tactile driving unit, and provides tangential force tactile feedback on the touch screen;

[0092] (2) Tactile driving unit 12: includes a signal generator, digital-to-analog converter, amplifier and power supply, used to synthesize the driving signals required for electrostatic force and ultrasonic vibration, and transmit them to the tactile presentation unit to drive and generate tactile sensation at the corresponding position;

[0093] (3) Finger positioning unit 13: includes a device with positioning technology, which can acquire the position coordinate information of the finger when it moves in real time and transmit it to the tactile processing unit for processing and calculation;

[0094] (4) Tactile processing unit 14: includes various smart terminals with touch screens, used to output visual and auditory information, can receive finger position information transmitted from the finger positioning unit and process and calculate, and at the same time calculate the parameters of the required driving signal according to the mapping model and transmit them to the tactile driving unit.

[0095] The virtual sliding button is used in the slider CAPTCHA to verify user registration and login, and to verify secure payment. The slider CAPTCHA is drawn and run on the Unity2017.2.4f1 platform.

[0096] like Figure 4 As shown, the virtual sliding button tactile reproduction rendering method based on variable friction control using either ultrasonic vibration or electrostatic force, or a combination of both, includes the following steps:

[0097] (1) Obtain the coordinates of the user's finger position on the touch screen when the user performs an interactive operation;

[0098] (2) Obtain the displacement and speed of the user's finger at different times during the interactive operation;

[0099] (3) Determine whether the user's gesture during the interaction is a touch click or a drag;

[0100] (4) Simulate tactile feedback similar to clicking when the user touches and presses the virtual sliding button;

[0101] (5) Increase tactile braking effect with tactile pawls at the start of sliding and when approaching the target;

[0102] (6) During the sliding process, displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage, and velocity-electrostatic force driving signal frequency mapping models are established in different regions respectively. The finger displacement distance and movement speed are used as inputs, and the driving signal voltage and frequency are used as outputs. The friction coefficient during sliding is controlled in real time according to the magnitude of displacement and movement speed to generate tactile feedback and realize the tactile reproduction rendering of the virtual sliding button.

[0103] The virtual sliding button is used in the slider CAPTCHA to verify user registration and login, and to verify secure payment. The slider CAPTCHA is drawn and run on the Unity2017.2.4f1 platform.

[0104] The implementation method of step (1) is as follows: Obtaining the position of the user's finger on the touchscreen during interaction includes real-time tracking and measurement of the finger position information using a touchscreen display technology that combines the haptic reproduction screen and the touch positioning screen into a single unit. Figure 2 As shown.

[0105] The implementation method of step (2) is as follows: Establish a two-dimensional coordinate system with the location of the virtual sliding button as the origin. The X-axis represents the horizontal direction, and the Y-axis represents the vertical direction. Calculate the distance x or y between the finger and the origin at different times, which is the displacement at the current time. Calculate the time taken for the finger to slide horizontally or vertically from the previous position to the current position. The horizontal or vertical sliding distance is or Then the current speed of the finger movement is or .

[0106] The implementation method of step (3) is as follows: calculate the position coordinate difference within 0.5s when the finger starts to touch the virtual sliding button. If the difference is equal to 0, it is a touch click; otherwise, it is a drag.

[0107] The implementation method of step (4) is as follows:

[0108] 1) Obtain the pressure applied when the user presses the virtual slider button. This includes installing force sensors on the terminal housing to measure and record data in real time, such as... Figure 3 As shown.

[0109] 2) Set the pressure threshold = 0.5N, when the user's interaction with the touchscreen is judged as a static gesture touch click and Immediately upon application, an ultrasonic vibration drive signal is generated. The sensation of clicking is simulated by the instantaneous change in frictional force and the perceived increase in roughness. Specifically, the applied signal is a square wave with a voltage amplitude of 120V and a carrier frequency of 25.4kHz. The friction level at this point is as follows: Figure 5 As shown.

[0110] The implementation method of step (5) is as follows: A square pawl of approximately 4mm is designed to provide tactile braking effect at the start of sliding and when the target is about to be reached, providing prompts for user operation. Specifically, the applied signal is an electrostatic square wave drive signal with a voltage amplitude of 150Vpp and a frequency of 20Hz. The position of the square pawl is as follows: Figure 6 As shown, these correspond to 802 in Figure 9(a) and 804 in Figure 9(b), respectively.

[0111] like Figure 7 As shown, the implementation method of step (6) is as follows:

[0112] (1) Divide the sliding track of the virtual sliding button to be simulated into two parts according to the distance from the target point: a low-friction region and a high-friction region. The maximum displacement of each region is as follows: and ;

[0113] (2) Establish displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage and velocity-electrostatic force driving signal frequency mapping models in different regions to realize the effect of tactile feedback changing with the movement displacement and velocity.

[0114] 1) Ultrasonic vibration tactile reproduction technology

[0115] Under the influence of a driving signal, the piezoelectric ceramic drives the touchscreen to vibrate at high frequency, forming an air film between the finger and the touchscreen. This reduces the actual contact area, thereby decreasing the friction between the two. When the finger contacts the touchscreen, the air film creates a squeezing force between them.

[0116]

[0117]

[0118]

[0119] in, For extrusion pressure, The offset ratio determines the magnitude of the extrusion pressure, and its range is... S represents the contact area between the finger and the touchscreen. Atmospheric pressure, The average amplitude of the fingerprint. For the surface roughness of the touchscreen, The vibration amplitude of the touchscreen is measured when the piezoelectric ceramic is operating at its resonant frequency.

[0120]

[0121] in, For the mechanical quality factor of the piezoelectric ceramic sheet, Where is the piezoelectric constant, and V is the driving signal voltage. For the resonator thickness, The length of the piezoelectric ceramic sheet, , The length of the plate under compression and stretching along the Z-axis. Extrusion force. The presence of this reduces the normal force on the fingers, resulting in tangential friction:

[0122]

[0123] in, This refers to the tangential frictional force between the finger and the screen. Apply pressure to the fingers, The coefficient of friction, Given the amplitude of the ultrasonic vibration driving signal voltage, it can be deduced from the formula that the larger the amplitude of the driving signal voltage, the greater the vibration amplitude of the touchscreen. The larger the offset ratio The larger size leads to increased squeezing force, which in turn reduces the tangential friction between the finger and the touchscreen.

[0124] The mapping relationship between tactile roughness and ultrasonic vibration driving signal frequency was obtained through subjective perception scoring experiments. A velocity-ultrasonic vibration driving signal frequency mapping model was then established. The scoring standard was: "the stronger the perceived roughness, the higher the subjective score; the weaker the perceived roughness, the lower the subjective score." The fitting relationship was obtained through experiments.

[0125]

[0126] in, The frequency of the ultrasonic vibration driving signal. For tactile perception of roughness;

[0127] 2) Electrostatic tactile reproduction technology

[0128] When a driving signal is applied to the transparent conductive layer of the touchscreen, an electrostatic attraction is generated between the finger and the touchscreen due to the difference in the polarity of the charge induction:

[0129]

[0130] in, For electrostatic attraction, The free space permittivity, and Here, A represents the relative permittivity of the skin and the insulating film, respectively, and A is the contact area between the finger and the touchscreen. and These are the thicknesses of the stratum corneum and the insulating layer of the skin, respectively. To drive the signal voltage, the electrostatic attraction can be seen from the formula. With drive signal voltage The tangential friction force generated at this time is directly proportional to the square of the number of tangential friction forces.

[0131]

[0132] in, For tangential friction, The coefficient of sliding friction between the finger and the touchscreen. Apply pressure to the fingers, The amplitude of the electrostatic driving signal voltage is determined by the electrostatic force. The larger the driving voltage amplitude, the greater the electrostatic attraction, which in turn increases the tangential friction between the finger and the touch screen.

[0133] A tactile roughness-electrostatic force driven signal frequency mapping model was established in the same manner, and the fitting relationship was obtained through experiments:

[0134]

[0135] in, The frequency of the signal is driven by electrostatic force. For tactile perception of roughness.

[0136] 3) Establish a displacement-driving signal voltage mapping model

[0137] The maximum frictional force occurs when the finger's displacement from the starting point is 0. The finger's maximum displacement is from the starting point. At that time, corresponding to the minimum friction force The displacement of the finger from the starting point is mapped proportionally to the frictional force experienced by the finger during the movement:

[0138]

[0139] in, This represents the displacement of the finger from the starting point. displacement The friction force experienced by the fingers The fitting coefficients for the displacement-friction mapping relationship are... For the intercept, compare it with... By combining these, the mapping relationship between displacement and ultrasonic vibration driving signal voltage amplitude can be obtained:

[0140]

[0141] in, The fitting coefficients for the displacement-friction mapping relationship in ultrasonic vibration tactile reproduction technology are given. The intercept represents the amplitude of the ultrasonic vibration driving signal voltage. This means that the greater the displacement of the finger from the starting point, the smaller the tangential frictional force on the finger (i.e., the greater the squeezing force), and consequently, the greater the amplitude of the ultrasonic vibration driving signal voltage. Similarly, the mapping relationship between displacement and electrostatic force driving signal voltage amplitude is as follows:

[0142]

[0143] in, The fitting coefficients for the displacement-friction mapping relationship in electrostatic tactile reproduction technology are given. The intercept is... The displacement of the finger from the target point is the smaller the tangential friction force on the finger, i.e., the smaller the electrostatic attraction force, and the smaller the amplitude of the electrostatic force driving signal voltage.

[0144] 4) Establish a speed-drive signal frequency mapping model

[0145] Based on the speed at which a person swipes the touchscreen under different conditions, a movement speed range of 0-100 mm / s is selected. As the movement speed increases, the tactile roughness is enhanced, that is:

[0146]

[0147] in, The speed of finger movement. In order to speed Tactile roughness at that time To perceive the maximum roughness, The fitting coefficients for the speed-tactile perception roughness mapping relationship are then compared with... Combining these, we can obtain the mapping relationship between velocity and the frequency of the ultrasonic vibration driving signal:

[0148]

[0149] in, These are the fitting coefficients for the velocity-tactile roughness mapping relationship in ultrasonic vibration tactile reproduction technology. That is, the greater the finger movement speed, the stronger the tactile roughness, and the lower the frequency of the ultrasonic vibration driving signal. Similarly, the velocity-electrostatic force driving signal frequency mapping relationship is as follows:

[0150]

[0151] in, The fitting coefficients for the velocity-tactile roughness mapping relationship in electrostatic force reproduction technology;

[0152] (3) Ultrasonic vibration tactile reproduction technology is used in low-friction areas. The specific applied signal is: a sine wave with a carrier frequency of 25.4 kHz; based on the mapping relationship between displacement and ultrasonic vibration driving signal voltage. As the displacement distance between the finger and the starting point increases, the driving signal voltage gradually increases between 30-120Vpp, causing the frictional resistance experienced by the finger to gradually decrease; based on the mapping relationship between velocity and ultrasonic vibration driving signal frequency... As the finger sliding speed gradually increases, the frequency of the amplitude modulation signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness.

[0153] (4) Electrostatic tactile reproduction technology is used in high-friction areas. The specific signal applied is a sine wave; based on the mapping relationship between displacement and electrostatic driving signal voltage. As the distance between the finger and the target point gradually decreases, the driving signal voltage gradually increases between 40-150Vpp, thus increasing the frictional resistance experienced by the finger. This is based on the mapping relationship between speed and electrostatic force driving signal frequency. As the finger slides faster, the frequency of the adjustment drive signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness. The drive signal is turned off the moment the finger is about to reach the target position.

[0154] Finally, based on the different driving signals provided during various finger gestures on the touchscreen as described above, tactile feedback is generated at the interaction location, thus realizing the tactile perception process of the virtual sliding button. That is, pressing the button produces a click sensation, and controlling the coefficient of friction during slider dragging provides tactile feedback, thereby guiding and confirming the interactive operation and enhancing the interactive experience.

[0155] The invention will be further illustrated below with specific examples:

[0156] In recent years, many application systems have begun using CAPTCHA technology to prevent access by bots. Slider CAPTCHAs are widely used due to their ease of use and relatively strong security. However, slider CAPTCHAs require dragging and matching images to the correct positions to complete the verification, relying heavily on visual cues, and their accuracy has a high degree of randomness. For example... Figure 9(a) , 9(b) As shown in 9(c), the present invention is further illustrated by rendering the virtual sliding button verification code on a smart terminal using tactile reproduction.

[0157] The tactile presentation unit 11 is an electrostatic touchscreen with piezoelectric ceramics symmetrically bonded on both sides. Under the action of a driving signal, the piezoelectric ceramics drive the touchscreen to vibrate at high frequency, forming an air film, thereby reducing the tangential friction between the finger and the touchscreen. When a driving signal is applied to the conductive layer of the electrostatic touchscreen, the finger generates a changing electrostatic attraction due to the different polarities of the charge induction, thereby increasing the tangential friction between the finger and the touchscreen. This achieves bidirectional adjustment of the tangential friction, which can both increase and decrease, making the tactile sensation richer and more realistic. Tactile feedback is added when the smart terminal interacts with the virtual sliding button verification code, as shown in Figures 9(a), 9(b), and 9(c).

[0158] When a finger touches the haptic presentation unit 11, the finger positioning unit 13 uses the smart terminal's own touch positioning technology to detect the finger position and feeds it back to the haptic processing unit 14. The unit then detects whether a virtual sliding button has been captured and calculates the finger's displacement and speed. When the virtual slider is touched, the force sensor obtains the pressure applied by the finger in real time. With the set pressure threshold The system compares the values ​​of the position coordinates within 0.5 seconds from the start of the touch to the contact point, and determines that the interaction gesture is a touch click. At the instant the preset conditions are met, the haptic drive unit 12 synthesizes a square wave, 120V, 25.4kHz ultrasonic vibration drive signal, and reduces the simulated click sensation through instantaneous friction.

[0159] At positions 802 in Figure 9(a) and 804 in Figure 9(b), a square wave, 150V, 20Hz electrostatic drive signal is applied to increase tangential friction. This high-friction pawl generates tactile braking to prompt the user's operation, preventing them from losing the target or failing to reach the target position during the swipe, thus reducing visual burden. During the dragging process, as the distance from the starting point increases, the ultrasonic vibration drive signal voltage amplitude is gradually increased within the range of 30-120Vpp to reduce frictional resistance and shorten the interaction time. When the image begins to enter the target area, the electrostatic drive signal voltage amplitude is gradually increased within the range of 40-150Vpp to increase frictional resistance, preventing the user from missing the target position. Throughout the overall swipe, the drive signal frequency is changed within the range of 0-1000Hz according to the movement speed at different times; a higher frequency is used for slower speeds, and a lower frequency is used for faster speeds. By sensing changes in roughness, the finger movement speed is controlled to be moderate, improving interaction accuracy. The final friction pattern generated by the overall applied drive signal is as follows: Figure 8 As shown.

[0160] Using the above method, tactile reproduction of a virtual sliding button on a smart terminal is achieved through variable friction tactile feedback based on any one or a combination of ultrasonic vibration and electrostatic tactile reproduction technologies. By real-time detection of finger pressure, displacement, and sliding speed, and adjusting the drive signal parameters to control the tactile feedback effect, the realism and immersion of the interaction are increased, and the interaction efficiency and accuracy are improved. The invention has been described in detail with examples, but its specific implementation is not limited thereto. Any person skilled in the art can make various obvious changes and modifications without departing from the spirit and principles of the method described herein.

Claims

1. A method for rendering the haptic feedback of a virtual sliding button on a smart terminal, characterized in that, Includes the following steps: (1) Obtain the coordinates of the user's finger position on the touch screen when the user performs an interactive operation; (2) Obtain the displacement and speed of the user's finger at different times during the interactive operation; (3) Determine whether the user's gesture during the interaction is a touch click or a drag; (4) Simulate tactile feedback similar to clicking when the user touches and presses the virtual sliding button; (5) Increase tactile braking effect with tactile pawls at the start of sliding and when approaching the target; (6) During the sliding process, displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage, and velocity-electrostatic force driving signal frequency mapping models are established in different regions respectively. The finger displacement distance and movement speed are used as inputs, and the driving signal voltage and frequency are used as outputs. The friction coefficient during sliding is controlled in real time according to the magnitude of displacement and movement speed to generate tactile feedback and realize the tactile reproduction rendering of the virtual sliding button; the specific implementation method is as follows: (1) Divide the sliding track of the virtual sliding button to be simulated into two parts according to the distance from the target point: a low-friction region and a high-friction region. The maximum displacement of each region is as follows: and ; (2) Establish displacement-ultrasonic vibration driving signal voltage, velocity-ultrasonic vibration driving signal frequency, displacement-electrostatic force driving signal voltage and velocity-electrostatic force driving signal frequency mapping models in different regions to realize the effect of tactile feedback changing with the movement displacement and velocity. 1) Ultrasonic vibration tactile reproduction technology; Under the influence of a driving signal, the piezoelectric ceramic drives the touchscreen to vibrate at high frequency, forming an air film between the finger and the touchscreen. This reduces the actual contact area, thereby decreasing the friction between the two. When the finger contacts the touchscreen, the air film creates a squeezing force between them. ； ； ； in, For extrusion pressure, The offset ratio determines the magnitude of the extrusion pressure, and its range is... S represents the contact area between the finger and the touchscreen. Atmospheric pressure, The average amplitude of the fingerprint. For the surface roughness of the touchscreen, The vibration amplitude of the touchscreen is measured when the piezoelectric ceramic is operating at its resonant frequency. ； in, For the mechanical quality factor of the piezoelectric ceramic sheet, Where is the piezoelectric constant, and V is the driving signal voltage. For the resonator thickness, The length of the piezoelectric ceramic sheet, , The compressive force is the length of the plate under compression and stretching along the Z-axis. The presence of this reduces the normal force on the fingers, resulting in tangential friction: ； in, This refers to the tangential frictional force between the finger and the screen. Apply pressure to the fingers, The coefficient of friction, Given the amplitude of the ultrasonic vibration driving signal voltage, it can be deduced from the formula that the larger the amplitude of the driving signal voltage, the greater the vibration amplitude of the touchscreen. The larger the offset ratio The larger size leads to increased squeezing force, which in turn reduces the tangential friction between the finger and the touchscreen. The mapping relationship between tactile roughness and ultrasonic vibration driving signal frequency was obtained through subjective perception scoring experiments. A velocity-ultrasonic vibration driving signal frequency mapping model was then established. The scoring standard was: "the stronger the perceived roughness, the higher the subjective score; the weaker the perceived roughness, the lower the subjective score." The fitting relationship was obtained through experiments. ； in, The frequency of the ultrasonic vibration driving signal. For tactile perception of roughness; 2) Electrostatic tactile reproduction technology; When a driving signal is applied to the transparent conductive layer of the touchscreen, an electrostatic attraction is generated between the finger and the touchscreen due to the difference in the polarity of the charge induction: ； in, For electrostatic attraction, The free space permittivity, and Here, A represents the relative permittivity of the skin and the insulating film, respectively, and A is the contact area between the finger and the touchscreen. and These are the thicknesses of the stratum corneum and the insulating layer of the skin, respectively. To drive the signal voltage, the electrostatic attraction can be seen from the formula. With drive signal voltage The tangential friction force generated at this time is directly proportional to the square of the number of tangential friction forces. ； in, For tangential friction, The coefficient of sliding friction between the finger and the touchscreen. Apply pressure to the fingers, The amplitude of the electrostatic driving signal voltage is determined by the electrostatic force. The larger the driving voltage amplitude, the greater the electrostatic attraction, which in turn increases the tangential friction between the finger and the touch screen. A tactile roughness-electrostatic force driven signal frequency mapping model was established in the same manner, and the fitting relationship was obtained through experiments: ； in, The frequency of the signal is driven by electrostatic force. For tactile perception of roughness; 3) Establish a displacement-drive signal voltage mapping model; The maximum frictional force occurs when the finger's displacement from the starting point is 0. The finger's maximum displacement is from the starting point. At that time, it corresponds to the minimum friction force The displacement of the finger from the starting point is mapped proportionally to the frictional force experienced by the finger during the movement: ； in, This represents the displacement of the finger from the starting point. displacement The friction force experienced by the fingers The fitting coefficients for the displacement-friction mapping relationship are... For the intercept, compare it with... By combining these, the mapping relationship between displacement and ultrasonic vibration driving signal voltage amplitude can be obtained: ； in, The fitting coefficients for the displacement-friction mapping relationship in ultrasonic vibration tactile reproduction technology are given. The intercept represents the amplitude of the ultrasonic vibration driving signal voltage. This means that the greater the displacement of the finger from the starting point, the smaller the tangential frictional force on the finger (i.e., the greater the squeezing force), and consequently, the greater the amplitude of the ultrasonic vibration driving signal voltage. Similarly, the mapping relationship between displacement and electrostatic force driving signal voltage amplitude is as follows: ； in, The fitting coefficients for the displacement-friction mapping relationship in electrostatic tactile reproduction technology are given. The intercept is... The displacement of the finger from the target point is the smaller the tangential friction force on the finger, i.e., the smaller the electrostatic attraction force, and the smaller the amplitude of the electrostatic force driving signal voltage. 4) Establish a speed-drive signal frequency mapping model; Based on the speed at which a person swipes the touchscreen under different conditions, a movement speed range of 0-100 mm / s is selected. As the movement speed increases, the tactile roughness is enhanced, that is: ； in, The speed of finger movement. In order to speed Tactile roughness at that time To perceive the maximum roughness, The fitting coefficients for the speed-tactile perception roughness mapping relationship are then compared with... Combining these, we can obtain the mapping relationship between velocity and the frequency of the ultrasonic vibration driving signal: ； in, These are the fitting coefficients for the velocity-tactile roughness mapping relationship in ultrasonic vibration tactile reproduction technology. That is, the greater the finger movement speed, the stronger the tactile roughness, and the lower the frequency of the ultrasonic vibration driving signal. Similarly, the velocity-electrostatic force driving signal frequency mapping relationship is as follows: ； in, The fitting coefficients for the velocity-tactile roughness mapping relationship in electrostatic tactile reproduction technology; (3) Ultrasonic vibration tactile reproduction technology is used in low-friction areas. The specific applied signal is: a sine wave with a carrier frequency of 25.4 kHz; based on the mapping relationship between displacement and ultrasonic vibration driving signal voltage. As the displacement distance between the finger and the starting point increases, the driving signal voltage gradually increases between 30-120Vpp, causing the frictional resistance experienced by the finger to gradually decrease; based on the mapping relationship between velocity and ultrasonic vibration driving signal frequency... As the finger sliding speed gradually increases, the frequency of the amplitude modulation signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness. (4) Electrostatic tactile reproduction technology is used in high-friction areas. The specific signal applied is a sine wave; based on the mapping relationship between displacement and electrostatic driving signal voltage. As the distance between the finger and the target point gradually decreases, the driving signal voltage gradually increases between 40-150Vpp, thus increasing the frictional resistance experienced by the finger. This is based on the mapping relationship between speed and electrostatic force driving signal frequency. As the finger slides faster, the frequency of the adjustment drive signal gradually decreases between 0-1000Hz. The faster the sliding speed, the lower the frequency, and the more obvious the roughness. The drive signal is turned off the moment the finger is about to reach the target position.

2. The haptic rendering method for virtual sliding buttons in smart terminals according to claim 1, characterized in that, The virtual sliding button is used in the slider CAPTCHA to verify user registration and login, and to verify secure payment. The slider CAPTCHA is drawn and run on the Unity2017.2.4f1 platform.

3. The haptic rendering method for virtual sliding buttons in smart terminals according to claim 1, characterized in that, The implementation method of step (1) is as follows: obtaining the position of the user's finger on the touch screen when interacting includes real-time tracking and measurement of the finger position information by using a touch display technology that combines the haptic reproduction screen and the touch positioning screen into one.

4. The haptic rendering method for virtual sliding buttons in smart terminals according to claim 1, characterized in that, The implementation method of step (2) is as follows: Establish a two-dimensional coordinate system with the location of the virtual sliding button as the origin. The X-axis represents the horizontal direction, and the Y-axis represents the vertical direction. Calculate the distance x or y between the finger and the origin at different times, which is the displacement at the current time. Calculate the time taken for the finger to slide horizontally or vertically from the previous position to the current position. The horizontal or vertical sliding distance is or Then the current speed of the finger movement is or .

5. The method for rendering the haptic feedback of a virtual sliding button in a smart terminal according to claim 1, characterized in that, The implementation method of step (3) is as follows: calculate the position coordinate difference within 1 second when the finger starts to touch the virtual sliding button. If the difference is equal to 0, it is a touch click; otherwise, it is a drag.

6. The haptic rendering method for virtual sliding buttons in smart terminals according to claim 1, characterized in that, The implementation method of step (4) is as follows: 1) Obtain the pressure applied when the user presses the virtual slider button. This includes installing a force sensor on the terminal housing to measure and record force in real time; 2) Set the pressure threshold = 0.5N, when the user's interaction with the touchscreen is judged as a static gesture touch click and At that moment, an ultrasonic vibration drive signal is immediately applied, and the click sensation is simulated by the instantaneous change in friction and the increase in perceived roughness.

7. The haptic rendering method for virtual sliding buttons in smart terminals according to claim 1, characterized in that, The implementation method of step (5) is as follows: a square pawl is designed to provide tactile braking effect when the sliding begins and when the target is about to be reached, and to prompt the user's operation.

8. A smart terminal virtual sliding button haptic reproduction rendering device for implementing the method as described in any one of claims 1 to 7, characterized in that, include: (1) Tactile presentation unit: including electrostatic touch screen and piezoelectric ceramic sheet, presenting human-computer interaction interface, receiving drive signals transmitted by tactile driving unit, and providing tangential force tactile feedback on touch screen; (2) Tactile driving unit: including signal generator, digital-to-analog converter, amplifier and power supply, used to synthesize the driving signals required for electrostatic force and ultrasonic vibration, and transmit them to the tactile presentation unit to drive and generate tactile sensation at the corresponding position; (3) Finger positioning unit: includes a device with positioning technology, which can acquire the position coordinate information of the finger when it moves in real time and transmit it to the tactile processing unit for processing and calculation; (4) Tactile processing unit: includes various smart terminals with touch screens, used to output visual and auditory information, can receive finger position information transmitted from the finger positioning unit and process and calculate, and at the same time calculate the parameters of the required driving signal according to the mapping model and transmit them to the tactile driving unit.

Images