Haptic feedback method and apparatus, and display device

Abstract

The present application relates to the technical field of touch control, and particularly relates to a haptic feedback method and apparatus, and a display device. In the haptic feedback method, a plurality of actuators are provided. When a certain touch point is required to provide vibration feedback, a vibration waveform required for the vibration feedback of the touch point is used as a desired value, and drive waveforms of the actuators are reversely deduced in view of a time-domain impact response of each actuator, thereby activating all the actuators, such that the waveforms of the actuators are superimposed so as to provide vibration feedback from a target touch point, and vibration waveforms of touch points other than the target touch point are cancelled out as much as possible, thereby providing a better haptic feedback effect for a user.

Description

Haptic feedback method, device and display device TECHNICAL FIELD

[0001] The present application relates to the technical field of touch control, in particular to a haptic feedback method, device and display device. BACKGROUND

[0002] Touch input is an important human-computer interaction mode between electronic devices and users, and its implementation is usually through a touch-sensitive surface on which sensors are deployed, so that the electronic device can obtain the touch position and force (touch pressure) of the user, and on this basis, providing haptic type feedback is a better way to improve user interaction experience. The existing haptic feedback mode provides low-frequency (below 1000Hz) vibration to the user through the touch-sensitive surface, which can be provided by devices such as linear motors, voice coil motors or piezoelectric ceramics. However, most of the existing haptic feedback modes rely on the whole screen or the whole module to vibrate, so the vibration feedback effect provided by each position on the whole touch surface to the user is the same, and different haptic feedback effects cannot be provided to different positions of the finger when the user touches the electronic device.

[0003] SUMMARY

[0004] The embodiments of the present application mainly solve the technical problem that the existing touch screen cannot provide regional haptic feedback to the user's finger.

[0005] To solve the above technical problem, in a first aspect, the embodiments of the present application provide a haptic feedback method applied to a display device, the display device comprising a touch surface, a plurality of touch points corresponding to touch operations on the touch surface, and an actuator arranged at the edge of the touch surface, the actuator being used to provide vibration feedback at the touch points, the method comprising:

[0006] obtaining the touch position and pressing pressure of the touch surface, and determining the position of the target touch point that needs to provide vibration feedback according to the touch position and the pressing pressure, wherein the vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold; constructing a system matrix according to the position of the target touch point; defining a desired vibration waveform according to the vibration feedback of the target touch point; calculating the driving waveform of the actuator according to the desired vibration waveform in combination with the system matrix; and providing a driving signal to the actuator based on the driving waveform, so that the actuator provides vibration feedback of the target touch point.

[0007] The scheme sets multiple actuators, when a certain touch point needs to provide vibration feedback, takes the vibration waveform required by the vibration feedback of the touch point as an expected value, combines the time domain impact response of each actuator, reverses the driving waveform of the actuators, and activates all the actuators, so that the waveforms of the actuators are superimposed to provide vibration feedback of the target touch point, and as much as possible to offset the vibration waveforms of the remaining touch points except the target touch point, only the local vibration feedback is reserved to provide better haptic feedback effect for the user.

[0008] In some embodiments, the number of actuators is greater than or equal to the number of touch points. The scheme provides more accurate vibration feedback for touch points by setting relatively more actuators.

[0009] In some embodiments, the positions of the touch points are recorded as T j Wherein j represents the jth touch point, and the expected vibration waveform defined according to the vibration feedback of the target touch point comprises: determining the target touch point as the Lth touch point according to the position of the target touch point; setting the expected vibration waveform Y:

[0010] Wherein y j represents the expected vibration waveform corresponding to the jth touch point. The scheme sets a model of the expected vibration waveform, that is, under the vibration superposition of all actuators, the target touch point is short pulse vibration, and the remaining touch points do not vibrate, thereby providing relatively accurate vibration feedback for the user.

[0011] In some embodiments, the combination of the system matrix and the calculation of the driving waveform of the actuator according to the expected vibration waveform comprises: calculating the driving waveform according to the following formula: Y = ΦS, wherein Y is the expected vibration waveform of the touch point, φ is the system matrix constructed according to the target touch point, and S is the driving waveform of the actuator. After defining the expected vibration waveform according to the position of the target touch point, the system matrix can be constructed according to the position of the target touch point, and the equation is solved to obtain the driving waveform of each actuator.

[0012] In some embodiments, the calculating the driving waveform of the actuator according to the expected vibration waveform in combination with the system matrix further comprises: setting an error range for the expected vibration waveform, so that, within the error range, the driving waveform of the actuator controls the vibration amplitude of the target touch point to be greater than a first preset threshold, and the vibration amplitude of the remaining touch points except the target touch point to be less than a second preset threshold, wherein the second preset threshold is less than the first preset threshold. Since the expected vibration waveform in the above equation is completely set according to the vibration feedback of the target touch point, the computing power requirement of the computing module is high, which may cause high feedback delay, or the calculated driving waveform is small, resulting in that the vibration feedback of the target touch point is not obvious, affecting the user experience, therefore, the error range is set for the expected vibration waveform, so that the vibration waveform of the remaining touch points is not necessarily not vibrating, so that, under the premise that the vibration feedback of the target touch point is obvious enough, the remaining touch points are allowed to have slight vibration feedback, and the user experience is not affected.

[0013] In some embodiments, the setting the allowed error range for the expected vibration waveform comprises: setting a constraint condition for the system input with the minimization of the energy of the system input as an optimization target:

[0014] argmin S T S,s.t.||y-y0|| 2 <ε

[0015] Wherein, the argmin function represents that, under the condition that ||y-y0||<ε is met, S 2 <ε T S is the value of S when S is the minimum value, s.t. represents the constraint condition; S T represents the transpose of the driving waveform of the actuator, S T S represents the energy of the system input, y represents the driving waveform of the actuator, y0 represents the expected vibration waveform, and ε represents the allowed error between the vibration waveform of the touch point and the expected vibration waveform under the constraint condition; the calculation of the driving waveform S of the actuator is realized based on the constraint condition. The constraint condition is set for the system input, and a tolerance is set for the remaining touch points except the target touch point, so that the vibration feedback is below the perception threshold of the finger haptics, that is, the vibration waveform here can not be strictly zero, but the generated vibration is also very weak, thereby achieving the effect that the vibration of the target touch point is obvious and the remaining touch points are relatively static, and the feedback experience of the user is ensured.

[0016] In some embodiments, the calculating the driving waveform of the actuator according to the expected vibration waveform in combination with the system matrix further comprises: realizing the calculation of the driving waveform S by the following formula:

[0017] S=(φT φ+βI) -1 φ T Y; wherein, φ T denotes the transpose of the system matrix φ, I is the identity matrix, and β is a regularization parameter. The introduction of the regularization term makes the solution process more stable. By adjusting the regularization parameter, a relatively more balanced solution between fitting ability and generalization ability can be obtained, improving the flexibility of the system.

[0018] In some embodiments, constructing the system matrix according to the position of the target touch point comprises: combining the system matrix φ with a system transfer function according to the position of the target touch point. The system matrix is constructed by combining the position of the target touch point with the system transfer function to establish a mathematical relationship between the driving waveform and the desired vibration waveform. The system transfer function is basically related to the structure and parameters of the actuator array, and then the control waveform of the actuator is solved according to the desired vibration waveform.

[0019] In some embodiments, the method further comprises: obtaining a pressing pressure change curve; obtaining a change trend of the touch area of each touch point; when the pressing pressure exceeds the pressure threshold and the touch area exceeds the area threshold, determining the touch point at the current touch position as the target touch point. The target touch point and its position are determined by the pressure and touch area, and by setting the detection threshold, which can accurately identify the user's finger pressing action on the screen.

[0020] In some embodiments, the method further comprises: triggering an excitation waveform when the actuator provides vibration feedback for the target touch point, the excitation waveform being used to represent the vibration of the user's pressing action; and triggering the excitation waveform when the pressing pressure of the target touch point is higher than the pressure threshold during the downward pressing phase. In some scenarios, the vibration feedback effect of the display device needs to be detected. The present scheme can cooperate with the vibration feedback display excitation waveform to show the vibration effect of the user's pressing action, thereby improving the user's interest in interaction.

[0021] In some embodiments, the method further comprises: triggering the excitation waveform when the pressing pressure of the target touch point is lower than the pressure threshold during the upward lifting phase. The present scheme combines the detection result of the pressing pressure, and also uses the excitation waveform to show the position of the user leaving the target touch point when the user stops the touch operation, thereby intuitively displaying the haptic feedback effect.

[0022] In a second aspect, the embodiments of the present application also provide a haptic feedback device applied to a display device, the display device comprising a touch surface, a plurality of touch points corresponding to touch operations on the touch surface, and a plurality of actuators arranged at the edge of the touch surface and used to provide vibration feedback at the touch points, the number of the actuators being greater than or equal to the number of the touch points, the device comprising: a touch detection module configured to acquire touch positions and pressing forces of the touch surface, and determine a position of a target touch point requiring vibration feedback according to the touch positions and the pressing forces, wherein the vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold; a microcontroller configured to construct a system matrix according to the position of the target touch point, define a desired vibration waveform according to the vibration feedback of the target touch point, and calculate a driving waveform of the actuators according to the desired vibration waveform in combination with the system matrix; and a vibration driving module configured to provide a driving signal to the actuators based on the driving waveform, so that the actuators provide the vibration feedback of the target touch point.

[0023] In a third aspect, the embodiments of the present application also provide a display device comprising a microcontroller, and a touch unit, a pressure detection unit and a vibration driving unit electrically connected to the microcontroller, the touch unit comprising a thin plate providing a touch surface, the pressure detection unit being arranged correspondingly to the touch surface, and the vibration driving unit being connected to a plurality of actuators arranged at the edge of the touch surface; the touch unit is configured to acquire touch positions and provide the touch positions to the microcontroller; the pressure detection unit is configured to detect pressing forces of the touch surface and provide the pressing forces to the microcontroller; the microcontroller is configured to determine a position of a target touch point requiring vibration feedback according to the touch positions and the pressing forces, construct a system matrix according to the position of the target touch point, define a desired vibration waveform according to the vibration feedback of the target touch point, calculate a driving waveform of the actuators according to the desired vibration waveform in combination with the system matrix, and provide the driving waveform to the vibration driving unit; and the vibration driving unit is configured to provide a driving signal to the actuators based on the driving waveform, so that the actuators provide the vibration feedback of the target touch point.

[0024] In some embodiments, the actuators are piezoelectric ceramics. The present application utilizes the high-speed response and small size characteristics of piezoelectric ceramics, uses them as actuators of the display device, can realize precise vibration control, and at the same time, utilizes the high energy density to realize high-efficiency energy conversion, indirectly improves the performance of the device.

[0025] In some embodiments, the vibration driving unit is further configured to: amplify the driving waveform in power, and provide a driving signal to the actuator based on the amplified driving waveform, so that the actuator provides the vibration feedback of the target touch point. This solution adjusts the driving waveform by amplifying the power, accurately controls the degree of power amplification and the waveform shape, finely adjusts the output of the actuator, and reduces errors.

[0026] In a fourth aspect, the embodiments of the present application further provide a display device, including: at least one processor and a memory; the memory is coupled to the processor, and the memory is configured to store instructions or programs, when the instructions or programs are executed by the at least one processor, the at least one processor executes the haptic feedback method as described above.

[0027] In a fifth aspect, the embodiments of the present application further provide a non-volatile computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are executed by a display device, the display device executes the method as described above.

[0028] In a sixth aspect, the embodiments of the present application further provide a computer program product, the computer program product includes a computer program stored on a non-volatile computer readable storage medium, the computer program includes program instructions, when the program instructions are executed by a display device, the display device executes the method as described above.

[0029] The haptic feedback method, device and display device provided by the embodiments of the present application, by setting multiple actuators, when a certain touch point needs to provide vibration feedback, taking the vibration waveform required by the vibration feedback of the touch point as an expected value, combining the time domain impulse response of each actuator, back-propagating the driving waveform of these actuators, and activating all the actuators, so that the waveforms of these actuators are superimposed to provide the vibration feedback of the target touch point, and as much as possible to offset the vibration waveforms of the touch points other than the target touch point, to provide better haptic feedback effect for the user. BRIEF DESCRIPTION OF DRAWINGS

[0030] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, which are schematic and not intended to be limiting of the embodiments, and in which like reference numerals refer to like elements in the various figures of the drawings in which: the figures are not to scale.

[0031] FIG. 1 is a structural schematic diagram of a display device according to an embodiment of the present application;

[0032] FIG. 2 is a position schematic diagram of an actuator in a display device according to an embodiment of the present application;

[0033] FIG. 3 is a flow diagram of a method of haptic feedback according to an embodiment of the present application;

[0034] FIG. 4 is an example diagram of vibration feedback of a display device according to an embodiment of the present application;

[0035] FIG. 5 is another example diagram of vibration feedback of a display device according to an embodiment of the present application;

[0036] FIG. 6 is a structural diagram of a haptic feedback device according to an embodiment of the present application;

[0037] FIG. 7 is another structural diagram of a display device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. It should be noted that, if there is no conflict, each feature in the embodiments of the present application can be combined with each other, and all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device schematic diagram or the order in the flowchart.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

[0040] For a display device supporting touch interaction, providing haptic feedback effect is a design that can improve user experience, and most of the existing haptic feedback methods rely on the whole screen or the whole module to vibrate, so the vibration feedback effect provided by each position on the whole touch surface to the user is the same, and different haptic feedback effects cannot be provided for different positions of the fingers when the user touches the display device. In order to improve the user's interaction experience, it is tried to design to provide regional vibration feedback effect at the position touched by the user's different fingers. However, the following technical problems are found:

[0041] In related solutions, the actuator for providing vibration feedback is usually arranged directly below the touch surface, and the actuator near the position where vibration is desired is taken as the main working unit to generate local vibration effect. However, such design has the following defects: on the one hand, the actuator is installed directly below the touch surface, which easily affects the installation and normal use of the display module; on the other hand, the vibration area of a single actuator can only cover a very small range (for example, a range with a radius of 30 mm) around the actuator, and can usually only provide vibration feedback of certain area, and can only achieve vibration effect in the installation area of the actuator; if the solution is to achieve vibration in the entire touch surface area, a considerable number of actuators need to be installed below the touch surface, which will increase the cost and further affect the installation and use of the display module.

[0042] Therefore, in the related art, either only consistent vibration feedback of the entire screen or the entire module can be provided, or only the local area around the installation area of the actuator can achieve vibration effect under the condition of affecting the installation and normal display of the display module.

[0043] To this end, embodiments of the present application design and arrange multiple actuators at the edge of the touch surface. When vibration feedback of a certain touch point is needed, the vibration waveform required for vibration feedback of the touch point is taken as the expected value, the time-domain impulse response of each actuator is combined, the driving waveform of the actuators is back-calculated, and all actuators are activated based on the driving waveform, so that the vibration feedback of the target touch point is provided after the waveforms of the actuators are superimposed, and the vibration waveforms of the remaining touch points except the target touch point are as much as possible cancelled out. Since the actuators are all arranged in the edge area of the touch surface, better local haptic feedback effect can be provided for the user without affecting the installation of the display module.

[0044] Referring to FIG. 1, which is a structural schematic diagram of a display device according to an embodiment of the present application, as shown in the figure, the display device 100 includes a microcontroller 101, a touch unit 102, a pressure detection unit 103 and a vibration driving unit 104 electrically connected to the microcontroller 101, and a plurality of actuators 105 connected to the vibration driving unit 104, wherein the touch unit includes a thin plate (not shown in the figure) providing a touch surface. When a user interacts with the display device by touching the touch surface, the touch unit 102 can obtain the touch position and provide the touch position to the microcontroller 101, and at the same time, the pressure detection unit 103 can detect the pressing pressure of the touch surface and provide the pressing pressure to the microcontroller 101. Based on this, the microcontroller 101 can determine the position of the target touch point that needs to provide vibration feedback according to the received touch position and pressing pressure, wherein the vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold. Then a system matrix is constructed according to the position of the target touch point, and a desired vibration waveform is defined according to the vibration feedback of the target touch point, and the driving waveform of the actuator is calculated according to the desired vibration waveform in combination with the system matrix, and the driving waveform is provided to the vibration driving unit 104, and the vibration driving unit 104 can provide a driving signal to the actuator 105 based on the received driving waveform, so that the actuator 105 provides the vibration feedback of the target touch point.

[0045] In the embodiment of the present application, the pressure detection unit is arranged corresponding to the touch surface, which facilitates the detection of the pressure applied by the user to the touch surface, and a plurality of brakes are arranged at the edge of the touch surface. Specifically, in the embodiment of the present application, piezoelectric ceramics are used as actuators, which take advantage of the high-speed response and small size of piezoelectric ceramics to facilitate precise vibration control, and take advantage of the high energy density of piezoelectric ceramics to facilitate efficient energy conversion, thereby indirectly improving the performance of the device. FIG. 2 is a schematic diagram of the position of the actuator (piezoelectric ceramic) in the display device provided by the embodiment of the present application, and the range of the diagram is a thin plate that provides a touch surface in the display device. The touch surface is the part of the thin plate used for touch interaction with the user, that is, the touch and display area shown in the diagram. As shown in the diagram, the piezoelectric ceramic (actuator) is arranged at the edge of the touch surface in the embodiment of the present application, rather than in the range corresponding to the touch surface, so as to avoid affecting the installation and use effect of the display module. The thin plate belongs to a touch unit, and the touch surface therein is the main interactive operation surface of the user, which usually has a touch sensing function and can sense the touch position of the user's finger, and supports multi-point touch. The touch operation based on the touch surface corresponds to a plurality of touch points, which can be any position on the touch surface, and the actuators can provide vibration to the thin plate, specifically to provide vibration to a certain touch point or certain touch points. In the embodiment of the present application, the number of actuators is greater than or equal to the number of touch points. For example, assuming that there are 5*3=15 touch points arranged uniformly on a touch surface, 16 actuators can be arranged at positions shown in FIG. 2, for example, to achieve vibration control of the 15 touch points. It can be understood that piezoelectric ceramics are only a preferred example of actuators in the present solution, and the actuators are not limited to piezoelectric ceramics. In other embodiments, other devices can be selected as actuators to provide vibration feedback according to actual use scenarios, such as voice coil motors.

[0046] In some embodiments, after receiving the driving waveform provided by the microcontroller, the vibration driving unit can also adjust the driving waveform by power amplification, accurately control the degree of power amplification and the waveform shape, and provide a driving signal to the actuator based on the adjusted driving waveform, so that the actuator provides vibration feedback of the target touch point, thereby achieving fine control of the output of the actuator and reducing errors.

[0047] Please refer to FIG. 3, the embodiment of the present application provides a kind of haptic feedback method, applied in the display device in the above-mentioned embodiment, and the method comprises:

[0048] S11, acquire a touch position and a pressing pressure of a touch surface, and determine a position of a target touch point requiring to provide vibration feedback according to the touch position and the pressing pressure, wherein a vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold. The target touch point is a position corresponding to a contact of a finger of a user, a touch operation of the user is determined through the pressing pressure, and the target touch point requiring to provide vibration feedback and the position thereof are determined, and the position requires to provide vibration feedback with a vibration amplitude greater than the first preset threshold. The first preset threshold can be set according to a vibration perception threshold of the finger of the user, for example, the vibration amplitude corresponding to the first preset threshold is limited to be more easily perceived by the finger.

[0049] In some embodiments, the acquiring the touch position and the pressing pressure of the touch surface, and determining the position of the target touch point requiring to provide vibration feedback according to the touch position and the pressing pressure specifically includes: acquiring a pressing pressure change curve; acquiring a change trend of a touch area of each touch point; and determining a touch point at a current touch position as the target touch point when the pressing pressure exceeds a pressure threshold and the touch area exceeds an area threshold. Specifically, the display device can include a pressure detection module, and the pressure change curve of each part of the touch surface, including the pressure change curve of each touch point, is acquired in real time through the pressure detection module. When it is detected that the pressing pressure exceeds the pressure threshold, it can be considered that the touch operation of the user is detected. The display device can further include a touch positioning module, and the touch position can be acquired when the touch operation of the user is detected, and then the target touch point corresponding to the touch position and the position thereof are determined. Therefore, the method provided in the embodiments of the present application can accurately identify the action of the finger of the user pressing the screen and determine the target touch point and the position thereof.

[0050] S12, constructing a system matrix according to the position of the target touch point. In the embodiments of the present application, the system matrix is composed of the position of the target touch point and a system transfer function, denoted as φ. The system transfer function refers to a transfer function corresponding to an array / system composed of actuators. Generally, the system transfer function is mainly related to the structure and parameters of the array / system of the actuators. For a display device, the structure and parameters of the array of actuators are relatively fixed, and can be pre-calculated and stored in a database. When the system matrix needs to be constructed, the relevant data corresponding to the target touch point are directly called from the database to construct the system matrix, facilitating subsequent calculation steps.

[0051] S13, defining an expected vibration waveform according to the vibration feedback of the target touch point. In the embodiments of the present application, the position of the touch point is denoted as T jwherein j represents that the touch point is the jth touch point, for example, assuming that there are 15 touch points to be controlled, T1 represents the position of the first touch point, T2 represents the position of the second touch point, and so on. Specifically, the definition of the expected vibration waveform according to the vibration feedback of the target touch point comprises:

[0052] According to the position of the target touch point, the target touch point is determined to be the Lth touch point, and the position of the target touch point can be recorded as T j＝L ; and the expected vibration waveform Y is set according to the target touch point:

[0053] wherein y j represents the expected vibration waveform corresponding to the jth touch point. It can be seen that in the current step, the expected vibration waveform includes a short pulse at the position of the target touch point, and no vibration at the positions of the remaining touch points except the target touch point, and then the driving waveform of the actuator can be further calculated according to the defined expected vibration waveform.

[0054] S14, in combination with the system matrix, the driving waveform of the actuator is calculated according to the expected vibration waveform. In the embodiment of the application, the driving waveform of the actuator can be calculated according to the formula Y = ΦS, wherein Y is the expected vibration waveform of the touch point, φ is the system matrix constructed according to the target touch point, and S is the driving waveform of the actuator.

[0055] In the embodiment of the application, an example is used to explain this, assuming that the number of actuators (such as piezoelectric ceramics) is I, and there are J touch points on the touch surface, the time-domain impulse response of the ith actuator causing vibration at the jth touch point is recorded as h ji (t), based on which, if the driving signal s(t) is input to the actuator, the vibration at the touch point is y(t) = h ji (t) * s(t), where * represents convolution operation. The discrete form of the convolution process can be written as the following matrix expression:

[0056] y j = H ji s i

[0057] Based on this, the vibration waveforms of all touch points can be represented by the following matrix:

[0058] Y = ΦS

[0059] Therefore, according to the solution of the above formula Y = ΦS, the driving waveforms of the actuators can be obtained.

[0060] In some embodiments, considering that the process of solving S can be too strict for the vibration waveform, and thus the calculated driving waveform of the actuator is weak for the driving force of the actuator, the vibration amplitude of the actuator is small, that is, the provided vibration feedback is not obvious, and the user's tactile feedback experience is affected, therefore, in the embodiments of the present application, the combination of the system matrix and the calculation of the driving waveform of the actuator according to the expected vibration waveform further comprises: setting an error range for the expected vibration waveform, so that under the error range, the driving waveform of the actuator controls the vibration amplitude of the target touch point to be greater than a first preset threshold, and the vibration amplitude of the remaining touch points except the target touch point is less than a second preset threshold, wherein the second preset threshold is less than the first preset threshold.

[0061] It can be understood that the feeling of a finger to the screen vibration has a certain threshold, when the vibration amplitude is less than the threshold, the finger cannot feel it, therefore, the error range can be set for the expected vibration waveform, when the vibration amount caused by the vibration waveform of the touch point which does not need to provide vibration feedback is below the threshold of the finger feeling, that is, the vibration amplitude is less than the second preset threshold, the vibration is very weak, although the vibration is not completely zero, but it also will not affect the user to bring better local vibration feedback experience. The second preset threshold can be set according to the vibration feeling threshold of the user's finger, for example, limiting the vibration amplitude corresponding to the second preset threshold is not easy to be felt by the finger. Therefore, in the embodiments of the present application, the driving waveform is calculated based on solving the optimization problem with a relaxation constraint condition, which allows a certain vibration waveform error of the remaining touch points except the target touch point, greatly improving the vibration excitation efficiency. Specifically, the minimization of system input energy is taken as the optimization objective, and the constraint condition is set for the system input:

[0062] argmin S T S,s.t.||y-y0|| 2 <ε

[0063] Wherein, the argmin function represents the value of S when S is the minimum value under the condition that ||y-y0|| < ε, and s.t. represents the constraint condition; S 2 <ε T S represents the value of S when S is the minimum value under the condition that ||y-y0|| < ε, and s.t. represents the constraint condition; S T S represents the transpose of the driving waveform of the actuator, S T S represents the energy of the system input, y represents the actual vibration waveform of the touch point allowed in the error range, y0 represents the expected vibration waveform, and ε represents the allowed error between the vibration waveform of the touch point under the constraint condition and the expected vibration waveform, and the calculation of the driving waveform S of the actuator is realized based on the constraint condition. Through the mode provided in the embodiments of the present application, the problem of calculating the driving waveform of the actuator can be described as a second-order cone problem, and the interior point iteration method is used for solving, so as to obtain a suitable driving waveform of the actuator.

[0064] In another embodiment of the present application, based on the above formula Y = ΦS, the solution of the driving waveform S can also be driven by using Tikhonov regularization. Specifically, the calculating the driving waveform of the actuator according to the expected vibration waveform in combination with the system matrix further comprises: solving the driving waveform S by the following formula:

[0065] S = (φ T φ+βI) -1 φ T Y; wherein φ T is the transpose matrix of the system matrix φ, I is the unit matrix, and β is the regularization parameter. It should be noted that in this solution method, there is the following rule: the greater the value of β, the higher the driving efficiency of the system, and at the same time, the greater the deviation of the Y corresponding to the input result of the system from the target. Therefore, in the application process, the staff can comprehensively consider the related requirements of driving efficiency and deviation value in combination with the actual use scene, select the appropriate regularization parameter β for the current scene to calculate. This scheme utilizes the characteristics of Tikhonov regularization, and is more helpful for the system to achieve a more balanced solution between fitting ability and generalization ability when solving the driving waveform S of the actuator, reduces the risk of overfitting, has better generalization ability for unknown data, and improves the flexibility of the system.

[0066] S15, providing a driving signal to the actuator based on the driving waveform, so that the actuator provides vibration feedback of the target touch point. After obtaining the appropriate driving waveform S of the actuator by the above-mentioned manner, a driving signal is provided to all actuators accordingly, and the vibration of the actuator can control the target touch point to produce obvious vibration feeling, while the remaining touch points except the target touch point do not produce vibration feeling or produce weak vibration feeling which is almost imperceptible to the finger, thereby providing the user with local vibration and local static tactile feedback effect.

[0067] The embodiment of the present application takes the virtual keyboard scene with vibration feedback as an example. Please combine FIG. 4, in which the touch screen is taken as the display device, and the virtual keyboard is taken as the touch panel. It is assumed that the positions of the fingers in the figure are the positions of ten touch points numbered 1-10. When the right-hand index finger is pressed, the vibration of each touch point is as shown on the right side of FIG. 4. According to the haptic feedback method provided by the present application, it can be seen that 7 is the target touch point, and the vibration feedback at this position is very obvious, that is, the vibration amplitude at this position is larger, which can bring obvious vibration feedback to the user; the vibration feedback at the rest of the touch points (1-6 and 8-10) is weaker, although it is not completely static, but because the vibration amplitude at these positions is very small, it is not easy to be felt by the fingers, therefore, the touch screen as a whole can provide better haptic feedback experience to the user. In a specific application example, the display device described above can be a double-screen notebook, and the scheme of the present application can implement the virtual keyboard on one of the display screens, and in the case of using multiple fingers to touch the virtual keyboard at the same time, only when a single finger is pressed down, the haptic feedback is excited at the finger, and there is no haptic feedback at the rest of the fingers, thereby simulating the real touch of the physical keyboard.

[0068] In some embodiments, the method further comprises: triggering an excitation waveform for representing the vibration of the user pressing action when the actuator provides the vibration feedback of the target touch point; triggering the excitation waveform when the pressing pressure of the target touch point is higher than a pressure threshold value in the downward pressing phase; triggering the excitation waveform when the pressing pressure of the target touch point is lower than the pressure threshold value in the upward lifting phase. Please combine FIG. 5, the embodiment of the present application is explained by taking the moving object in the display screen as an example. Still taking the touch screen as the display device, specifically, as shown in FIG. 4, it is assumed that the positions of the fingers in the figure are the positions of five touch points numbered 1-5. The user's finger touches the action trajectory, and the feedback vibration is provided for the positions of the fingers in turn during the movement process of controlling or following the car. At time 1, 1 is the target touch point with obvious vibration feedback effect, and because the car moves along the path in the continuous time period, in order to simulate the haptic effect of the target touch point moving along the path, the vibration waveform size of different touch points on the path can be adjusted to improve the immersive haptic experience of the user. The haptic feedback effect can be intuitively displayed by the excitation waveform as shown on the right side of FIG. 5 during the process.

[0069] The haptic feedback method provided by the embodiment of the present application sets multiple actuators, when a certain touch point needs to provide vibration feedback, takes the vibration waveform required by the vibration feedback of the touch point as the expected value, reverses the driving waveform of each actuator in combination with the time-domain impact response of each actuator, and activates all the actuators, so that the waveforms of these actuators are superimposed to provide the vibration feedback of the target touch point, and as far as possible, the vibration waveforms of the rest of the touch points except the target touch point are offset, so as to provide better haptic feedback effect to the user.

[0070] The embodiment of the present application provides a kind of haptic feedback device, it is applied to display device, display device includes the thin plate of providing touch surface, and actuator is arranged at the edge of touch surface, actuator is used to provide vibration for thin plate, based on the touch operation of this touch surface corresponds multiple touch points, the number of actuator is greater than or equal to the number of touch points.Please combine Figure 6, haptic feedback device 200 includes: touch detection module 201, microcontroller 202 and vibration drive module 203.

[0071] Specifically, touch detection module 201 can obtain the touch position and pressing pressure of touch surface, and determine the position of target touch point needing to provide vibration feedback according to the touch position and the pressing pressure, wherein the vibration amplitude of the vibration feedback of the target touch point is greater than the first preset threshold;Microcontroller 202 can construct system matrix according to the position of the target touch point;According to the vibration feedback of the target touch point, define the expected vibration waveform;And combine the system matrix, calculate the drive waveform of the actuator according to the expected vibration waveform;Vibration drive module 203 can provide drive signal to the actuator based on the drive waveform, so that the actuator provides the vibration feedback of the target touch point.

[0072] It should be noted that the above-mentioned haptic feedback device can execute the haptic feedback method provided by the embodiment of the present application, and has the corresponding function modules and beneficial effects of executing the method. The technical details not described in detail in the haptic feedback device embodiment can be referred to the haptic feedback method provided by the embodiment of the present application.

[0073] The embodiment of the present application provides a kind of display device, please combine Figure 7, the display device 300 includes: one or more processors 301 and memory 302, Figure 7 with one processor 301 as an example.

[0074] Processor 301 and memory 302 can be connected by bus or other ways, Figure 7 with connection by bus as an example.

[0075] Memory 302 as a kind of non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program instructions / modules of the haptic feedback method in the embodiment of the present application. Processor 301 executes various function applications and data processing by running non-volatile software programs, instructions and modules stored in memory 302, that is, realizes the haptic feedback method in the above method embodiment.

[0076] The memory 302 can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function. In addition, the memory 302 can include a high-speed random access memory, and can also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, or other nonvolatile solid-state memory device. In some embodiments, the memory 302 can optionally include a memory disposed remotely from the processor 301.

[0077] The one or more modules are stored in the memory 302, and when executed by the one or more processors 301, perform the haptic feedback method in any of the method embodiments described above, for example, perform the method steps in FIG. 3 described above, to realize the functions of the modules in FIG. 6.

[0078] The above product can perform the method provided by the embodiments of the present application, and has the corresponding function modules and beneficial effects of performing the method. Technical details not described in detail in the embodiments can be referred to the method provided by the embodiments of the present application.

[0079] The display device of the embodiments of the present application exists in various forms, including but not limited to:

[0080] (1) Mobile communication device: This kind of device is characterized by having mobile communication function, and providing voice and data communication as the main target. This kind of terminal includes: smart phone (such as iPhone), multimedia phone, functional phone, and low-end phone, etc.

[0081] (2) Ultra-mobile personal computer device: This kind of device belongs to the category of personal computer, has computing and processing function, and generally also has the characteristics of mobile Internet. This kind of terminal includes: PDA, MID and UMPC device, etc., such as iPad.

[0082] (3) Portable entertainment device: This kind of device can display and play multimedia content. This kind of device includes: audio and video player (such as iPod), handheld game console, electronic book, and smart toy and portable car navigation device.

[0083] (4) Other display devices with data interaction function.

[0084] The embodiments of the present application provide a non-volatile computer readable storage medium, the computer readable storage medium stores computer executable instructions, the computer executable instructions are executed by one or more processors, for example, one processor 301 in FIG. 7, so that the above one or more processors can execute the haptic feedback method in any of the method embodiments described above, for example, execute the method steps in FIG. 3 described above, realize the functions of the modules in FIG. 6.

[0085] The embodiment of the present application provides a computer program product, the computer program product comprises a computer program stored on a non-volatile computer readable storage medium, the computer program comprises program instructions, when the program instructions are executed by the display device, the display device can execute the haptic feedback method in any method embodiment described above, for example, execute the method steps in figure 3 described above, realize the function of the module in figure 6.

[0086] The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software plus a general hardware platform, and of course can also be realized by hardware. Those skilled in the art can understand that all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in details; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A haptic feedback method applied to a display device, characterized in that, The display device includes a touch surface, a plurality of touch points corresponding to a touch operation of the touch surface, and edges of the touch surface are provided with actuators for providing vibration feedback at the touch points. The method includes: acquiring touch positions and pressing pressures of the touch surface, and determining a position of a target touch point requiring vibration feedback according to the touch positions and the pressing pressures, wherein a vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold; constructing a system matrix according to the position of the target touch point; defining a desired vibration waveform according to the vibration feedback of the target touch point; combining the system matrix, calculating a driving waveform of the actuator according to the desired vibration waveform; providing a driving signal to the actuator based on the driving waveform, so that the actuator provides the vibration feedback of the target touch point.

2. The method of claim 1, wherein, The number of the actuators is greater than or equal to the number of the touch points.

3. The method of claim 1, wherein, The position of the touch point is recorded as T j wherein j represents that the touch point is the jth touch point, and the defining of the expected vibration waveform according to the vibration feedback of the target touch point comprises: The position of the target touch point is determined according to the position of the target touch point. Setting the desired vibration waveform Y: wherein y j represents the desired vibration waveform corresponding to the jth touch point.

4. The method of claim 1, wherein, The combination of the system matrix and the calculation of the driving waveform of the actuator according to the desired vibration waveform includes: The driving waveform is calculated according to the following formula: Y = ΦS, wherein Y is the desired vibration waveform of the touch point, φ is the system matrix constructed according to the target touch point, and S is the driving waveform of the actuator.

5. The method of claim 4, wherein, The combination of the system matrix and the calculation of the driving waveform of the actuator according to the desired vibration waveform further includes: An error range is set for the desired vibration waveform, so that under the error range, the driving waveform of the actuator controls the vibration amplitude of the target touch point to be greater than a first preset threshold, and the vibration amplitudes of the remaining touch points except the target touch point are less than a second preset threshold, wherein the second preset threshold is less than the first preset threshold.

6. The method of claim 5, wherein, The setting of the allowable error range for the desired vibration waveform includes: The energy minimization of the system input is taken as an optimization target, and a constraint condition is set for the system input: argmin S T S,s.t.||y-y0|| 2 <ε where argmin denotes the value of S for which ||y-y0||2 is minimized under the constraint that S 2 <ε T S is the value of S for which S is minimized, s.t. denotes the constraint S T denotes the transpose of the drive waveform of the actuator, S T S denotes the energy of the system input, y denotes the drive waveform of the actuator, y0 denotes the desired vibration waveform, and ε denotes the allowed error between the vibration waveform of the touch point under the constraint and the desired vibration waveform; The driving waveform S of the actuator is calculated based on the constraint condition.

7. The method of claim 4, wherein, The combination of the system matrix and the calculation of the driving waveform of the actuator according to the desired vibration waveform further includes: The calculation of the driving waveform S is realized by the following formula: S = (φ T φ + βI) -1 φ T Y; where φ T denotes the transpose of the system matrix φ, I is the identity matrix, and β is a regularization parameter.

8. The method of claim 1, wherein, The construction of the system matrix according to the position of the target touch point includes: The system matrix φ is composed of a system transfer function according to the position of the target touch point.

9. The method of claim 1, wherein, The acquisition of the touch positions and the pressing pressures of the touch surface, and the determination of the position of the target touch point requiring vibration feedback according to the touch positions and the pressing pressures includes: acquiring a pressing pressure change curve; acquiring a change trend of a touch area of each touch point; when the pressing pressure exceeds a pressure threshold and the touch area exceeds an area threshold, determining that the touch point of the current touch position is the target touch point.

10. The method of claim 1, wherein, The method further includes: triggering an excitation waveform when the actuator provides the vibration feedback of the target touch point, the excitation waveform being used to represent the vibration of the user pressing action; In the downward pressing phase, the excitation waveform is triggered when the pressing force of the target touch point is higher than a pressure threshold.

11. The method of claim 10, wherein, The method further comprises: In the upward lifting phase, the excitation waveform is triggered when the pressing force of the target touch point is lower than the pressure threshold.

12. A haptic feedback device, applied to a display device, characterized in that, The display device comprises a touch surface corresponding to a plurality of touch points based on touch operations on the touch surface, and an actuator is arranged at the edge of the touch surface for providing vibration feedback at the touch points. The device comprises: a touch detection module for acquiring touch positions and pressing forces of the touch surface, and determining a position of a target touch point requiring vibration feedback according to the touch positions and the pressing forces, wherein the vibration amplitude of the vibration feedback of the target touch point is greater than a first preset threshold; a microcontroller for constructing a system matrix according to the position of the target touch point, defining a desired vibration waveform according to the vibration feedback of the target touch point, and calculating a driving waveform of the actuator according to the desired vibration waveform in combination with the system matrix; a vibration driving module for providing a driving signal to the actuator based on the driving waveform, so that the actuator provides vibration feedback of the target touch point.

13. A display device, characterized by It comprises a microcontroller, and a touch unit, a pressure detection unit and a vibration driving unit electrically connected with the microcontroller. The touch unit comprises a thin plate providing a touch surface. The pressure detection unit is arranged corresponding to the touch surface. The vibration driving unit is connected with a plurality of actuators, and the actuators are arranged at the edge of the touch surface. The touch unit is used for acquiring touch positions and providing the touch positions to the microcontroller. The pressure detection unit is used for detecting pressing forces of the touch surface and providing the pressing forces to the microcontroller. The microcontroller is used for determining a position of a target touch point requiring vibration feedback according to the touch positions and the pressing forces, constructing a system matrix according to the position of the target touch point, defining a desired vibration waveform according to the vibration feedback of the target touch point, and calculating a driving waveform of the actuator according to the desired vibration waveform in combination with the system matrix, and providing the driving waveform to the vibration driving unit. The vibration driving unit is used for providing a driving signal to the actuator based on the driving waveform, so that the actuator provides vibration feedback of the target touch point.

14. The display device of claim 13, wherein, The actuator is a piezoelectric ceramic.

15. The display device of claim 13, wherein, The vibration driving unit is further used for: amplifying the driving waveform in power, and providing a driving signal to the actuator based on the amplified driving waveform, so that the actuator provides vibration feedback of the target touch point.

16. A display device, characterized by It comprises: at least one processor and a memory; the memory is coupled with the processor, and the memory is used for storing instructions or programs, when the instructions or programs are executed by the at least one processor, the at least one processor executes the haptic feedback method as claimed in any one of claims 1-11.

17. A non-transitory computer readable storage medium, comprising: The non-transitory computer readable storage medium stores computer executable instructions that, when executed by a display device, cause the display device to perform the method of any of claims 1-11.

18. A computer program product, characterised in that, The computer program product comprises a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a display device, cause the display device to perform the method of any of claims 1-11.

Images