Method and device for rendering visual and tactile fusion of virtual buttons of interface of vehicle terminal

Abstract

The application provides a visual and tactile fusion rendering method and device for a virtual button of a vehicle terminal interface, and belongs to the technical field of tactile reproduction. The method generates a standard tactile feedback driving signal according to the mechanical properties of a physical button, and establishes the consistency of the visual feedback and the tactile feedback of the virtual button in the subjective perception intensity based on the Stevens power law, thereby determining the mapping relationship between the visual characteristic parameter representing the visual feedback stimulation intensity of the virtual button and the tactile characteristic parameter representing the tactile feedback stimulation intensity of the virtual button. Then, in combination with the mapping relationship between the tactile characteristic parameter and the tactile feedback driving signal scaling coefficient, the target tactile characteristic parameter is calculated according to the visual characteristic parameter, and the final visual and tactile fusion feedback driving signal is output. The application can realize the quantitative fusion of the visual and tactile feedback intensities of the virtual button, and improve the tactile reality and operation confirmation during the operation of the vehicle terminal.

Description

Technical Field

[0001] This invention belongs to the field of tactile reproduction technology, specifically relating to a visual-tactile fusion rendering method and device for virtual buttons on an in-vehicle terminal interface. Background Technology

[0002] With the continuous improvement of automotive intelligence and connectivity, in-vehicle touchscreens are gradually replacing traditional physical controls. However, touchscreens struggle to provide the same mechanical feedback as physical buttons when touched or pressed, lacking clear tactile confirmation signals. Existing virtual button tactile feedback solutions based on linear motors and other methods can provide some tactile feedback, but they primarily focus on the tactile output itself.

[0003] Current research has explored haptic feedback for virtual buttons from various perspectives. The paper "PerceivedDepth and Roughness of Virtual Buttons with Touchscreens" investigates the impact of different haptic parameters, such as the envelope shape, duration, frequency, and combined waveforms of the haptic signal, on the perception of depth and roughness of virtual buttons. The paper "Where's My Button? Evaluating the User Experience of Surface Haptics in Featureless Automotive User Interfaces" compares different feedback conditions, including visual and visual-haptic feedback, in a multi-tasking in-vehicle scenario. The results show that adding haptic feedback on top of visual feedback can improve users' subjective evaluations of interaction enjoyment, perceptual accuracy, quality, and operational confidence, and most participants prefer multi-sensory feedback methods.

[0004] However, the aforementioned studies primarily examine the impact of tactile parameters on tactile perception separately, or compare the interaction effects of different feedback modalities by setting discrete conditions such as the presence or absence of tactile feedback. Currently, there is a lack of a quantitative fusion rendering method based on user perception intensity for the visual and tactile feedback of virtual buttons. The inconsistency in perception intensity between vision and touch can lead to a significant modal disconnect for users, making it difficult to fully leverage the enhancement effect of multi-sensory integration on the human-computer interaction experience. Therefore, there is an urgent need to provide a method and device for fusing and rendering the visual and tactile feedback of virtual buttons on in-vehicle terminal interfaces to solve the above problems. Summary of the Invention

[0005] This invention provides a visual-tactile fusion rendering method and apparatus for virtual buttons on in-vehicle terminal interfaces, aiming to solve the problem that existing virtual button visual and tactile feedback lacks perceptual intensity correlation, leading to fragmented feedback and difficulty in accurately confirming operation results. By fusing and rendering visual and tactile feedback on a unified perceptual intensity dimension, the two collaboratively represent the interactive state of the virtual button. Furthermore, tactile feedback is applied based on the segmented mechanical characteristics of the physical button, improving the tactile realism of the virtual button and thus enhancing the operational certainty and user experience of in-vehicle touch interaction.

[0006] The technical solution adopted by this invention includes the following steps:

[0007] S1: Select a reference physical button and construct a haptic feedback rendering model for the virtual button based on its force feedback characteristics, and generate a standard haptic feedback driving signal;

[0008] S2: Determine the visual feature parameters used to characterize the intensity of the visual feedback stimulus of the virtual button and the tactile feature parameters used to characterize the intensity of the tactile feedback stimulus of the virtual button;

[0009] S3: Based on the standard tactile feedback drive signal generated in step S1, determine its maximum undistorted drive signal, and adjust the drive signal scaling factor according to the preset step size to determine the tactile feature parameters corresponding to different drive signal scaling factors, and establish the mapping relationship between tactile feature parameters and drive signal scaling factor.

[0010] S4: Based on Stevens' power law, establish the relationship between visual feature parameters and visual perception intensity, and the relationship between tactile feature parameters and tactile perception intensity, respectively. Using the equality of visual perception intensity and tactile perception intensity as the condition for visual-tactile fusion, establish an equal-strength fusion relationship between visual feature parameters and tactile feature parameters. Calculate the target tactile feature parameters based on the visual feature parameters of the current virtual button, and determine the target driving signal scaling factor using the mapping relationship between the tactile feature parameters and the driving signal scaling factor established in step S3. Scale the maximum undistorted driving signal to generate and output the final visual-tactile fusion feedback driving signal.

[0011] The implementation method of step S1 in this invention is as follows: The force-displacement curve of the reference physical button is divided into three tactile intervals according to the force change characteristics: smooth interval, jump interval, and bottoming-out interval. The smooth interval appears once or multiple times in a single tactile feedback process. The virtual button tactile feedback rendering model is as follows:

[0012] ;

[0013] in The standard haptic feedback drive signal output by the haptic feedback rendering model. For the time variable in a single tactile feedback process, The total number of segments of the tactile feedback signal. The tactile feedback drive signal corresponding to the segment of the j-th tactile signal. The set of model parameters for segmenting the j-th tactile signal. and These are the start and end times of the j-th tactile signal segment, respectively;

[0014] Based on the characteristics of the force-displacement curve of the physical button to be simulated and the characteristics of human tactile perception, determine the tactile feedback drive signal to be used for each tactile signal segment and the model parameters of each tactile signal segment, and generate a standard tactile feedback drive signal. .

[0015] In step S2 of this invention, the virtual button is a triggerable control displayed on the vehicle terminal touch screen interface, including one or more of text buttons, icon buttons, circular buttons, rectangular buttons, and irregularly shaped buttons, used to realize function confirmation, parameter adjustment, menu selection, or vehicle function control.

[0016] The implementation method of step S2 in this invention is as follows: determine the visual feature parameter S based on the visual feedback type of the virtual button. V The visual feedback types include changes to the visual state of the virtual buttons' size, color, brightness, transparency, and contrast, and the visual feature parameter S V Determined based on the amount of change, proportion of change, speed of change, duration of change, frequency of change, area of ​​change, or a combination thereof of the visual state;

[0017] For tactile feature parameter S H The determination of the haptic feature parameter S is first achieved by acquiring the haptic response signal generated by the haptic feedback mechanism applied to the touchscreen or the user's finger, and then determining the haptic feature parameter S based on the haptic response signal. H The tactile response signal includes one or more of the following: vibration displacement, vibration velocity, vibration acceleration, electrostatic attraction, friction force, and coefficient of friction; the tactile feature parameter S H It is at least one of the root mean square value, peak-to-peak value, mean value, and integral value of the tactile response signal calculated within a preset sampling period.

[0018] The implementation method of step S3 in this invention is as follows: gradually increase the standard tactile feedback driving signal generated in step S1. The overall amplitude is measured, and the output waveform of the signal driving unit and the vibration response of the touch screen surface are detected. When the output waveform or vibration response meets the preset distortion criterion, the driving signal corresponding to the previous scaling level is determined as the maximum undistorted driving signal. With the maximum undistorted drive signal Based on this, the scaling factor λ of the driving signal is adjusted according to a preset step size to obtain haptic feedback driving signals at different scaling levels. where 0 < λ ≤ 1;

[0019] Tactile feature parameters corresponding to different driving signal scaling factors are obtained through actual measurement, simulation or theoretical calculation, and fitting functions or lookup tables are used to characterize the mapping relationship between the two.

[0020] The implementation method of step S4 in this invention is as follows: According to Stevens' power law, the visual feature parameter S determined in step S2... V With visual perception intensity and tactile feature parameter S H With tactile perception intensity The relationship between them is as follows:

[0021] ;

[0022] in These are the visual and tactile perception intensities, respectively. These are the ratio coefficients for visual and tactile perception intensity, respectively; These are the power exponents of visual and tactile perception intensity, respectively. The specific values ​​of the proportionality coefficient and the power exponent are determined through experimental calibration based on the type of visual feedback, the type of tactile feedback, and the characteristics of human perception.

[0023] With the aforementioned visual perception intensity With tactile perception intensity Equality is used as a condition for visual-touch fusion to establish visual feature parameters. With tactile feature parameters The equal-strength fusion relationship between them: ,right Solving for the visual feature parameters yields the results. With tactile feature parameters The intensity matching relationship between them:

[0024] ;

[0025] The proportionality coefficient and power exponent in the equal-strength fusion relationship were determined through psychophysical experiments, and based on the visual feature parameters of the current virtual buttons. Calculate target tactile feature parameters ;

[0026] Based on the mapping relationship between the tactile feature parameters and the scaling factor of the driving signal established in step S3 Determine the target driving signal scaling factor corresponding to the target tactile feature parameter. And based on the target drive signal scaling factor, the maximum undistorted drive signal is... Scaling is performed to generate target haptic feedback drive signals:

[0027] ;

[0028] in, This is the final output feedback drive signal for visual-touch fusion.

[0029] A device for visual-tactile fusion rendering of virtual buttons on an in-vehicle terminal interface, wherein the rendering device is a tactile rendering device that integrates three tactile feedback mechanisms: electrostatic force, mechanical vibration, and air pressure film, and includes a tactile display and positioning unit, an information processing unit, and a signal driving unit.

[0030] The tactile display and positioning unit of this invention includes a capacitive screen, a piezoelectric ceramic sheet, a positioning detection module, a vibration isolation device, and a vibration motor. The capacitive screen is used to display the vehicle terminal interface and detect the finger touch position, and the capacitive screen has an electrostatic tactile feedback function. The piezoelectric ceramic sheet is attached to the back of the capacitive screen to realize the air pressure film tactile feedback function. The vibration motor is installed on the capacitive screen or its supporting structure through the vibration isolation device to generate mechanical vibration tactile feedback on the surface of the capacitive screen. The vibration isolation device includes an elastic vibration isolator to transmit vibration to the capacitive screen and suppress the vibration from spreading to the vehicle terminal host and the mounting frame.

[0031] The information processing unit of this invention is installed in an in-vehicle terminal and includes a processor and a memory. It is used to output visual rendering signals, receive finger contact position coordinates sent by the haptic display and positioning unit, and calculate visual feature parameters based on the visual feedback type of the virtual buttons. The visual feature parameters are calculated based on Stevens' power law. Target tactile feature parameters matched in perceived intensity And based on tactile feature parameters With drive signal scaling factor The mapping relationship between them determines the scaling factor of the target drive signal. .

[0032] The signal driving unit of the present invention includes a signal generator and a power amplifier, which are used to generate a visual-touch fusion feedback driving signal according to the control parameters of the tactile feedback driving signal sent by the information processing unit, and output the signal to the capacitive screen, piezoelectric ceramic sheet and vibration motor in the tactile display and positioning unit after power amplification.

[0033] The advantages of this invention are:

[0034] 1. This invention establishes a quantitative relationship between visual perception intensity and tactile perception intensity based on Stevens' power law. It determines the target tactile feedback intensity based on the visual feedback intensity of the virtual button, thereby achieving the fusion rendering of visual and tactile feedback in the dimension of perception intensity. This allows the two types of feedback to collaboratively express the pressing state of the virtual button, avoiding inconsistencies between the strength of visual and tactile feedback.

[0035] 2. This invention generates tactile feedback rendering signals by referencing the force feedback characteristics of physical buttons during the pressing process, so that the tactile feedback can dynamically change with the pressing stage of the virtual button, overcoming the shortcomings of fixed intensity or preset waveforms in representing the button travel and force changes, and improving the realism of the virtual button's tactile feel.

[0036] 3. By combining visual and tactile feedback, this invention can more clearly represent the pressing state and triggering result of virtual buttons, improving the operational certainty and user experience of in-vehicle touch interaction. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall structure of the virtual button visual-touch fusion rendering device;

[0038] Figure 2 This is a flowchart of the visual-touch fusion rendering method;

[0039] Figure 3 It selects the force-displacement curve of the physical button to be simulated;

[0040] Figure 4 This is a schematic diagram showing the division of three types of tactile regions and four consecutive regions;

[0041] Figure 5 This is a schematic diagram of the tactile feedback signal of a virtual button;

[0042] Figure 6 This is a schematic diagram of the vibration acceleration signal on the screen surface;

[0043] Figure 7 This is a schematic diagram showing the relationship between tactile feature parameters and the scaling factor of the tactile feedback drive signal;

[0044] Figure 8 This is a schematic diagram illustrating the relationship between visual feature parameters and target tactile feature parameters. Detailed Implementation

[0045] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The visual-touch fusion rendering method for virtual buttons on the vehicle terminal interface described in this invention can be implemented using at least one tactile feedback mechanism among mechanical vibration, electrostatic force, and air pressure film, or a combination of the above tactile feedback mechanisms.

[0047] To better understand this invention, the following is combined with... Figures 1 to 8 The present invention provides a detailed description of the visual-touch fusion rendering method and apparatus for virtual buttons on a vehicle terminal interface.

[0048] This invention provides a visual-touch fusion rendering device for virtual buttons on an in-vehicle terminal interface. The rendering device is a tactile rendering device that integrates three tactile feedback mechanisms: electrostatic force, mechanical vibration, and air pressure film. It includes a tactile display and positioning unit, an information processing unit, and a signal driving unit.

[0049] (1) Tactile display and positioning unit 11: includes a capacitive screen, a piezoelectric ceramic sheet, a positioning detection module, a vibration isolation device, and a vibration motor. The capacitive screen is used to display the vehicle terminal interface and detect the finger touch position, and the capacitive screen has an electrostatic tactile feedback function. The piezoelectric ceramic sheet is attached to the back of the capacitive screen to realize the air pressure tactile feedback function. The vibration motor is installed on the capacitive screen or its supporting structure through the vibration isolation device to generate mechanical vibration tactile feedback on the surface of the capacitive screen. The vibration isolation device includes an elastic vibration isolator to transmit vibration to the capacitive screen and suppress the spread of vibration to the vehicle terminal host and the mounting frame.

[0050] (2) Information processing unit 12: Located in the vehicle terminal, including a processor and a memory, used to output visual rendering signals, receive the finger contact position coordinates sent by the tactile display and positioning unit, and calculate visual feature parameters according to the visual feedback type of the virtual buttons. The visual feature parameters are calculated based on Stevens' power law. Target tactile feature parameters matched in perceived intensity And based on tactile feature parameters With drive signal scaling factor The mapping relationship between them determines the scaling factor of the target drive signal. .

[0051] (3) Signal driving unit 13: The driving unit includes a signal generator and a power amplifier, which are used to generate a visual-touch fusion feedback driving signal according to the control parameters of the tactile feedback driving signal sent by the information processing unit, and output the signal to the capacitive screen, piezoelectric ceramic sheet and vibration motor in the tactile display and positioning unit after power amplification.

[0052] Specifically, the vibration motor is mounted on the capacitive screen used to achieve electrostatic tactile feedback through a vibration isolation device. The vibration isolation device is used to constrain the vibration transmission path and reduce the spread of vibration to the vehicle terminal housing or other non-target structures, so that the mechanical vibration generated by the vibration motor mainly acts on the touch area of ​​the capacitive screen.

[0053] The information processing unit calculates visual feature parameters based on the visual feedback type of the virtual buttons. And calculate the visual feature parameter. Target tactile feature parameters matched in perceived intensity Subsequently, based on tactile feature parameters With drive signal scaling factor The mapping relationship between them determines the target scaling factor. The relevant parameters are then sent to the signal driving unit to generate the final visual-touch fusion feedback driving signal.

[0054] The visual-touch fusion rendering method for virtual buttons on an in-vehicle terminal interface according to this invention can be applied to various types of interactive devices, such as... Figure 1 A multi-mechanism fusion haptic feedback rendering device. Figure 2 This is a flowchart of the rendering method of the present invention, as shown below. Figure 2 As shown, the visual-touch fusion rendering method for virtual buttons on an in-vehicle terminal interface according to the present invention may specifically include the following steps:

[0055] S1: Select a reference physical button and construct a haptic feedback rendering model for the virtual button based on its force feedback characteristics, and generate a standard haptic feedback driving signal;

[0056] S2: Determine the visual feature parameters used to characterize the intensity of the visual feedback stimulus of the virtual button and the tactile feature parameters used to characterize the intensity of the tactile feedback stimulus of the virtual button;

[0057] S3: Based on the standard tactile feedback drive signal generated in step S1, determine its maximum undistorted drive signal, and adjust the drive signal scaling factor according to the preset step size to determine the tactile feature parameters corresponding to different drive signal scaling factors, and establish the mapping relationship between tactile feature parameters and drive signal scaling factor.

[0058] S4: Based on Stevens' power law, establish the relationship between visual feature parameters and visual perception intensity, and the relationship between tactile feature parameters and tactile perception intensity, respectively. Using the equality of visual perception intensity and tactile perception intensity as the condition for visual-tactile fusion, establish an equal-strength fusion relationship between visual feature parameters and tactile feature parameters. Calculate the target tactile feature parameters based on the visual feature parameters of the current virtual button, and determine the target driving signal scaling factor using the mapping relationship between the tactile feature parameters and the driving signal scaling factor established in step S3. Scale the maximum undistorted driving signal to generate and output the final visual-tactile fusion feedback driving signal.

[0059] The implementation method of step S1 is as follows: The force-displacement curve of the reference physical button is divided into three tactile regions according to the force change characteristics: smooth region, jump region, and bottoming-out region. The smooth region appears once or multiple times in a single tactile feedback process. The virtual button tactile feedback rendering model is as follows:

[0060] ;

[0061] in The standard haptic feedback drive signal output by the haptic feedback rendering model. For the time variable in a single tactile feedback process, The total number of segments of the tactile feedback signal. The tactile feedback drive signal corresponding to the segment of the j-th tactile signal. The set of model parameters for segmenting the j-th tactile signal. and These are the start and end times of the j-th tactile signal segment, respectively.

[0062] Based on the characteristics of the force-displacement curve of the physical button to be simulated and the characteristics of human tactile perception, determine the tactile feedback drive signal to be used for each tactile signal segment and the model parameters of each tactile signal segment, and generate a standard tactile feedback drive signal. .

[0063] This embodiment selects a physical button with typical hard-tactile mechanical characteristics as a reference, such as... Figure 3 As shown, based on the mechanical characteristics of the force-displacement curve of the physical button to be simulated, a single press process is divided into four continuous intervals: the first smooth interval, the jump interval, the second smooth interval, and the bottoming-out interval. The diagram illustrates this division. Figure 4 As shown, different intervals are obtained by dividing the force-displacement curve.

[0064] In this embodiment, the tactile feedback drive signal for each tactile signal segment uses an exponential envelope sine wave signal, which has the following form:

[0065] ;

[0066] in For amplitude coefficient, The vibration frequency, As an exponential factor, The duration of this section, For time.

[0067] The selection method for the parameters of the haptic feedback model in each interval is as follows: The first and second smooth intervals use the same parameters, mainly based on the human body's compliance illusion to simulate the feeling of pressing a button, therefore the exponential factor of the exponential envelope function is... Taking a negative value makes the envelope amplitude decrease over time, simulating the continuous tactile changes during a physical button press. The exponential factor of the jump interval exponential envelope. Taking a larger negative value causes the envelope amplitude to decay rapidly over time, corresponding to the mechanical sensation of "pulse-like force changes at the moment of buckling jump." The exponential factor of the envelope function in the bottoming-out region. A positive value is used to simulate the tactile sensation of a button being fully pressed down. A schematic diagram of the standard haptic feedback drive signal determined by the above steps is shown below. Figure 5 As shown.

[0068] The specific implementation of step S2 is as follows: Step S2 is used to determine the visual feature parameters characterizing the intensity of the visual feedback of the virtual buttons. tactile feature parameters of tactile feedback intensity .

[0069] The visual feature parameters This is used to characterize the degree of change in the visual state of a virtual button during its triggering, thereby reflecting the intensity of the visual feedback. Visual feature parameters are determined based on the type of visual feedback from the virtual button, which includes changes in the virtual button's size, color, brightness, transparency, contrast, and other visual states. These visual feature parameters... It is determined based on the amount of change, the proportion of change, the speed of change, the duration of change, the frequency of change, the area of ​​change, or a combination thereof of the visual state.

[0070] In this embodiment, the virtual button uses a press-and-scale visual feedback type, meaning that the virtual button shrinks in size after being triggered. Therefore, the speed of the virtual button's size scaling is selected as a visual feature parameter. The calculation method is as follows:

[0071] ;

[0072] in, This is the scaling factor of the virtual button's size after being triggered, relative to its initial size before being triggered. The duration of the animation effect for the virtual button as it changes from its initial size to its size after being triggered, in seconds. ); The unit is .

[0073] The tactile feature parameters This is used to characterize the intensity of tactile feedback when a virtual button is triggered, to acquire the tactile response signal generated by the tactile feedback mechanism on the touchscreen or the user's finger, and to determine tactile feature parameters based on the tactile response signal. The tactile response signal includes one or more of the following: vibration displacement, vibration velocity, vibration acceleration, electrostatic attraction, friction force, and coefficient of friction. The tactile feature parameters... It is at least one of the root mean square value, peak-to-peak value, mean value, and integral value of the tactile response signal calculated within a preset sampling period.

[0074] In this embodiment, the tactile feature parameters Defined as the root mean square value of the vibration velocity generated on the surface of the capacitive screen by the haptic feedback driving signal. In discrete signal processing, the corresponding discrete integral formula is:

[0075] ;

[0076] The formula for calculating the discrete root mean square is:

[0077] ;

[0078] in, For the first Vibration acceleration at each sampling point For the first Vibration velocity at each sampling point The sampling time interval, This represents the number of sampling points within the acquisition period. The calculated root mean square value of the vibration velocity is then used. As a tactile feature parameter ,Right now .

[0079] The implementation method of step S3 is as follows: gradually increase the standard tactile feedback driving signal generated in step S1. The overall amplitude is measured, and the output waveform of the signal driving unit and the vibration response of the touch screen surface are detected. When the output waveform or vibration response meets the preset distortion criterion, the driving signal corresponding to the previous scaling level is determined as the maximum undistorted driving signal. With the maximum undistorted drive signal Based on the preset step size, adjust the scaling factor of the drive signal. To obtain haptic feedback drive signals at different scaling levels , where 0 < ≤1.

[0080] Tactile feature parameters corresponding to different driving signal scaling factors are obtained through actual measurement, simulation or theoretical calculation, and the mapping relationship between the two is characterized by fitting function or lookup table.

[0081] In this embodiment, while maintaining the standard haptic feedback drive signal generated in step S1 While keeping the waveform structure unchanged, the overall amplitude is gradually increased, and the output waveform of the signal driving unit and the vibration response of the touch screen surface are detected. When any of the following conditions are detected: the output waveform shows clipping, the vibration motor enters saturation, or the similarity between the vibration response and the standard haptic feedback driving signal is lower than a preset threshold, the signal corresponding to the previous scaling level is determined as the maximum undistorted driving signal. In this embodiment, the vibration response is vibration acceleration.

[0082] Preferably, the value is taken in the range of 0.2 to 1.0 with a step size of 0.1. For each drive signal scaling factor... Below, an accelerometer arranged on the capacitive screen collects vibration acceleration signals from the screen surface. The collected vibration acceleration signals are as follows: Figure 6 As shown, and determined according to the above-mentioned root mean square method for calculating vibration velocity. ,Will As a tactile feature parameter at this scaling level After completing the tests at each stage, multiple sets of driving signal scaling factors were obtained. With tactile feature parameters Based on the changing trends of each data pair, linear functions, power functions, polynomial functions, or other monotonic functions are selected as candidate fitting functions. In this embodiment, the driving signal scaling factor... With tactile feature parameters The relationship is approximately linear; therefore, the following linear function is used for fitting, and the mapping relationship between the two is established through function fitting:

[0083] ;

[0084] The specific fitting relationship in this embodiment is as follows: Figure 7 As shown, the coefficient of determination of the fitting function This indicates that the linear fitting function can effectively characterize the relationship between the scaling factor of the driving signal and the root mean square value of the vibration velocity. The expression of the fitting function obtained in this embodiment is:

[0085] ;

[0086] in is the root mean square of the velocity, used to describe the intensity of the vibration. This is the scaling factor for the drive signal.

[0087] The specific implementation of step S4 is as follows: Based on Stevens' power law in psychophysics:

[0088] ,in Indicates the intensity of subjective perception. Indicates the intensity of physical stimulus. It represents the power exponent.

[0089] Different physical stimuli correspond to and They are usually different.

[0090] First, visual feature parameters are established separately. With visual perception intensity Mapping relationship between and tactile feature parameters With tactile perception intensity Mapping relationship between .in These are the ratio coefficients for visual and tactile perception intensity, respectively. These are the power exponents of the intensity of visual and tactile perception, respectively.

[0091] In this embodiment, visual perception intensity is used. With tactile perception intensity Equality as the fusion target, i.e. Solve the equation Visual feature parameters are obtained. With tactile feature parameters The intensity matching relationship between them:

[0092] ;

[0093] proportionality coefficient Power index Calibration was performed through psychophysical experiments. In this embodiment, multiple visual change speed levels were pre-set for visual feedback of size scaling and tactile feedback of mechanical vibration, and the corresponding visual change speeds were used as visual feature parameters. At each visual rate of change, subjects were presented with visual feedback of virtual button size scaling. Simultaneously, an adjustment method was used to change the scaling factor of the drive signal for mechanical vibration tactile feedback until the subjects perceived the visual and tactile feedback as equal in subjective intensity. The root mean square value of the screen surface vibration velocity at the point of intensity matching was recorded and used as a parameter for the current visual features. Corresponding tactile feature parameters The matching results obtained from multiple subjects at various visual change rate levels were statistically summarized to obtain multiple sets of visual feature parameters and tactile feature parameters data pairs. Based on the visual-tactile equal strength relation derived from Stevens' power law, let:

[0094] ;

[0095] in, , .

[0096] Taking the natural logarithm of both sides of the above equation, we get:

[0097] ;

[0098] by For independent variable, Perform least squares linear regression on the dependent variable, and determine the power exponent based on the slope of the regression line. The proportionality coefficient is determined based on the intercept of the regression line. The fitted function is as follows: Figure 8 As shown, the fitting results indicate that the scaling factor in this embodiment... =0.0104, power exponent Coefficient of determination This demonstrates that the established power function can effectively characterize the subjectively equal strength relationship between visual feedback of size scaling and tactile feedback of mechanical vibration. Therefore, the visual feature parameters in this embodiment... With tactile feature parameters The fusion relationship between them is as follows:

[0099]

[0100] Based on the visual feature parameters of the current virtual buttons The target tactile feature parameters are calculated using the aforementioned visual-tactile fusion relationship. Then, the calculated target tactile feature parameters Substituting the mapping relationship from step S3, we obtain the target driving signal scaling factor:

[0101]

[0102] in, To achieve the target tactile feature parameters The required drive signal scaling factor. The target drive signal scaling factor should be within the valid calibration range determined in step S3, that is:

[0103]

[0104] When the calculated target drive signal scaling factor is lower than the lower calibration limit, it is set as the lower calibration limit; when it is higher than the upper calibration limit, it is set as the upper calibration limit.

[0105] Based on the target drive signal scaling factor, the maximum undistorted drive signal determined in step S3 is... Scaling is performed to obtain the final output view-touch fusion feedback drive signal:

[0106]

[0107] Will and After unfolding, we get:

[0108]

[0109] By employing the aforementioned visual-touch fusion rendering method, the visual feature parameters of virtual buttons are converted into target tactile feature parameters, and corresponding tactile feedback driving signals are further generated. This establishes a quantitative correlation between visual and tactile feedback in terms of perceptual intensity, enabling the two types of feedback to collaboratively represent the interactive state of the virtual buttons and reducing the possibility of inconsistencies in visual and tactile feedback strength. Based on the complementary effect of visual and tactile information, this invention helps to enhance the state expression and operation result prompts of virtual buttons, improving the operational certainty and user experience of in-vehicle touch interaction.

Claims

1. A visual-touch fusion rendering method for virtual buttons on an in-vehicle terminal interface, characterized in that, Includes the following steps: S1: Select a reference physical button and construct a haptic feedback rendering model for the virtual button based on its force feedback characteristics, and generate a standard haptic feedback driving signal; S2: Determine the visual feature parameters used to characterize the intensity of the visual feedback stimulus of the virtual button and the tactile feature parameters used to characterize the intensity of the tactile feedback stimulus of the virtual button; S3: Based on the standard tactile feedback drive signal generated in step S1, determine its maximum undistorted drive signal, and adjust the drive signal scaling factor according to the preset step size to determine the tactile feature parameters corresponding to different drive signal scaling factors, and establish the mapping relationship between tactile feature parameters and drive signal scaling factor. S4: Based on Stevens' power law, establish the relationship between visual feature parameters and visual perception intensity, and the relationship between tactile feature parameters and tactile perception intensity. Use the equality of visual perception intensity and tactile perception intensity as the condition for visual-tactile fusion, and establish an equal-strength fusion relationship between visual feature parameters and tactile feature parameters. Calculate the target tactile feature parameters based on the visual feature parameters of the current virtual button, and use the mapping relationship between the tactile feature parameters and the scaling factor of the driving signal established in step S3 to determine the scaling factor of the target driving signal. Scale the maximum undistorted driving signal to generate and output the final visual-tactile fusion feedback driving signal.

2. The visual-touch fusion rendering method for virtual buttons on a vehicle terminal interface according to claim 1, characterized in that, The implementation method of step S1 is as follows: The force-displacement curve of the reference physical button is divided into three tactile regions according to the force change characteristics: smooth region, jump region, and bottoming-out region. The smooth region appears once or multiple times in a single tactile feedback process. The virtual button tactile feedback rendering model is as follows: ； in The standard haptic feedback drive signal output by the haptic feedback rendering model. For the time variable in a single tactile feedback process, The total number of segments of the tactile feedback signal. The tactile feedback drive signal corresponding to the segment of the j-th tactile signal. The set of model parameters for segmenting the j-th tactile signal. and These are the start and end times of the j-th tactile signal segment, respectively; Based on the characteristics of the force-displacement curve of the physical button to be simulated and the characteristics of human tactile perception, determine the tactile feedback drive signal to be used for each tactile signal segment and the model parameters of each tactile signal segment, and generate a standard tactile feedback drive signal. .

3. The visual-touch fusion rendering method for virtual buttons on an in-vehicle terminal interface according to claim 1, characterized in that, In step S2, the virtual buttons are triggerable controls displayed on the vehicle terminal touch screen interface, including one or more of text buttons, icon buttons, circular buttons, rectangular buttons, and irregularly shaped buttons, used to realize function confirmation, parameter adjustment, menu selection, or vehicle function control.

4. The visual-touch fusion rendering method for virtual buttons on an in-vehicle terminal interface according to claim 3, characterized in that, The implementation method of step S2 is as follows: determine the visual feature parameter S based on the visual feedback type of the virtual button. V The visual feedback types include changes to the visual state of the virtual buttons' size, color, brightness, transparency, and contrast, and the visual feature parameter S V Determined based on the amount of change, proportion of change, speed of change, duration of change, frequency of change, area of ​​change, or a combination thereof of the visual state; For tactile feature parameter S H The determination of the haptic feature parameter S is first achieved by acquiring the haptic response signal generated by the haptic feedback mechanism applied to the touchscreen or the user's finger, and then determining the haptic feature parameter S based on the haptic response signal. H The tactile response signal includes one or more of the following: vibration displacement, vibration velocity, vibration acceleration, electrostatic attraction, friction force, and coefficient of friction; the tactile feature parameter S H It is at least one of the root mean square value, peak-to-peak value, mean value, and integral value of the tactile response signal calculated within a preset sampling period.

5. The visual-touch fusion rendering method for virtual buttons on a vehicle terminal interface according to claim 1, characterized in that, The implementation method of step S3 is as follows: gradually increase the standard tactile feedback driving signal generated in step S1. The overall amplitude is measured, and the output waveform of the signal driving unit and the vibration response of the touch screen surface are detected. When the output waveform or vibration response meets the preset distortion criterion, the driving signal corresponding to the previous scaling level is determined as the maximum undistorted driving signal. With the maximum undistorted drive signal Based on this, the scaling factor λ of the driving signal is adjusted according to a preset step size to obtain haptic feedback driving signals at different scaling levels. where 0 < λ ≤ 1; Tactile feature parameters corresponding to different driving signal scaling factors are obtained through actual measurement, simulation or theoretical calculation, and fitting functions or lookup tables are used to characterize the mapping relationship between the two.

6. The visual-touch fusion rendering method for virtual buttons on a vehicle terminal interface according to claim 1, characterized in that, The implementation method of step S4 is as follows: According to Stevens' power law, the visual feature parameter S determined in step S2 is... V With visual perception intensity and tactile feature parameter S H With tactile perception intensity The relationship between them is as follows: ； in These are the visual and tactile perception intensities, respectively. These are the ratio coefficients for visual and tactile perception intensity, respectively; These are the power exponents of visual and tactile perception intensity, respectively. The specific values ​​of the proportionality coefficient and the power exponent are determined through experimental calibration based on the type of visual feedback, the type of tactile feedback, and the characteristics of human perception. With the aforementioned visual perception intensity With tactile perception intensity Equality is used as a condition for visual-touch fusion to establish visual feature parameters. With tactile feature parameters The equal-strength fusion relationship between them: ,right Solving for the visual feature parameters yields the results. With tactile feature parameters The intensity matching relationship between them: ； The proportionality coefficient and power exponent in the equal-strength fusion relationship were determined through psychophysical experiments, and based on the visual feature parameters of the current virtual buttons. Calculate target tactile feature parameters ; Based on the mapping relationship between the tactile feature parameters and the scaling factor of the driving signal established in step S3 Determine the target driving signal scaling factor corresponding to the target tactile feature parameter. And based on the target drive signal scaling factor, the maximum undistorted drive signal is... Scaling is performed to generate target haptic feedback drive signals: ； in, This is the final output feedback drive signal for visual-touch fusion.

7. An apparatus for implementing the visual-touch fusion rendering method for virtual buttons on a vehicle terminal interface as described in any one of claims 1 to 6, characterized in that, The rendering device is a tactile rendering device that integrates three tactile feedback mechanisms: electrostatic force, mechanical vibration, and air pressure film. It includes a tactile display and positioning unit, an information processing unit, and a signal driving unit.

8. The visual-touch fusion rendering device for virtual buttons on a vehicle terminal interface according to claim 7, characterized in that: The tactile display and positioning unit includes a capacitive screen, a piezoelectric ceramic sheet, a positioning detection module, a vibration isolation device, and a vibration motor. The capacitive screen is used to display the vehicle terminal interface and detect the finger touch position, and the capacitive screen has an electrostatic tactile feedback function. The piezoelectric ceramic sheet is attached to the back of the capacitive screen to realize the air pressure tactile feedback function. The vibration motor is installed on the capacitive screen or its supporting structure through the vibration isolation device to generate mechanical vibration tactile feedback on the surface of the capacitive screen. The vibration isolation device includes an elastic vibration isolator to transmit vibration to the capacitive screen and suppress the vibration from spreading to the vehicle terminal host and the mounting frame.

9. The visual-touch fusion rendering device for virtual buttons on a vehicle terminal interface according to claim 7, characterized in that: The information processing unit, located in the vehicle-mounted terminal, includes a processor and a memory. It outputs visual rendering signals, receives finger contact position coordinates from the haptic display and positioning unit, and calculates visual feature parameters based on the visual feedback type of the virtual buttons. The visual feature parameters are calculated based on Stevens' power law. Target tactile feature parameters matched in perceived intensity And based on tactile feature parameters With drive signal scaling factor The mapping relationship between them determines the scaling factor of the target drive signal. .

10. The visual-touch fusion rendering device for virtual buttons on a vehicle terminal interface according to claim 7, characterized in that: The signal driving unit includes a signal generator and a power amplifier, which are used to generate a visual-touch fusion feedback driving signal according to the control parameters of the tactile feedback driving signal sent by the information processing unit, and output the signal to the capacitive screen, piezoelectric ceramic sheet and vibration motor in the tactile display and positioning unit after power amplification.

Images