Interactive interface design parameterization method guided by tactile feedback

Abstract

The invention provides an interactive interface design parameterization method guided by tactile feedback. The method comprises the following steps: acquiring physiological features, use habits and use scene information of a user, and presetting basic tactile feedback parameters according to a scene-parameter mapping rule; during interaction, touch parameters such as vibration frequency and amplitude are dynamically adjusted based on user operation behaviors, physiological reactions and scenes by using a deep learning model; and meanwhile, a multi-modal parameter fusion model is established, so that tactile, visual and auditory parameters have a synergistic effect, for example, when a button is clicked, the three parameters are mutually associated to enhance a feedback effect. After the operation is completed, the feedback effect is evaluated through objective indexes such as the subjective score of the user and the operation accuracy, and the user personalized model and the parameter adjustment model are optimized. According to the method, the problems that traditional parameters are fixed, and multi-modal fusion is insufficient are solved, accurate and personalized man-machine interaction is achieved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, and in particular to a haptic feedback-oriented interactive interface design parameterization method for realizing precise and personalized haptic interaction experience. BACKGROUND

[0002] In the current field of human-computer interaction, the haptic feedback-oriented interactive interface design parameterization method has made certain progress, but there are still many problems to be solved.

[0003] Existing methods generally use fixed parameter settings to realize haptic feedback. However, there are significant differences in physiological characteristics and usage habits among different users, such as the different sensitivity of the elderly and young people to the perception of haptic intensity, and the different operation frequency and habits of game players and ordinary office users. At the same time, the usage scenarios of users are diverse, from quiet indoor office scenarios to noisy outdoor mobile scenarios. Fixed parameter settings cannot adapt to these changes and cannot meet the personalized needs of users for haptic feedback in different situations, resulting in poor adaptability and comfort of the interaction experience.

[0004] In terms of multi-modal information fusion, the existing technology has a low degree of integration of haptic feedback with visual, auditory and other modal information. In actual interaction processes, each modal information is often independent and lacks effective coordination and cooperation. For example, in some applications, the rhythm of haptic feedback and the playback speed of visual animation are inconsistent, or the information conveyed by auditory cues and haptic feedback is biased, which cannot form a coordinated and unified multi-sensory interaction experience, reducing the overall perception and understanding efficiency of users for interactive operations and affecting the fluency and immersion of human-computer interaction.

[0005] In addition, the diversity of hardware devices and the individual differences in users' sensitivity to haptic feedback also pose great challenges to existing parameterization design methods. Different types of hardware devices, such as mobile phones, game controllers, smart wearable devices, etc., have different performance and parameters of haptic feedback hardware, which makes it difficult for uniform parameter settings to achieve ideal results on all types of devices. At the same time, research shows that the sensitivity of different users to haptic feedback can differ by up to 30%, and existing parameterization design methods cannot accurately match the haptic perception characteristics of each user, resulting in some users being unable to obtain satisfactory haptic feedback experience, which limits the widespread application and effectiveness of haptic feedback technology in interactive interface design. SUMMARY

[0006] Although the application of tactile feedback in interactive interface design has gradually become popular, it still faces many technical bottlenecks. On the one hand, traditional parameterized design uses fixed tactile feedback parameters, which cannot adapt to the significant differences in physiological characteristics, operation habits and perception preferences of different users. For example, the elderly user group needs stronger tactile feedback to confirm the operation due to the decline of skin perception ability; while young users, especially game players, prefer delicate and rhythmic tactile feedback to enhance the immersive experience. On the other hand, the existing tactile feedback design lacks the ability to respond to dynamic changes in the use scenario. In noisy outdoor environments, quiet indoor office scenarios or intense game battle scenarios, the same set of tactile feedback parameters cannot balance the accuracy of information transmission and the comfort of user experience. In addition, the integration of tactile feedback with visual, auditory and other modal information often stays on the surface, and there is a lack of depth collaboration between modalities, making it difficult for users to obtain unified and smooth multi-sensory experience in the interaction process.

[0007] The present application aims to break through the above technical limitations and build a highly flexible and intelligent tactile feedback interactive interface design system through innovative parameterized design strategies. It realizes the precise customization and dynamic optimization of tactile feedback parameters, which can adapt to the individual needs of different users and multiple use scenarios in real time; at the same time, it deeply integrates tactile, visual, auditory and other multi-modal information, improves the intelligence and humanization level of human-computer interaction, brings users a more high-quality, efficient and immersive interactive experience, and promotes the innovative development and wide application of tactile feedback technology in the field of interactive interface design. Technical scheme

[0008] The present application provides a tactile feedback oriented interactive interface design parameterization method, which specifically includes the following detailed steps and technical points: S1: User and scenario information collection A comprehensive information collection system is constructed using high-precision sensors. In terms of user physiological characteristic information collection, a galvanic skin response sensor is used to capture real-time changes in skin sensitivity. Based on the principle of skin electrical activity, the sensor can detect microvolt-level electrical signal fluctuations caused by tactile stimulation through electrode contact with the skin surface, and then accurately determine the sensitivity of the user's skin to different intensity and frequency tactile stimuli. Electromyography sensors collect bioelectrical signals generated by hand muscles during contraction and relaxation to obtain electrical activity data of muscles in different operation states such as clicking, sliding and holding, in order to analyze the tension, force mode and fatigue state of the user's hand muscles.

[0009] In the user habit information collection, the system log record and the user behavior monitoring software are combined. The system log automatically records the operation track of the user in various application programs, including the operation timestamp, operation type, operation object and other information, so as to accurately count the user's document editing, format setting operation in office software, game level challenge, prop use operation and other common operation types in entertainment software. Through big data analysis technology, the operation frequency is accurately calculated, and the user's use frequency of specific operation is quantified. At the same time, through the user feedback mechanism, such as pop-up window evaluation, scoring system and text input box, the subjective evaluation of the user on the different tactile feedback contacted in the past is collected, covering whether the feedback strength is comfortable, whether the rhythm is reasonable, whether the mode is easy to understand and other dimensions.

[0010] The built-in accelerometer, gyroscope, ambient light sensor and microphone of the device are used to realize real-time sensing of the device posture, ambient light intensity and ambient noise information. The accelerometer and gyroscope can accurately determine the motion state and spatial posture of the device in three-dimensional space based on the principle of inertial measurement, whether it is the horizontal and vertical screen switching, rapid rotation of the mobile phone, or the tilt and shaking action of the smart wearable device. The ambient light sensor detects the intensity of ambient light through a light-sensitive element, which provides a basis for determining whether the user is in an indoor, outdoor, daytime or nighttime scene. The microphone collects real-time ambient noise data, combined with the preset noise threshold and feature analysis, to assist in determining whether the user is in a quiet indoor conference room, a noisy outdoor street and other different environments, so as to comprehensively and accurately determine the use scene of the user.

[0011] S2: Parameter pre-setting Based on the scene-parameter mapping rules constructed through a large number of experiments and data analysis in advance, the basic tactile feedback parameters are preset for the user according to the real-time sensed scene. The construction process of the mapping rule adopts the method of combining machine learning and statistics. The research team conducts large-scale user testing in different scenes, collects a sample set containing thousands of users and tens of thousands of operation data. Through clustering analysis, regression analysis and other statistical methods, the internal correlation between scenes and tactile feedback parameters is analyzed in depth; at the same time, support vector machine, decision tree and other machine learning algorithms are used to construct a scene-parameter prediction model, and the model parameters are continuously optimized through cross-validation, so as to establish an accurate mapping model.

[0012] The preset basic haptic feedback parameters cover two dimensions of physical layer parameters and perception layer parameters. In the physical layer parameters, the vibration frequency setting range is 10-200 Hz, the vibration in the low frequency band (10-50 Hz) can simulate strong impact feeling, and is suitable for warning scenes such as system warning, attack in game and the like which need to attract the user's high attention; the vibration in the high frequency band (100-200 Hz) can bring delicate and soft touch, and is suitable for daily operations such as feedback of mobile phone message input, file browsing and the like. The amplitude adjustment range is 0.1-1.5 mm, and the strength of the haptic feedback is precisely adjusted by controlling the eccentric wheel rotation amplitude of the micro vibration motor or the deformation degree of the piezoelectric ceramic. The waveforms include sine wave, square wave, pulse wave and the like, the sine wave is often used for regular operation feedback due to its smooth change characteristic, and gives the user a gentle touch; the square wave has obvious high and low level change, and shows strong and crisp characteristics, and is suitable for key operation confirmation, such as payment password input completion prompt; the pulse wave can simulate short and sharp touch, and is used for special prompt, such as device low power warning. The duration is within 50-500 ms, according to the importance of the operation and the scene requirement, the duration of the single feedback is reasonably set, for example, the feedback duration of the fast click operation can be set to 50-100 ms, and the feedback duration of the important file deletion confirmation can be prolonged to 300-500 ms.

[0013] In the aspect of the perception layer parameters, the intensity is dynamically adjusted according to the user's early feedback and scene characteristics. By establishing a user preference model, combining factors such as the urgency and importance of the scene, the intensity of the haptic feedback is automatically adjusted to ensure that the intensity of the haptic feedback matches the user's perception preference and the scene atmosphere. The rhythm is determined by carefully setting the time interval of multiple feedbacks, for example, in the continuous click operation, a fast vibration sequence with a short time interval is adopted to create a smooth and natural operation rhythm, and the user's operation experience is improved. The mode design follows the user's cognitive habit, and a specific combination logic is adopted, such as "short vibration - long vibration - short vibration" representing error prompt, "continuous short vibration" representing operation success, "long vibration - short vibration - long vibration" representing information prompt, and the like, so that the user can quickly understand the information conveyed by the feedback.

[0014] S3: Dynamic self-adaptive adjustment Throughout the entire process of user-device interaction, advanced behavior analysis algorithms and physiological monitoring technologies are used to monitor user operation behavior and physiological response in real time and accurately. The behavior analysis algorithm is based on computer vision, pattern recognition and deep learning technology. Through the device camera, it can capture user gesture actions, or through the touch screen, it can collect operation trajectory data, accurately identify user operation speed, force changes, and click, slide, pinch and other operation gesture features. The physiological monitoring technology uses wearable devices or built-in heart rate sensors, blood pressure sensors, skin conductance sensors, etc. to obtain physiological response data such as increased heart rate, increased hand muscle tension, and skin conductance changes in real time.

[0015] When abnormal changes in user operation behavior or physiological response are detected, a pre-trained deep learning model is immediately started. The deep learning model uses a multi-layer convolutional neural network (CNN) combined with a long short-term memory network (LSTM) architecture, where CNN is used to extract spatial features from operation behavior data, and LSTM is used to process time series data and capture dynamic changes in user behavior and physiological response. User personalized information (including user physiological feature preference models, operation habit models, perception threshold models, etc. established based on previous collection and analysis), current scene information (such as scene type, environmental parameters, task urgency, etc.), and real-time operation behavior information (such as operation speed, force, trajectory, gesture type, etc.) and physiological response information are used as multi-dimensional inputs. After complex convolution operations, loop calculations and learning processes within the neural network, the model can quickly and accurately output the optimal haptic feedback parameter combination that adapts to the current user state and scene, achieving dynamic and adaptive adjustment of haptic feedback parameters. For example, when the user's operation speed increases and heart rate rises during a game battle, the model will automatically increase the intensity and frequency of haptic feedback to enhance the tension and immersion of the game; when the user performs a long-time document editing operation, causing an increase in hand muscle tension, the model will reduce the intensity of haptic feedback to reduce user fatigue, ensuring that the interaction process always meets user needs and improves the naturalness and smoothness of the interaction.

[0016] S4: Multi-modal fusion design By constructing complex mathematical correlation models and intelligent control algorithms, a close and dynamic correlation between haptic feedback parameters and visual and auditory parameters is established, forming an integrated multi-modal parameter fusion model. When the user performs an interactive operation, the system adjusts the haptic, visual and auditory parameters simultaneously and collaboratively based on the operation type (such as button click, page slide, file deletion, etc.) and the current scene (such as game scene, office scene, navigation scene, etc.).

[0017] Specifically, in the button click operation, the tactile feedback simulates the physical button touch feeling with carefully tuned vibration frequency and amplitude, giving the user a real pressing feedback; the visual interface uses particle special effects, light and shadow changes, etc. Synchronous presentation of zooming animation when the button is pressed, visually enhancing the operation feedback; the auditory system plays the corresponding click sound effect through sound synthesis technology, enhancing the realism of the operation. And through the intelligent control algorithm, a nonlinear function relationship between the tactile intensity, visual zoom amplitude and sound volume is established, ensuring that the information of the three modalities is mutually echoed, coordinated in strength, rhythm and time. In the page sliding operation, the tactile feedback simulates the sliding resistance change, dynamically adjusts the vibration frequency and amplitude with the change of sliding speed and intensity; the visual interface uses smooth animation transition effect to display the page sliding effect in real time; the auditory system plays a soft sliding sound effect that changes dynamically according to the sliding speed and direction, and the modal parameters are dynamically adjusted according to the operation intensity and speed through the preset parameter linkage rules, together creating an immersive interactive experience. In the navigation scene, when the user approaches the destination, the tactile feedback prompts the direction change with gradually increasing vibration frequency, the visual interface highlights the target position and enlarges the map scale, and the auditory system plays the distance prompt voice, realizing the organic integration of multi-modal information, helping users more clearly and accurately obtain navigation information.

[0018] S5: Feedback effect evaluation and optimization After each interaction operation is completed, a comprehensive and scientific feedback effect evaluation process is immediately started. In terms of subjective evaluation, through a carefully designed user scoring system, users are guided to rate the intensity, rhythm, mode, etc. Satisfaction of this tactile feedback in multiple dimensions. The scoring system uses a ladder type scoring interface, combined with intuitive icon examples and text instructions, to ensure that users can accurately express their feelings. At the same time, an open-ended text input box is set up to collect users' specific improvement suggestions and opinions on tactile feedback. Objective index monitoring relies on the built-in operation recording tool to accurately record the operation accuracy (i.e. the proportion of users correctly completing the operation), operation time (the length of time from the start of the operation to the completion), heart rate, blood pressure, skin electrical reaction changes, and other physiological indicator data before and after the operation. Through the sensor data acquisition and analysis module, physiological indicator data is processed and features are extracted in real time, such as calculating heart rate variability, skin electrical reaction peak value, etc.

[0019] Based on the evaluation results of subjective and objective combination, the advanced model optimization algorithm is used to update the user personalized model and dynamic parameter adjustment model. The optimization process includes adjusting the parameter weight in the model by using gradient descent, genetic algorithm and other optimization methods according to the evaluation data, highlighting the consideration of key factors affecting user experience; Using model diagnosis tools and structural analysis algorithms to modify the model structure, such as adding or deleting neural network layers, adjusting network connection methods, etc., so that it can better adapt to the changes of different users and scenes. For example, when it is found that a certain group of users has a lower satisfaction with the intensity of tactile feedback in a specific scene, the weight of the tactile intensity parameter in this scene is increased through the optimization algorithm, and the structure of the deep learning model is adjusted to improve the perception ability of this scene, so as to continuously improve the accuracy of tactile feedback parameter adjustment and user satisfaction, so that the system can continuously evolve in the process of continuous use, and always maintain good interaction performance.

[0020] The above technical scheme can bring the following technical effects: 1. Precise satisfaction of user individual needs: By collecting user physiological characteristics and usage habit information through high-precision sensors, an individual model is constructed. For elderly users with declining skin perception ability, the tactile feedback intensity can be automatically enhanced to ensure operation confirmation; For young game players, the system can provide delicate and rhythmic tactile feedback according to their operation habits and preferences, such as simulating the impact feeling with high-frequency vibration when releasing game skills to enhance the immersive experience. Compared with traditional fixed parameter design, this scheme realizes "thousand faces for thousands of people" in tactile feedback, greatly improving user satisfaction and adaptability to the interactive interface.

[0021] 2. Efficient adaptation to multiple dynamic scenes: By using multiple types of sensors to perceive scene changes in real time, combined with pre-built scene-parameter mapping rules and dynamic adjustment models, the system can quickly switch and adapt to the tactile feedback parameters of the scene. In noisy outdoor scenes, the tactile feedback intensity and duration are automatically enhanced to avoid missing important operation prompts due to environmental interference; In quiet indoor office scenes, the feedback intensity is reduced to reduce interference to the user. Whether the device posture changes, the environment light brightness, or the noise intensity changes, the system can respond in real time to ensure that the tactile feedback accurately conveys information in various scenes without affecting the user experience.

[0022] 3. Deeply integrate multi-modal information to improve interaction experience: By constructing mathematical correlation models and intelligent control algorithms, the parameters of touch, vision, and hearing are deeply coordinated. In the button click operation, the touch simulates the pressing touch, the vision presents the zooming dynamic effect, and the hearing plays the corresponding sound effect. The three modal information is closely related in intensity, rhythm, and time, and the operation feedback is jointly strengthened. In the navigation scene, the touch prompts the direction with vibration frequency change, the vision highlights the target, and the hearing broadcasts the distance. The multi-modal information is organically integrated to provide users with a full range of immersive interaction experience. Compared with the traditional design relying on single or loosely combined multi-modal, the user's information acquisition efficiency and interaction participation are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 Parametric method steps for the haptic feedback guided interaction interface design of the present application DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. Figure 1 It should be apparent that the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0025] Example 1: Daily interaction scene of mobile phone In the daily use scene of mobile phone, the user information acquisition module takes high-precision sensors as the core to build a multi-dimensional data acquisition system. The heart rate sensor built-in the mobile phone adopts the photoelectric plethysmogram (PPG) technology, irradiates the skin through LED light source, receives the reflected light signal, and through analog-to-digital conversion and digital signal processing, real-time analyzes the dynamic change curve of the user's heart rate in the process of using the mobile phone for a long time. The pressure sensor is distributed below the screen based on the piezoresistive effect principle, which can accurately capture the spatial distribution of the pressure size when touching the screen, and record the intensity data of each press with a resolution of 0.1N.

[0026] At the same time, the system records the user's usage habits through deep log recording and behavior analysis algorithm. In the use scene of social software, the input method of the user when sending messages is analyzed by natural language processing (NLP) technology, and the voice input, handwriting input or keyboard input are distinguished, and the use frequency of each input method is counted. In addition, through various feedback mechanisms such as pop-up questionnaire and star rating, the detailed evaluation of the user on the past different input method haptic feedback is collected, including the satisfaction degree of feedback intensity, frequency, waveform and other parameters, and the improvement suggestions on the feedback mode.

[0027] The scene perception module integrates multi-type sensor data fusion algorithm to realize accurate judgment of the scene. The accelerometer and gyroscope fuse three-axis acceleration and angular velocity data through a complementary filtering algorithm to output the attitude quaternion of the phone in real time at a sampling frequency of 100 Hz, accurately judge the horizontal and vertical screen state, shaking amplitude, etc. The ambient light sensor uses a digital ambient light sensor chip that can perceive the ambient light intensity in a wide dynamic range of 0-100000 lux. The microphone array combines a voice activity detection (VAD) algorithm to identify the intensity and type of environmental noise and a machine learning classification model to determine whether the user is in a noisy subway environment. When it is detected that the user is using the phone to browse news on the subway (environmental noise > 70 dB, light intensity < 50 lux, and the phone is in a shaking state), the corresponding scene response mechanism is triggered.

[0028] The parameter pre-setting module customizes the basic haptic feedback parameters for the current scene based on the scene-parameter mapping model trained by big data. For the sliding operation of news browsing, the best parameter combination is extracted from the test data of millions of users: vibration frequency 30 Hz, which matches the neural perception frequency when the human finger naturally slides through Fourier transform analysis; amplitude 0.2 mm, which ensures that the user can perceive it in a noisy subway environment, and it will not cause hand fatigue due to excessive feedback; sine wave waveform can simulate the soft page turning touch, duration 80 ms, which meets the time response characteristics of human tactile perception. When the user clicks on the news title to enter the detail page, the feedback uses a frequency of 80 Hz, an amplitude of 0.3 mm, a square wave waveform, and a duration of 100 ms. The steep rising and falling edges of the square wave can simulate the clear touch of a physical button, enhancing the operation confirmation feeling.

[0029] During the interaction, the dynamic self-adaptive adjustment module adopts a real-time machine learning algorithm to realize dynamic optimization of the tactile feedback. Through wavelet transform analysis of the pressure sensor data, hand muscle tension characteristic parameters are extracted, and behavior data such as sliding speed and operation frequency are input into a pre-trained long short-term memory (LSTM) model. If it is detected that the user quickly slides to browse multiple news (sliding speed > 100px / s) and the hand muscle tension increases (standard deviation of pressure sensor output value > 0.3N), the LSTM model calculates the optimal parameter adjustment scheme according to the user behavior pattern learned from historical data, and increases the vibration frequency of the sliding operation to 40Hz and the amplitude to 0.25mm to enhance the feedback intensity. At the same time, the multi-modal fusion module realizes precise coordination of tactile, visual and auditory feedback based on a time synchronization protocol such as the PTP protocol. When sliding the news list, the visual interface uses a Bezier curve interpolation algorithm to present a smooth page scrolling effect, and simultaneously plays a slight “rustling” sound effect generated by a convolutional neural network, and the sound volume (achieved by adjusting the amplitude of the audio signal), the visual scrolling speed (adjusting the animation frame rate) and the tactile feedback intensity (controlling the vibration motor driving current) are positively correlated through a linear mapping function.

[0030] After the interaction is completed, the feedback effect evaluation module adopts a subjective and objective evaluation system. In terms of subjective evaluation, a pop-up window is used to display a Likert scale containing 10 dimensions, and the user is invited to rate the tactile feedback of the browsing operation; objective evaluation is achieved by recording the operation accuracy (such as the number of accidental touches / total number of operations) and operation time in the system log. If the user feedbacks that the tactile feedback intensity is insufficient (score < 3), the system integrates the feedback data with operation behavior data and scene data, analyzes the key factors affecting satisfaction using a random forest algorithm, adjusts the weights of related parameters in the user individualization model and the dynamic parameter adjustment model, and increases the tactile feedback intensity weight in subsequent news browsing scenarios. Through actual testing, compared with traditional mobile phone tactile feedback design, the user's news browsing operation accuracy in complex environments is improved by 18%, and the interactive experience satisfaction is improved by 22% through the method of the present application.

[0031] Example 2: Virtual reality game scenario In the VR game Star Battle, the user information collection module relies on professional level sensing equipment to achieve deep collection of physiological and behavioral data. The pressure sensor built into the tactile glove uses a flexible thin film pressure sensor array that can cover 12 key parts such as finger joints and palm, with a 0.01N accuracy resolution to monitor the degree of finger force during shooting in real time; the electromyography sensor collects the surface electromyography signal of the hand muscle through a dry electrode array, after band-pass filtering (5-500Hz), full-wave rectification, and moving average processing, it extracts characteristic parameters such as muscle contraction frequency and fatigue degree. At the same time, the system records the user's game operation habits in detail through the in-game behavior recorder, including the commonly used weapon switching method (such as key switching, gesture switching), the type of evasive action (side flash, back jump) and the frequency of use, and constructs a user-specific operation behavior graph.

[0032] The scene perception module is based on the deep fusion of the positioning sensor of the VR device and the game engine data, and realizes the accurate identification of the virtual scene. The nine-axis inertial measurement unit (IMU) built into the VR headset combines external base station positioning data, and through the extended Kalman filtering algorithm, it outputs the 6-degree-of-freedom pose information of the user in the virtual space at a refresh rate of 200Hz; the game engine outputs scene data such as scene type (space battle, planet exploration), number of enemy units, weapon equipment status in real time. When the user enters the space battle scene, the scene perception response process is triggered.

[0033] The parameter pre-setting module presets individualized tactile feedback parameters according to the characteristics of the game scene and the user's preferences. When using a laser gun to shoot, set the feedback parameters as vibration frequency 50Hz, amplitude 1.0mm, pulse wave waveform, and duration 200ms. This parameter combination is calculated through a dynamic model that simulates the recoil force of a real gun, and the instantaneous impact of the pulse wave can enhance the realistic experience of shooting; when attacked by the enemy, use feedback parameters with a frequency of 30Hz, an amplitude of 1.2mm, an irregular waveform, and a duration of 300ms. The irregular waveform is generated by a chaotic system, simulating the chaotic vibration feeling when being hit.

[0034] The dynamic self-adaptive adjustment module adopts a reinforcement learning algorithm to realize intelligent optimization of the tactile feedback. The heart rate sensor is used to monitor the heart rate change (sampling frequency of 1 Hz), and game behavior data such as shooting frequency and evasion success rate are combined to construct a state space. When it is detected that the user continuously and rapidly shoots (shooting frequency > 5 times / s) and the heart rate increases (heart rate > 100 times / min), it indicates that the user is in a state of intense battle. A deep Q network (DQN) model calculates the optimal parameter adjustment strategy according to a reward function (such as the number of enemy kills and the self-blood volume retention rate), and the vibration frequency of the shooting operation is increased to 70 Hz and the amplitude is increased to 1.3 mm, thereby enhancing the battle immersion. If the user does not perform effective operation for a long time (more than 30 seconds without attack or evasion behavior), the tactile feedback strength is reduced to reduce unnecessary interference.

[0035] The multi-modal fusion module realizes deep cooperation of tactile, visual and auditory based on the rendering pipeline of the game engine. During shooting, the visual interface presents laser beam special effects and enemy spaceship damage pictures using physical-based rendering (PBR) technology, and simulates explosion debris through a particle system. The auditory system plays laser emission sound effects and enemy spaceship explosion sounds using a convolution reverberation algorithm, and the rhythm of tactile feedback is synchronized with the rhythm of sound effects through timestamp alignment technology. After the game ends, the model parameters are optimized using a genetic algorithm based on the evaluation results such as user game experience score, battle score, task completion time, etc., and the weights and biases of the neural network are adjusted to improve the adaptability of the model to different users and scenes. After optimization by the method, the player's immersion in the game is improved by 35%, the game operation response speed is improved by 25%, and the game interaction experience is significantly enhanced.

[0036] Embodiment 3: Car intelligent cockpit interaction scenario In the car intelligent cockpit, the user information collection module realizes comprehensive monitoring of the driving state through a distributed sensor network. The seat pressure sensor uses a capacitive sensing array covering 100 detection units on the seat surface to collect the body pressure distribution in real time with an accuracy resolution of 0.1 kPa, and extracts characteristic parameters such as pressure center offset and pressure uniformity through a principal component analysis (PCA) algorithm. The steering wheel grip force sensor is based on the strain gauge principle and can monitor the hand grip force change in real time, and analyzes the hand fatigue degree in combination with the holding time. At the same time, the system records driving habits such as commonly used air conditioning temperature adjustment frequency and multimedia operation preferences (music type, volume setting) through the vehicle entertainment system log.

[0037] The scene perception module fuses multi-source vehicle and environment data to accurately determine the driving scene. The vehicle-mounted accelerometer and gyroscope output the vehicle's acceleration and angular velocity information in real time through the strapdown inertial navigation algorithm; the GPS positioning module combines high-precision map data to obtain vehicle position, speed, road type and other information; and the environment sensors (light, rainfall, temperature, humidity) construct an environment state vector through a data fusion algorithm. When the vehicle is driving on a congested urban road (vehicle speed < 20 km / h, following distance < 5 m, environmental noise > 60 dB), the user adjusts the air conditioning temperature, triggering the scene response mechanism.

[0038] The parameter pre-setting module pre-sets comfortable haptic feedback parameters based on the driving scene and user preference model. The temperature increase operation sets the feedback with a vibration frequency of 80 Hz, an amplitude of 0.3 mm, a sine wave waveform, and a duration of 100 ms, and the temperature decrease operation adopts feedback with a frequency of 70 Hz, an amplitude of 0.25 mm, a sine wave waveform, and a duration of 90 ms. The parameters are designed by simulating the damping characteristics of traditional knob adjustment, and the smooth change of the sine wave can provide a delicate tactile experience.

[0039] The dynamic self-adaptive adjustment module uses a real-time decision tree algorithm to realize intelligent adjustment of haptic feedback. If it is detected that the user frequently adjusts the temperature (adjustment frequency > 3 times in 1 minute) and the hand muscle is tense (grip sensor data standard deviation > 10 N), it indicates that the user is not satisfied with the current temperature. The decision tree model predicts the user's preference based on historical adjustment data and environmental temperature, appropriately enhances the haptic feedback intensity and duration, and simultaneously highlights the temperature adjustment progress bar using a gradual animation on the visual interface and plays a voice prompt sound with a directional prompt on the auditory interface, thereby multi-modal collaborative guiding the user's operation.

[0040] After the driving is completed, the model is optimized using a gradient descent algorithm based on the user's satisfaction evaluation of the cockpit interaction system, the number of operation errors and other evaluation results. After using the method of the present application, the interactive operation error rate of the driver in a complex driving scene is reduced by 20%, and the interactive operation efficiency is improved by 28%, effectively ensuring driving safety and comfort.

[0041] Example 4: Intelligent watch health monitoring interaction scene In the smart watch health monitoring scenario, the user information collection module uses miniaturized sensor technology to achieve efficient collection of physiological and behavioral data. The built-in heart rate sensor uses reflective PPG technology, combining a green light LED and a photodiode to monitor heart rate changes in real time at a sampling frequency of 1 Hz during exercise; the galvanic skin response sensor is based on bioelectricity detection principles and can detect microvolt-level changes in the skin electricity signal to analyze the galvanic skin response under stress. At the same time, the system records user usage habits through the watch application log, such as the frequency of viewing health data, and the preference for receiving message reminders (vibration, ringtone, silent).

[0042] The scene perception module combines motion sensor data and time information to determine the user's usage scenario. The accelerometer determines the user's activity state (still, walking, running) through a threshold detection algorithm, and combines time information (such as 6-8 am) to determine the user's scenario of viewing exercise data in the morning.

[0043] When the user clicks on the watch screen to view heart rate data during running, the parameter pre-setting module presets clear tactile feedback. The click operation sets the feedback with a vibration frequency of 100 Hz, an amplitude of 0.35 mm, a square wave waveform, and a duration of 120 ms, simulating a button confirmation feeling; when the heart rate data is refreshed, the feedback is set with a frequency of 60 Hz, an amplitude of 0.2 mm, a sine wave waveform, and a duration of 80 ms, prompting data updates. This parameter setting is optimized through ergonomic experiments to ensure that users can clearly perceive feedback in a motion environment.

[0044] The dynamic self-adaptive adjustment module uses an adaptive filtering algorithm to achieve dynamic optimization of tactile feedback. If it is detected that the user's running speed is increasing (accelerometer data integral calculation speed > 5 m / s) and the heart rate is rising (heart rate > 130 beats per minute), when displaying the heart rate data, the adaptive gain control algorithm is used to enhance the intensity of tactile feedback, while the visual interface uses color coding technology to display the heart rate value in a more eye-catching color, and the auditory system plays a variable pitch voice broadcast, with multiple modalities working together to allow users to quickly obtain key information.

[0045] After the user uses it, the model is optimized using a Bayesian optimization algorithm based on evaluation results such as satisfaction scores for health monitoring interactions and information acquisition accuracy. Using the method of the present application, the efficiency of users obtaining health information in a motion scenario is improved by 30%, and the ease of use of smart watch interactions is significantly improved.

[0046] Example 5: Industrial equipment operation interaction scenario In the industrial production line equipment operation, the user information acquisition module realizes the accurate monitoring of the state of the operator through professional industrial sensors. The pressure sensor on the operation handle adopts a piezoresistive sensing unit, which can withstand a pressure range of 0-100N, and can collect the hand force situation in real time when the operator adjusts the parameters with an accuracy resolution of 0.5N; the electromyography sensor adopts a wired differential electrode, which extracts the hand muscle fatigue degree characteristic parameters (such as average power frequency, integral electromyography value) caused by long-time operation after signal amplification and filtering processing. At the same time, the system records the operation habits through the industrial control system log, such as the commonly used device start, stop, parameter adjustment operation mode.

[0047] The scene perception module utilizes the sensor network built-in the device to realize accurate judgment of the operation scene. The temperature sensor, vibration sensor, and current sensor monitor the device running state (normal operation, fault warning); the environmental sensor (temperature and humidity, dust concentration) transmits data in combination with the industrial Internet of Things protocol (such as OPC UA) to judge the operation scene, such as the device maintenance and repair scene (device shutdown, environmental dust concentration > 5mg / m³).

[0048] When the operator adjusts the parameters during device maintenance and repair, the parameter pre-setting module pre-sets clear tactile feedback. The parameter increase operation sets the feedback of vibration frequency 40Hz, vibration amplitude 0.8mm, pulse wave waveform, and duration 150ms, and the parameter decrease operation sets the feedback of frequency 35Hz, amplitude 0.7mm, pulse wave waveform, and duration 140ms, which simulates the resistance feeling of traditional knob adjustment. The waveform parameters are optimized through finite element analysis to ensure that the operator can clearly perceive the operation feedback in the industrial environment.

[0049] The dynamic self-adaptive adjustment module adopts an anomaly detection algorithm to realize intelligent warning of the tactile feedback. If it is detected that the operator continuously adjusts the parameters incorrectly (the parameters still do not reach the target value after 3 adjustments) and the hand muscle tension increases significantly (the integral electromyography value exceeds 1.5 times the threshold value), a special tactile feedback mode is triggered to warn the operation error with high-intensity, low-frequency vibration (frequency 20Hz, amplitude 1.2mm, duration 200ms), and a red prompt box with a fault code is popped up on the visual interface, and a high-decibel alarm sound is played, which timely reminds the operator to correct the error in multiple modes.

[0050] After the operation is completed, the evaluation results such as operation accuracy, task completion time, and operator feedback are used to optimize the model by using the simulated annealing algorithm. The method of the present application reduces the error rate of industrial equipment operation by 32%, improves the device maintenance efficiency by 25%, and effectively guarantees the safety and efficiency of industrial production.

[0051] Embodiment 6: Voice interaction assistance in smart home scenarios In the smart home environment, the user information collection module realizes deep analysis of user voice features through the microphone array of the smart speaker combined with voice signal processing technology. The target voice signal is enhanced using beamforming algorithm, the voice features are extracted using Mel frequency cepstral coefficient (MFCC), and the user emotional state is analyzed using a deep learning model (such as ResNet-LSTM) to judge the voice tone. At the same time, the system records the user's usage habits through the smart home control log, such as commonly used voice control instructions (turn on the light, adjust the air conditioner temperature), and usage time.

[0052] The scene perception module fuses multi-type sensor data to realize accurate identification of home scenes. The door and window sensor detects the door and window state through a magnetic reed switch, the temperature and humidity sensor collects real-time indoor environmental parameters, and the human body infrared sensor judges the personnel activity situation, and the fuzzy reasoning algorithm is used to judge the home scene, such as the user's scene after coming home at night (time > 22:00, door and window closed, personnel stationary time > 10 minutes).

[0053] When the user's voice instruction "turn off the living room light", the parameter pre-setting module pre-sets a soft tactile feedback. The instruction receiving confirmation sets the feedback (implemented through a smart bracelet or a mobile phone) with a vibration frequency of 70Hz, an amplitude of 0.25mm, a sine wave waveform, and a duration of 100ms. When the light is turned off, a feedback with a frequency of 60Hz, an amplitude of 0.2mm, a sine wave waveform, and a duration of 80ms is used to prompt the successful operation. The parameter setting is optimized through user experience testing to ensure that it does not cause interference in a quiet home environment.

[0054] The dynamic self-adaptive adjustment module uses a combination of natural language processing and machine learning to achieve intelligent adjustment of tactile feedback. If the user's voice instruction is detected to be ambiguous (voice recognition confidence <0.7) or repeated multiple times (the same instruction is repeated >2 times), the recurrent neural network (RNN) is used to analyze the user's historical instruction data to enhance the tactile feedback intensity and duration, and the instruction analysis result is displayed on the visual interface (smart screen or mobile phone APP) using animation effects, and the repeated voice prompts are heard, and the multi-modal collaboration helps the user to accurately complete the operation.

[0055] After the user uses it, the model is optimized using evolutionary strategy algorithm based on the evaluation results such as the satisfaction score of smart home voice interaction and the operation success rate. Using the method of the present application, the success rate of smart home voice interaction is improved by 28%, and the user's operation convenience and home smart experience are significantly enhanced.

[0056] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A haptics feedback directed interface design parameterization method, characterized in that, Comprise the following steps: S1: Collecting user physiological characteristic information by using high-precision sensors, the physiological characteristic information collection includes capturing skin sensitivity changes by skin electric response sensors, monitoring hand muscle response by electromyography sensors; at the same time, collecting user usage habit information by using system log records and user behavior monitoring software, the usage habit information includes commonly used operation types in different application programs, operation frequency, and user subjective evaluation of past haptic feedback; and perceiving device posture, ambient light intensity and ambient noise information through the built-in accelerometer, gyroscope, ambient light sensor and microphone of the device to determine the use scene of the user; S2: According to the scene-parameter mapping rules constructed by a large number of experiments and data analysis in advance, preset the basic haptic feedback parameters for the real-time perceived scene, the basic haptic feedback parameters include physical layer parameters and perception layer parameters, wherein the vibration frequency of the physical layer parameters is set in the range of 10-200Hz, the amplitude adjustment range is 0.1-1.5mm, the waveform includes sine wave, square wave and pulse wave, and the duration is within 50-500ms; the perception layer parameters include adjusting the intensity according to the user's early feedback and the characteristics of the scene, determining the rhythm by setting multiple feedback time intervals, and designing a combination logic that conforms to the user's cognitive habits as a mode; S3: In the process of user interaction with the device, real-time monitoring of user operation behavior and physiological response is realized by using behavior analysis algorithm and physiological monitoring technology, when detecting changes in user operation speed and force, or detecting physiological response abnormalities such as increased heart rate and increased hand muscle tension through heart rate sensor and blood pressure sensor, starting the pre-trained deep learning model, taking user personalized information, current scene information and real-time operation behavior information as input, adjusting the haptic feedback parameters in real time through neural network operation, and outputting the optimal parameter combination that adapts to the current user state and scene; S4: By constructing a mathematical correlation model and an intelligent control algorithm, a dynamic correlation between haptic feedback parameters and visual and auditory parameters is established, forming a multi-modal parameter fusion model, when the user performs interactive operation, the haptic, visual and auditory parameters are adjusted synchronously according to the operation type and the current scene, and the haptic intensity, visual scaling amplitude and sound volume are positively correlated; S5: After each interactive operation is completed, the user's satisfaction score for the intensity, rhythm, mode, etc. of the haptic feedback this time is collected as a subjective evaluation through a user scoring system, at the same time, objective indexes such as operation accuracy, operation time, heart rate, blood pressure, skin electric response changes before and after operation are recorded by using the built-in operation recording tool of the system, based on the subjective and objective evaluation results, the user personalized model and the dynamic parameter adjustment model are optimized and updated by using model optimization algorithm.

2. The haptic feedback directed interface design parameterization method of claim 1, wherein, The establishment of the user personalized information is obtained by analyzing and processing the collected user physiological characteristic information and usage habit information by using machine learning algorithm.

3. The haptic feedback directed interface design parameterization method of claim 1, wherein, The construction of the scene-parameter mapping rule is determined based on a large amount of user test data in different scenes, analysis of the correlation between each scene and the haptic feedback parameter, and the like.

4. The haptic feedback directed interface design parameterization method of claim 1, wherein, The training process of the deep learning model includes collecting a large amount of user operation behavior data, physiological response data, scene data and corresponding haptic feedback parameter adjustment records in different scenes, taking the above data as training samples, optimizing the parameters of the deep learning model through multiple iterations, and accurately adjusting the haptic feedback parameter.

5. The haptic feedback directed interface design parameterization method of claim 1, wherein, In the multi-modal parameter fusion model, the collaborative adjustment of the haptic, visual and auditory parameters is specifically as follows: in the button clicking operation, the haptic feedback simulates the physical button touch feeling, the visual interface presents the button pressing zooming animation, and the auditory system plays the clicking sound effect; in the page sliding operation, the haptic feedback simulates the sliding resistance change, the visual interface displays the page sliding effect, and the auditory system plays the corresponding sliding sound effect, and each modal parameter is dynamically adjusted according to the operation force and speed.

6. The haptic feedback directed interface design parameterization method of claim 1, wherein, The optimization and update of the user individualization model and the dynamic parameter adjustment model include adjusting the parameter weight in the model and correcting the model structure according to the evaluation result, so as to improve the accuracy of the haptic feedback parameter adjustment and the user satisfaction.

7. The haptic feedback directed interface design parameterization method of claim 1, wherein, In the user and scene information collection step, the user individualization information is supplemented and improved through user registration information, historical interaction records and the like.

8. The haptic feedback directed interface design parameterization method of claim 1, wherein, In the dynamic self-adaptive adjustment step, when it is detected that the user is in a specific operation state, such as continuous error operation or long-time no operation, a special haptic feedback parameter adjustment strategy is triggered.

Images