Electrostatic force tactile rendering method and system based on physiology and physics modeling

Abstract

The invention provides an electrostatic force tactile rendering method and system based on physiology and physics modeling, and belongs to the technical field of virtual reality and man-machine interaction, and the method comprises the steps: responding to tactile event triggering, matching tactile scene nerve pulse features corresponding to a tactile event from a nerve pulse database, the nerve pulse database comprises a plurality of tactile scene nerve pulse features; generating a neural pulse signal based on the neural pulse feature, and converting the neural pulse signal into a driving parameter corresponding to the electrostatic force tactile representation device according to a preset rule; and controlling the electrostatic force tactile representation device to generate a target tactile sense corresponding to the tactile event according to the driving parameter. According to the electrostatic force tactile rendering method and system based on physiology and physics modeling, the authenticity of tactile representation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of virtual reality and human-computer interaction, in particular to a static electric force tactile rendering method and system based on physiological physical modeling. BACKGROUND

[0002] Tactile reproduction is a cutting-edge technology in the field of human-computer interaction and virtual reality, which can provide better immersive experience for people. Compared with traditional vision and hearing, touch pays more attention to the interactivity between man and machine. Among them, static electric tactile reproduction technology is currently a research hotspot and focus in the field of international tactile reproduction, and has broad development prospects in the fields of education and teaching, business display, medical treatment and entertainment.

[0003] In the existing tactile rendering algorithm of the static electric force tactile reproduction device, the tactile stimulation is directly simulated based on physical modeling, ignoring the physiological mechanism of human tactile perception, resulting in insufficient tactile reality, especially in complex tactile scenes, it is difficult to accurately match the user's perception needs.

[0004] Therefore, there is an urgent need for a static electric force tactile rendering method based on physiological physical modeling to improve the reality of tactile reproduction. SUMMARY

[0005] In order to solve the above technical problems, the present application provides a static electric force tactile rendering method and system based on physiological physical modeling to improve the reality of tactile reproduction.

[0006] The first aspect of the embodiment of the present application provides a static electric force tactile rendering method based on physiological physical modeling, comprising: In response to a tactile event trigger, matching a tactile scene neural pulse feature corresponding to the tactile event from a neural pulse database, the neural pulse database comprising a plurality of tactile scene neural pulse features; Generating a neural pulse signal based on the neural pulse feature, and converting the neural pulse signal into a corresponding driving parameter of a static electric force tactile reproduction device according to a predetermined rule; Controlling the static electric force tactile reproduction device to generate a target tactile sensation corresponding to the tactile event according to the driving parameter.

[0007] The second aspect of the embodiment of the present application provides a static electric force tactile rendering system based on physiological physical modeling, comprising: The feature matching module is configured to match a tactile scene neural pulse feature corresponding to the tactile event from a neural pulse database in response to a tactile event trigger, the neural pulse database comprising a plurality of tactile scene neural pulse features; a parameter conversion module, configured to generate a neural pulse signal based on the neural pulse feature, and convert the neural pulse signal into a driving parameter corresponding to the electrostatic force tactile reproduction device according to a preset rule; a parameter execution module, configured to control the electrostatic force tactile reproduction device to generate a target tactile sensation corresponding to the tactile event according to the driving parameter.

[0008] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the above-mentioned electrostatic force tactile rendering method based on physiological physical modeling when running the computer program.

[0009] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the above-mentioned electrostatic force tactile rendering method based on physiological physical modeling when executed by a processor.

[0010] To sum up, the electrostatic force tactile rendering method and system based on physiological physical modeling provided by the embodiments of the present application have the following advantages: by constructing a neural pulse database and extracting time-frequency domain features, the present application can accurately match the tactile scene and physiological neural response characteristics, effectively overcome the problem of disconnection between traditional physical modeling and perception, and make the tactile sensation more suitable for human real perception needs. Secondly, the present application uses a feature extraction method based on wavelet transform to reduce the noise interference of the pulse signal, improves the robustness of the pulse signal feature, and ensures the reliability of the data. Based on the mapping relationship between the preset neural pulse signal and the driving parameter, the present application can quickly generate personalized tactile feedback. In summary, by combining the neural signal feature and the electrostatic force driving technology, the present application improves the authenticity and adaptability of tactile reproduction. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 a flowchart of the electrostatic force tactile rendering method based on physiological physical modeling provided by an embodiment of the present application; Figure 2 a structural block diagram of the electrostatic force tactile rendering system based on physiological physical modeling provided by an embodiment of the present application; Figure 3 a schematic block diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0012] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0013] In order to make the objects, technical solutions and advantages of the present application clearer, the following will be combined with the accompanying drawings to make a detailed description. Figures 1-3 The present application is described by specific embodiments.

[0014] Reference will be made to Figure 1 , Figure 1 The flowchart of the electrostatic force tactile rendering method based on physiological physical modeling provided by an embodiment of the present application comprises the following steps. S101: In response to a tactile event trigger, matching the tactile scene neural pulse feature corresponding to the tactile event from the neural pulse database, the neural pulse database comprising a plurality of tactile scene neural pulse features.

[0015] In the present embodiment, the neural pulse database is obtained through a large number of neurophysiological experiments and data collection and analysis; when the tactile event occurs, the present application extracts the features of the tactile event, and compares them with the tactile scene neural pulse features in the neural pulse database through cosine similarity, Euclidean distance and other algorithms, to find the tactile scene neural pulse feature with the highest matching degree.

[0016] In the present embodiment, the tactile event is the starting condition of the whole tactile rendering method flow, which refers to the relevant behavior or situation that the user expects to obtain tactile feedback when interacting with virtual objects or interfaces in virtual or augmented reality interactive scenes. For example, in a virtual environment, the user "touches" the virtual object, "presses" the virtual button, "grabs" the virtual tool, and these interactive situations that require specific tactile feedback all belong to the tactile event.

[0017] In the present embodiment, the tactile event includes different types of interactive actions, each action corresponding to different tactile scene neural pulse features to achieve precise tactile feedback simulation.

[0018] In the embodiment, the construction process of the neural pulse database includes: using professional equipment to collect physiological experimental data of different material surface friction, pressure change, object shape perception and other tactile scenes; using machine learning algorithm to perform clustering analysis and feature extraction on the collected neural pulse data, and constructing a representative tactile scene neural pulse feature library; at the same time, in order to ensure the real-time and accuracy of the neural pulse database, the application also sets a regular updating mechanism to optimize the database by continuously incorporating new experimental data.

[0019] S102: generating a neural pulse signal based on the neural pulse feature, and converting the neural pulse signal into driving parameters corresponding to the electrostatic force tactile reproduction device according to a preset rule.

[0020] In the embodiment, the step of generating a neural pulse signal based on the neural pulse feature includes: constructing a biophysical neuron model, which is based on a neuron membrane potential dynamics equation; inputting the matched neural pulse feature parameters into the model, simulating the neuron discharge behavior through numerical calculation, and generating a neural pulse sequence with biological authenticity, i.e. the neural pulse signal.

[0021] In the embodiment, the preset rule is a mapping relationship derived by establishing a mathematical model based on the physical characteristics and working principle of the electrostatic force tactile reproduction device. Specifically, the frequency, amplitude and other parameters of the neural pulse signal are mapped to the driving voltage, current intensity, pulse width and other driving parameters of the electrostatic force tactile reproduction device; therefore, the embodiment can pre-establish a mapping relationship table of the neural pulse signal and the driving parameters, or a mathematical model based on the physical characteristics of the electrostatic force tactile reproduction device, which establishes the mapping relationship between the electrode material properties, electrode spacing, electric field strength of the device and the intensity and frequency of the neural pulse signal. The embodiment can also set an adaptive adjustment factor in the mathematical model to dynamically correct the driving parameters according to external conditions such as environmental temperature and humidity.

[0022] S103: controlling the electrostatic force tactile reproduction device to generate a target tactile sensation corresponding to the tactile event according to the driving parameters.

[0023] In this embodiment, the method controls the electrostatic force haptic reproduction device according to the generated driving parameters, so that the electrostatic force is generated on the surface of the user's skin, and then the target tactile sensation corresponding to the haptic event is generated. Wherein the electrostatic force haptic reproduction device is composed of an electrode array, by applying different driving parameters to each electrode, the simulation of complex tactile sensation is realized; whether it is to simulate the texture details of the object, or to restore the size and direction change of the force, it can be achieved by accurately controlling the electrostatic force, so as to bring the user an immersive tactile experience. In addition, in the process of generating the tactile sensation, the working state and output effect of the device are also monitored in real time, and the driving parameters are dynamically adjusted and optimized through the feedback mechanism, so as to ensure that the generated target tactile sensation can meet the expectation to the greatest extent, and bring the user a realistic tactile experience.

[0024] From the above, it can be concluded that by constructing a neural pulse database and extracting time-frequency domain features, the application can accurately match the haptic scene and physiological neural response characteristics, effectively overcome the problem of disconnection between traditional physical modeling and perception, and make the tactile sensation more in line with the real perception needs of the human body. Secondly, the feature extraction method based on wavelet transform reduces noise interference and improves the robustness of pulse signal features, ensuring the reliability of the data. Finally, based on the dynamic mapping relationship between neural pulse signals and driving parameters, personalized haptic feedback can be quickly generated. In summary, by combining neural signal features and electrostatic force driving technology, the application improves the authenticity and adaptability of haptic reproduction.

[0025] In an embodiment of the application, the method for constructing a neural pulse database comprises: Collecting pulse signal data generated by fingertip tactile nerves of the user when operating in a haptic scene; Extracting features from the pulse signal data to obtain pulse feature data, the pulse feature data including: pulse interval sequence and frequency band energy distribution characteristics; Inputting the pulse feature data into a machine learning algorithm for classification to obtain a classification result associated with the haptic scene; Constructing a neural pulse database including neural pulse features of multiple haptic scenes according to the classification result.

[0026] In this embodiment, a high-density microelectrode array or a neural interface chip is used to collect pulse signal data generated by fingertip tactile nerves of the user when operating in different haptic scenes through transcutaneous implantation or non-invasive epidermal electrode array. The haptic scene includes but is not limited to: sliding and pressing on the surface of different materials (such as glass, rubber, sandpaper), gripping different shaped objects (cylinders, spheres, prisms), touching different textures (smooth, rough, granular), etc.

[0027] In the embodiment, preprocessing is required for the collected original pulse signal data before pulse feature data extraction; wherein the preprocessing operation includes: noise removal, filtering, signal enhancement and the like. The embodiment extracts features from the pulse signal data to obtain pulse feature data, including: calculating the pulse interval sequence and extracting its statistical features; at the same time, the signal is subjected to short-time Fourier transform or wavelet packet decomposition, and the energy distribution features of different frequency bands are calculated to obtain the pulse feature data. Wherein, the pulse interval sequence reflects the time pattern of neural pulse firing, and different tactile perception corresponds to a unique pulse interval rule; the frequency band energy distribution feature analyzes the proportion of energy in each frequency band by decomposing the signal into different frequency bands, and captures the frequency characteristics of the neural signal.

[0028] In the embodiment, the pulse feature data is input into a machine learning algorithm for classification to obtain the classification result of the associated tactile scene. Specifically, support vector machines, random forests or deep learning models are used to train and classify the pulse feature data. In the training process, the labeled pulse feature data samples are used to let the model learn the mapping relationship between different tactile scenes and pulse features; through cross-validation and parameter optimization, the accuracy and generalization ability of the classifier are improved, and finally the unknown pulse feature data belonging to the tactile scene is accurately classified.

[0029] The embodiment constructs a database based on real user neural pulse data, avoiding the limitations of traditional tactile rendering relying on physical models, and better meeting the human perception characteristics; secondly, by extracting the pulse interval sequence and the frequency band energy distribution feature, the time and frequency characteristics of the neural signal are comprehensively captured, and the scene differentiation accuracy is improved. The embodiment also uses machine learning algorithms to automatically associate tactile scenes, reducing the cost of manual annotation.

[0030] In an embodiment of the present application, feature extraction is performed on the pulse signal data, including: Wavelet transform algorithm is used for feature extraction of the pulse signal data; wherein, The wavelet basis function of the wavelet transform algorithm is determined based on the noise type of the pulse signal data and the frequency component of the pulse signal data; The decomposition level of the wavelet transform algorithm is determined based on the noise type of the pulse signal data and the frequency of the pulse signal data; The discrimination threshold of the wavelet transform algorithm is determined based on the noise type of the pulse signal data and the required accuracy; The pulse signal data is extracted based on the wavelet basis function, the decomposition level and the discrimination threshold.

[0031] In the embodiment, the wavelet base function of the wavelet transform algorithm is determined based on the noise type of the pulse signal data and the frequency component of the pulse signal data, which specifically includes that, for a neural pulse signal of Gaussian white noise, a Daubechies (dbN) series wavelet base function is preferentially selected, where the value of N is determined according to the signal frequency component; when the signal is mainly composed of low frequency components, a smaller N value is selected; when the signal contains more high frequency components, a larger N value (such as db8) is selected.

[0032] In the embodiment, the main frequency range of the signal is determined by analyzing the spectral characteristics of the signal, and then the appropriate decomposition layer number is selected according to the frequency band division rule of the wavelet transform, so as to ensure that the neural pulse characteristics can be effectively separated into different frequency bands.

[0033] In the embodiment, the soft threshold or hard threshold method can be used to remove noise interference from the wavelet coefficients; wherein the size of the threshold is determined by statistical analysis of the standard deviation of the noise, and is adjusted by the noise level and the signal characteristics.

[0034] In the embodiment, the specific steps of feature extraction of the pulse signal data based on the wavelet base function, the decomposition layer number and the threshold value are as follows: wavelet decomposition is performed on the pulse signal data to obtain wavelet coefficients of each layer; the wavelet coefficients are threshold processed according to the set threshold value, the coefficients greater than the threshold value are retained, and the small coefficients caused by noise are removed; the processed wavelet coefficients are reconstructed to obtain the denoised pulse signal; the pulse interval sequence and the frequency band energy distribution characteristics are extracted from the pulse signal; wherein the pulse interval sequence is obtained by detecting the peak value in the pulse signal and calculating the time interval between adjacent peak values; the frequency band energy distribution characteristics are obtained by calculating the energy distribution of the wavelet coefficients of each decomposition layer.

[0035] In the embodiment, the wavelet base function and the decomposition layer number are selected according to the noise type, which effectively filters out environmental interference and retains the characteristics of the real neural signal; the feature extraction accuracy is adjusted by the threshold value, and the calculation efficiency and the feature integrity are balanced.

[0036] In an embodiment of the present application, in response to a haptic event trigger, the neural pulse database is matched with the haptic scene neural pulse characteristics corresponding to the haptic event, including: Calculate a plurality of target similarities of the haptic event and each haptic scene neural pulse characteristic in the neural pulse database; Sort the plurality of target similarities, and determine the haptic scene neural pulse characteristics corresponding to the haptic event according to the sorting result.

[0037] In the embodiment, when calculating the multiple target similarities of the haptic event and the haptic scene neural pulse features in the database, the physical parameters (such as contact force, contact area, action time, etc.) contained in the haptic event and the physiological perception parameters are taken as the dimensions of the feature vector; for each haptic scene neural pulse feature in the database, the cosine similarity formula is used to calculate the similarity of each dimension between the haptic event feature vector and the haptic scene neural pulse feature; in addition, the dynamic time warping algorithm can also be introduced to calculate the similarity of the haptic event and the haptic scene neural pulse feature in the time sequence, so as to adapt to the characteristics of different haptic events in the time variation.

[0038] In the embodiment, when sorting the multiple target similarities, various sorting strategies can be used; if the sorting efficiency is required to be higher, the quicksort algorithm can be used to quickly arrange the multiple target similarities in descending or ascending order; if the size of the neural pulse database is small, the insertion sort algorithm can be used. When the embodiment determines the haptic scene neural pulse feature corresponding to the haptic event according to the sorting result, the haptic scene neural pulse feature with the highest similarity is selected; or multiple high-similarity features are fused to generate more rich and real haptic feedback.

[0039] The application selects the optimal match by sorting the target similarities, ensures that the most suitable feature for the haptic event is quickly and accurately found in a large amount of neural pulse feature data, and then provides highly adaptive neural pulse data for subsequent electrostatic force haptic rendering, thereby improving the rendering accuracy and realism of the electrostatic force haptic rendering method based on physiological and physical modeling, and enhancing the user's haptic experience.

[0040] In an embodiment of the application, calculating the multiple target similarities of the haptic event and the haptic scene neural pulse features in the database comprises: For each haptic scene neural pulse feature: calculating the time domain similarity of the haptic event and the haptic scene neural pulse feature; calculating the frequency domain similarity of the haptic event and the haptic scene neural pulse feature; weighting and fusing the time domain similarity and the frequency domain similarity to obtain the target similarity.

[0041] In the embodiment, for each haptic scene neural spike feature in the neural spike database, firstly, the time domain similarity of the haptic event and the haptic scene neural spike feature is calculated; wherein the time domain similarity is used to measure the similarity of the haptic event and the haptic scene neural spike feature in the time dimension. Since the original haptic event data and the haptic scene neural spike feature data may be interfered by noise, and the parameters such as the scale and length of the data are inconsistent, the two data need to be preprocessed before calculation, wherein the preprocessing operation includes filtering processing and normalization operation. In the embodiment, the cross-correlation calculation method is used to calculate the cross-correlation value of the haptic event and the haptic scene neural spike feature based on the preprocessed haptic event data and the haptic scene neural spike feature data, wherein the cross-correlation value can represent the similarity of two signals at different time delays. The maximum cross-correlation value is taken and divided by the norm product of the two signals to obtain the final time domain similarity. In addition, the dynamic time warping algorithm can also be used to calculate the time domain similarity of the haptic event data and the haptic scene neural spike feature data.

[0042] Secondly, the frequency domain similarity of the haptic event and the haptic scene neural spike feature is calculated, wherein the frequency domain similarity is used to explore the similarity of the haptic event and the haptic scene neural spike feature in the frequency component. In the embodiment, the Fourier transform is performed on the preprocessed haptic event data and the haptic scene neural spike feature data respectively to convert the time domain signal into the frequency domain signal and obtain the respective frequency spectrum distribution; the Euclidean distance or the cosine similarity is used to compare the amplitude difference of the same frequency component in the frequency spectrum of the two to calculate the frequency domain similarity.

[0043] Finally, the time domain similarity and the frequency domain similarity are fused by weighting to obtain the target similarity. Since the time domain similarity and the frequency domain similarity reflect the similarity of the haptic event and the haptic scene neural spike feature from different angles, both of them are important for judging the scene to which the haptic event belongs, but their importance may be different in different application scenarios. Therefore, according to the actual demand, appropriate weights are given to the time domain similarity and the frequency domain similarity, and the two are fused in the form of weighted summation to obtain a target similarity which comprehensively reflects the similarity of the two. The target similarity can more comprehensively and accurately measure the similarity between the haptic event and the haptic scene neural spike feature, and provide a reliable basis for subsequent haptic scene recognition, classification and other applications based on similarity.

[0044] In the application, the similarity between the haptic event and the neural spike feature is more comprehensively described through the time domain analysis to capture the similarity in the time sequence and the frequency domain analysis to mine the correlation of the frequency component, the identification of the haptic scene is more accurate, the haptic scenes with slight differences can be distinguished, and the recognition error is reduced.

[0045] In an embodiment of the present application, the time domain similarity and the frequency domain similarity are weighted and fused to obtain a target similarity, comprising: In response to the temperature data around the user being greater than a first temperature threshold, adjusting the first weight reference value based on a first weight adjustment step to obtain a first weight, and adjusting the second weight reference value based on the first weight adjustment step to obtain a second weight; Based on the first weight, the second weight, the time domain similarity and the frequency domain similarity, a target similarity is obtained by weighted calculation; The first weight is a weight corresponding to the time domain similarity, and the second weight is a weight corresponding to the frequency domain similarity. The adjustment directions of the first weight reference value and the second weight reference value are different.

[0046] In the present embodiment, the temperature data around the user is collected in real time by a temperature collection device, and compared with a preset first temperature threshold. When the temperature data is greater than the first temperature threshold, the weight adjustment logic is triggered so that the present application can adapt to the physiological changes of tactile perception in different temperature environments.

[0047] In the present embodiment, in response to the temperature data around the user being greater than a first temperature threshold, it indicates that the ambient temperature is high, at this time the time domain characteristics of the pulse signal will be affected, and the weight distribution of the time domain similarity and the frequency domain similarity needs to be adjusted. The specific adjustment method is as follows: increasing the first weight reference value based on a first weight adjustment step to obtain a first weight, and decreasing the second weight reference value based on the first weight adjustment step to obtain a second weight; wherein the first weight is a weight corresponding to the time domain similarity, and the second weight is a weight corresponding to the frequency domain similarity. The adjustment directions of the first weight reference value and the second weight reference value are different, and satisfy.

[0048] In the present embodiment, further, in response to the temperature data around the user being less than a second temperature threshold (the second temperature threshold is less than the first temperature threshold), it indicates that the ambient temperature is low, at this time the frequency domain characteristics of the signal may be more affected, and the adjustment method is: decreasing the first weight reference value based on a first weight adjustment step to obtain a first weight, and increasing the second weight reference value based on the first weight adjustment step to obtain a second weight; wherein the first weight is a weight corresponding to the time domain similarity, and the second weight is a weight corresponding to the frequency domain similarity. The adjustment directions of the first weight reference value and the second weight reference value are different.

[0049] The specific adjustment method is as follows: decreasing the first weight reference value based on a first weight adjustment step to obtain a first weight, and increasing the second weight reference value based on the first weight adjustment step to obtain a second weight; wherein the first weight is a weight corresponding to the time domain similarity, and the second weight is a weight corresponding to the frequency domain similarity. The adjustment directions of the first weight reference value and the second weight reference value are different.

[0050] The application can adjust the weights of time domain similarity and frequency domain similarity according to the temperature around the user by introducing an ambient temperature sensing mechanism, and considers the influence of environmental factors on human tactile perception. Changes in skin temperature can significantly affect the discharge characteristics of tactile nerve fibers and the perception threshold of humans to tactile stimulation. For example, in a warm environment, the human perception of high-frequency vibrations may be enhanced. Therefore, by adjusting the temperature adaptive weight, the embodiment can make the tactile rendering system better match the tactile perception characteristics of the human body in different temperature environments, and improve the realism and accuracy of tactile feedback.

[0051] In an embodiment of the present application, the preset rule includes a mapping relationship table of the neural pulse signal and the driving parameter; The neural pulse signal is converted into the corresponding driving parameter of the electrostatic force tactile reproduction device according to the preset rule, including: Determining the initial driving parameter of the neural pulse signal in the mapping relationship table of the neural pulse signal and the driving parameter; According to the skin humidity of the user, a corresponding correction strategy in the correction strategy library is selected, and the correction strategy library includes a plurality of correction strategies; According to the correction strategy, the initial driving parameter is corrected to obtain the corresponding driving parameter of the electrostatic force tactile reproduction device.

[0052] In the embodiment, the mapping relationship table of the neural pulse signal and the driving parameter is constructed by experimental data; the initial driving parameter vector includes the driving voltage and the driving frequency of the electrostatic force tactile reproduction device, etc. In the embodiment, the skin state is divided into a plurality of humidity intervals according to the humidity value; therefore, the correction strategy library includes a correction strategy corresponding to each humidity interval, wherein each correction strategy is a correction function for the initial driving parameter. In the embodiment, according to the detected skin humidity of the user, a corresponding correction strategy is called to correct the initial driving parameter to obtain the corrected driving parameter.

[0053] The application can compensate the tactile perception difference caused by environmental factors in real time by selecting a correction strategy based on skin humidity to correct the initial driving parameter to obtain the corrected driving parameter, and ensure the consistency and accuracy of tactile feedback under different humidity conditions.

[0054] The electrostatic force tactile rendering method based on physiological physics modeling corresponding to the above embodiment, Figure 2 A structural block diagram of an electrostatic force tactile rendering system based on physiological physics modeling is provided for an embodiment of the present application. For ease of illustration, only parts related to the embodiments of the present application are shown. For reference Figure 2 The electrostatic force tactile rendering system 20 based on physiological physics modeling includes a feature matching module 21, a parameter conversion module 22, and a parameter execution module 23.

[0055] The feature matching module 21 is configured to match, in response to the haptic event trigger, the haptic scene neural pulse feature corresponding to the haptic event from a neural pulse database, the neural pulse database including a plurality of haptic scene neural pulse features. The parameter conversion module 22 is configured to generate a neural pulse signal based on the neural pulse feature, and convert the neural pulse signal into a driving parameter corresponding to the electrostatic force haptic reproduction device according to a preset rule. The parameter execution module 23 is configured to control the electrostatic force haptic reproduction device to generate a target tactile sensation corresponding to the haptic event according to the driving parameter.

[0056] In an embodiment of the present application, the feature matching module 21 is further configured to: Collect pulse signal data generated by fingertip haptic nerves of the user when operating in the haptic scene; Extract features from the pulse signal data to obtain pulse feature data, the pulse feature data including a pulse interval sequence and a frequency band energy distribution feature; Input the pulse feature data into a machine learning algorithm to obtain a classification result associated with the haptic scene; Construct a neural pulse database including a plurality of haptic scene neural pulse features according to the classification result.

[0057] In an embodiment of the present application, the feature matching module 21 is configured to: Extract features from the pulse signal data using a wavelet transform algorithm; wherein, Determine a wavelet basis function of the wavelet transform algorithm based on a noise type of the pulse signal data and a frequency component of the pulse signal data; Determine a decomposition layer number of the wavelet transform algorithm based on the noise type of the pulse signal data and a frequency of the pulse signal data; Determine a threshold value of the wavelet transform algorithm based on the noise type of the pulse signal data and a required accuracy; Extract features from the pulse signal data based on the wavelet basis function, the decomposition layer number, and the threshold value.

[0058] In an embodiment of the present application, the feature matching module 21 is configured to: Calculate a plurality of target similarities between the haptic event and each haptic scene neural pulse feature in the neural pulse database; Sort the plurality of target similarities, and determine the haptic scene neural pulse feature corresponding to the haptic event according to a sorting result.

[0059] In an embodiment of the present application, the feature matching module 21 is configured to: For each haptic scene neural pulse feature: Calculate the time domain similarity of the haptic event and the haptic scene neural pulse feature; Calculate the frequency domain similarity of the haptic event and the haptic scene neural pulse feature; Weighted fusion of time domain similarity and frequency domain similarity to obtain target similarity.

[0060] In an embodiment of the present application, the feature matching module 21 is specifically configured to: In response to the temperature data around the user being greater than the first temperature threshold, adjust the first weight reference value based on the first weight adjustment step to obtain the first weight; adjust the second weight reference value based on the first weight adjustment step to obtain the second weight; Weighted calculation based on the first weight, the second weight, the time domain similarity and the frequency domain similarity to obtain the target similarity; The first weight is the weight corresponding to the time domain similarity, and the second weight is the weight corresponding to the frequency domain similarity. The adjustment direction of the first weight reference value and the second weight reference value is different.

[0061] In an embodiment of the present application, the parameter conversion module 22 is specifically configured to: The preset rule includes: a mapping relationship table of neural pulse signals and driving parameters; The neural pulse signal is converted into the corresponding driving parameter of the electrostatic force haptic reproduction device according to the preset rule, including: Determine the initial driving parameter of the neural pulse signal in the mapping relationship table of the neural pulse signal and the driving parameter; Select the corresponding correction strategy in the correction strategy library according to the skin humidity of the user, and the correction strategy library includes a plurality of correction strategies; According to the correction strategy, the initial driving parameter is corrected to obtain the corresponding driving parameter of the electrostatic force haptic reproduction device.

[0062] Referring to Figure 3 , Figure 3 The schematic block diagram of the electronic device provided in an embodiment of the present application is shown. As Figure 3 The electronic device 300 in the embodiment can include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The above-mentioned processors 301, input devices 302, output devices 303 and memories 304 complete mutual communication through a communication bus 305. The memory 304 is used to store a computer program, and the computer program includes program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. Among them, the processor 301 is configured to call the program instructions to execute the functions of each module in the above-mentioned device embodiments, such as Figure 2 The functions of the infrared module 21, the heat transfer module 22 and the target module 23 shown.

[0063] It should be understood that, in the embodiments of the present application, the processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0064] The input device 302 can include a touchpad, a fingerprint collection sensor (for collecting fingerprint information and direction information of a fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0065] The memory 304 can include a read-only memory and a random access memory, and provide instructions and data for the processor 301. A part of the memory 304 can also include a non-volatile random access memory. For example, the memory 304 can also store device type information.

[0066] In specific implementations, the processor 301, the input device 302 and the output device 303 described in the embodiments of the present application can perform any implementation described in the embodiments of the electrostatic force tactile rendering method based on physiological physical modeling provided by the embodiments of the present application, and can also perform the implementation of the electronic device described in the embodiments of the present application, which will not be described here.

[0067] In another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also instruct related hardware to complete the implementation. The computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0068] The computer readable storage medium can be an internal storage unit of the electronic device of any of the preceding embodiments, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0069] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0070] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here.

[0071] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; the division of the units is merely logical function division; an actual implementation can be divided into different units depending on actual conditions; or a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, or can be in electrical, mechanical or other forms.

[0072] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0073] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0074] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto; any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for electrostatic force haptics rendering based on physiologically physical modeling, characterized by, The method comprises the following steps: in response to a haptic event trigger, matching a haptic scene neural pulse feature corresponding to the haptic event from a neural pulse database, the neural pulse database comprising a plurality of haptic scene neural pulse features; generating a neural pulse signal based on the neural pulse feature, and converting the neural pulse signal into driving parameters corresponding to an electrostatic force haptic reproduction device according to a preset rule; controlling the electrostatic force haptic reproduction device to generate a target touch corresponding to the haptic event according to the driving parameters.

2. The method of claim 1, wherein the method further comprises: The construction method of the neural pulse database comprises: collecting pulse signal data generated by fingertip haptic nerves of a user when operating in a haptic scene; extracting features from the pulse signal data to obtain pulse feature data, the pulse feature data comprising: pulse interval sequence and frequency band energy distribution characteristics; inputting the pulse feature data into a machine learning algorithm for classification to obtain a classification result associated with the haptic scene; constructing a neural pulse database comprising a plurality of haptic scene neural pulse features according to the classification result.

3. The method of claim 2, wherein the electrostatic force haptics rendering is based on a physiological physics modeling. The feature extraction of the pulse signal data comprises: using a wavelet transform algorithm to extract features from the pulse signal data; wherein, determining a wavelet basis function of the wavelet transform algorithm based on the noise type of the pulse signal data and the frequency component of the pulse signal data; determining the decomposition level of the wavelet transform algorithm based on the noise type of the pulse signal data and the frequency of the pulse signal data; determining the threshold of the wavelet transform algorithm based on the noise type of the pulse signal data and the required accuracy; extracting features from the pulse signal data based on the wavelet basis function, the decomposition level and the threshold.

4. The method of claim 1, wherein the method further comprises: The method comprises the following steps: calculating a plurality of target similarities between the haptic event and each haptic scene neural pulse feature in the neural pulse database; sorting the plurality of target similarities, and determining the haptic scene neural pulse feature corresponding to the haptic event according to the sorting result.

5. The method of claim 4, wherein the electrostatic force haptics rendering is based on a physiological physics modeling. The method comprises the following steps: for each haptic scene neural pulse feature: calculating the time domain similarity between the haptic event and the haptic scene neural pulse feature; calculating the frequency domain similarity between the haptic event and the haptic scene neural pulse feature; performing weighted fusion on the time domain similarity and the frequency domain similarity to obtain the target similarity.

6. The method of claim 5, wherein the electrostatic force haptics rendering is based on a physiological physics modeling. The method comprises the following steps: in response to the temperature data around the user being greater than a first temperature threshold, adjusting a first weight reference value based on a first weight adjustment step to obtain a first weight, and adjusting a second weight reference value based on a second weight adjustment step to obtain a second weight; performing weighted calculation based on the first weight, the second weight, the time domain similarity and the frequency domain similarity to obtain the target similarity; The first weight is a weight corresponding to the time domain similarity, the second weight is a weight corresponding to the frequency domain similarity, and the first weight reference value and the second weight reference value have different adjustment directions.

7. The method of claim 1, wherein the method further comprises: The preset rule includes a mapping relationship table of the neural pulse signal and the driving parameter. The converting the neural pulse signal into the driving parameter corresponding to the electrostatic force tactile reproduction device according to the preset rule includes: Determining an initial driving parameter of the neural pulse signal in the mapping relationship table of the neural pulse signal and the driving parameter; Selecting a corresponding correction strategy in a correction strategy library according to the skin humidity of the user, the correction strategy library including a plurality of correction strategies; Correcting the initial driving parameter according to the correction strategy to obtain the driving parameter corresponding to the electrostatic force tactile reproduction device.

8. An electrostatic force haptics rendering system based on physiologically physical modeling, characterized by, It includes: A feature matching module configured to match, in response to a haptic event trigger, a haptic scene neural pulse feature corresponding to the haptic event from a neural pulse database, the neural pulse database including a plurality of haptic scene neural pulse features; A parameter conversion module configured to generate a neural pulse signal based on the neural pulse feature and convert the neural pulse signal into a driving parameter corresponding to an electrostatic force tactile reproduction device according to a preset rule; A parameter execution module configured to control the electrostatic force tactile reproduction device to generate a target touch corresponding to the haptic event according to the driving parameter.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images