Virtual reality material perception simulation method and system based on dynamic auditory feedback

Abstract

The invention discloses a virtual reality material perception simulation method and system based on dynamic auditory feedback, and the method comprises the steps: obtaining real-time pressure received by a pressure sensor, and judging a pressure region of the real-time pressure; selecting a target function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the target function; selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate and a target center frequency adjustment function; and carrying out volume change according to the current volume gain value, carrying out frequency change on a target audio type according to the target center frequency, and obtaining a target sound according to a result after the volume change and a result after the frequency change. According to the method, the playing parameters of the audio are correspondingly adjusted through the pressure, so that multi-material touch simulation is realized.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality interaction technology, and in particular to a virtual reality material perception simulation method, system, terminal and computer-readable storage medium based on dynamic auditory feedback. Background Art

[0002] Material tactile simulation in virtual reality (VR) primarily relies on complex and costly physical tactile feedback devices (such as force feedback devices or surface texture simulation devices). However, technologies based on visual or tactile pseudo-feedback can also achieve the same effect.

[0003] However, in the current simulation of material tactile sensations in virtual reality, physical tactile devices are expensive and bulky, making them difficult to popularize and apply. Furthermore, the range of material simulation is limited, and it is impossible to flexibly switch between multiple material tactile sensations through a single material interface. Furthermore, the real-time dynamic matching technology of auditory feedback and tactile feedback also has certain technical constraints, resulting in poor material tactile simulation effects in virtual reality, making it difficult to meet user needs.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a virtual reality material perception simulation method, system, terminal and computer-readable storage medium based on dynamic auditory feedback. The purpose is to solve the problems in the existing technology of material tactile simulation in virtual reality. The physical tactile devices are high in cost and bulky, making them difficult to popularize and apply. The material simulation range is limited, and it is impossible to flexibly switch between multiple material tactile sensations through a single material interface. In addition, the real-time dynamic matching technology of auditory feedback and tactile feedback also has certain technical constraints, resulting in poor material tactile simulation effects in virtual reality and difficulty in meeting user needs.

[0006] To achieve the above-mentioned object, the present invention provides a method for simulating the perception of virtual reality materials based on dynamic auditory feedback, the method comprising the following steps:

[0007] Acquire the real-time pressure received by the pressure sensor, and determine the pressure zone of the real-time pressure;

[0008] selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function;

[0009] selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function;

[0010] The volume is changed according to the current volume gain value, the frequency of the target audio type is changed according to the target center frequency, and the target sound is obtained according to the results after the volume change and the results after the frequency change.

[0011] Optionally, the acquiring the real-time pressure received by the pressure sensor and determining the pressure zone of the real-time pressure specifically includes:

[0012] Acquire the real-time pressure received by the pressure sensor and the preset first pressure zone and second pressure zone;

[0013] The pressure zones of the real-time pressure are determined according to the real-time pressure, wherein the pressure zones include a first pressure zone and a second pressure zone.

[0014] Optionally, selecting a target function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the target function specifically includes:

[0015] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function;

[0016] When the pressure zone is the first pressure zone, the second quadratic piecewise function is selected as the objective function, and the target volume gain value is calculated according to the corresponding objective function.

[0017] Optionally, when the pressure zone is the first pressure zone, selecting a first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0018] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function;

[0019] Substituting the real-time pressure into the first quadratic piecewise function to obtain a corresponding target volume gain value;

[0020] Among them, the first quadratic piecewise function is:

[0021]

[0022] G l (F) represents the target volume gain value under the first pressure partition, F represents the real-time pressure, a1, a2, b1, b2, c1 and c2 represent the coefficients of the first quadratic piecewise function, F l min Indicates the starting threshold of the first pressure zone, F trans Indicates the starting point of the crossover transition area between the first pressure zone and the second pressure zone, F th Indicates the pressure threshold at which the second pressure zone begins to dominate.

[0023] Optionally, when the pressure zone is the first pressure zone, selecting the second quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0024] When the pressure zone is the second pressure zone, selecting a second quadratic piecewise function as the objective function;

[0025] Substituting the real-time pressure into the second quadratic piecewise function to obtain a corresponding target volume gain value;

[0026] Among them, the second quadratic piecewise function is:

[0027]

[0028] G h (F) represents the target volume gain value under the second pressure partition, F represents the real-time pressure, a3, a4, b3, b4, c3 and c4 represent the coefficients of the second quadratic piecewise function, Indicates the starting threshold of the second pressure zone, F max Indicates the maximum pressure threshold of the system, F th Indicates the pressure threshold at which the second pressure zone begins to dominate, G sat Indicates the upper limit of saturation gain.

[0029] Optionally, selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function specifically includes:

[0030] When the pressure zone is the first pressure zone, select the first pressure channel audio as the target audio type, select the first pressure zone center frequency decrease rate as the target frequency modulation rate, substitute the real-time pressure and the first pressure zone center frequency decrease rate into the target center frequency adjustment function, and calculate the corresponding target center frequency;

[0031] When the pressure zone is the second pressure zone, the second pressure channel audio is selected as the target audio type, and the center frequency decrease rate of the second pressure zone is selected as the target frequency modulation rate. The corresponding target center frequency is calculated by substituting the real-time pressure and the center frequency decrease rate of the second pressure zone into the target center frequency adjustment function.

[0032] Optionally, the step of changing the volume according to the current volume gain value, changing the frequency of the target audio type according to the target center frequency, and obtaining the target sound according to the result after the volume change and the result after the frequency change specifically includes:

[0033] Controlling the volume to change according to the current volume gain value to obtain a result after the volume change;

[0034] Performing frequency change on the target audio type according to the target center frequency to obtain a result after the frequency change;

[0035] Substitute the result after the volume change and the result after the frequency change into the final output formula to obtain the target sound.

[0036] In addition, to achieve the above-mentioned purpose, the present invention further provides a virtual reality material perception simulation system based on dynamic auditory feedback, wherein the virtual reality material perception simulation system based on dynamic auditory feedback includes:

[0037] a pressure zone determination module, configured to obtain the real-time pressure received by the pressure sensor and determine the pressure zone of the real-time pressure;

[0038] a first calculation module, configured to select an objective function according to the pressure partition and the real-time pressure, and calculate a target volume gain value according to the objective function;

[0039] a second calculation module, configured to select a target audio type and a target frequency modulation rate according to the pressure partition, and calculate a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function;

[0040] The result output module is used to change the volume according to the current volume gain value, change the frequency of the target audio type according to the target center frequency, and obtain the target sound according to the results after the volume change and the frequency change.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a virtual reality material perception simulation program based on dynamic auditory feedback stored in the memory and runnable on the processor, wherein the virtual reality material perception simulation program based on dynamic auditory feedback, when executed by the processor, implements the steps of the virtual reality material perception simulation method based on dynamic auditory feedback as described above.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a virtual reality material perception simulation program based on dynamic auditory feedback, and when the virtual reality material perception simulation program based on dynamic auditory feedback is executed by a processor, the steps of the virtual reality material perception simulation method based on dynamic auditory feedback as described above are implemented.

[0043] In the present invention, the real-time pressure received by the pressure sensor is obtained, and the pressure zone of the real-time pressure is determined; a target function is selected according to the pressure zone and the real-time pressure, and a target volume gain value is calculated according to the target function; a target audio type and a target frequency modulation rate are selected according to the pressure zone, and a target center frequency is calculated according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function; the volume is changed according to the current volume gain value, and the frequency of the target audio type is changed according to the target center frequency, and the target sound is obtained according to the result after the volume change and the result after the frequency change. The present invention collects the pressure of the user's finger through a pressure sensor, combines the corresponding volume adjustment method and frequency adjustment method, and dynamically adjusts the playback parameters of the pre-recorded material scratching sound, thereby using a material and a corresponding target sound to make the user produce an auditory illusion and a perceptual illusion, thereby achieving the effect of simulating the tactile sensation of multiple materials in VR. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a preferred embodiment of the virtual reality material perception simulation method based on dynamic auditory feedback of the present invention;

[0045] Figure 2 This is a hardware schematic diagram of a preferred embodiment of the virtual reality material perception simulation method based on dynamic auditory feedback of the present invention;

[0046] Figure 3 This is a schematic diagram of the processing process of an embodiment of the virtual reality material perception simulation method based on dynamic auditory feedback of the present invention;

[0047] Figure 4 This is a schematic diagram of perception implementation of an embodiment of a virtual reality material perception simulation method based on dynamic auditory feedback according to the present invention;

[0048] Figure 5 1 is a structural diagram of a preferred embodiment of a virtual reality material perception simulation system based on dynamic auditory feedback according to the present invention;

[0049] Figure 6 FIG. 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] At present, the tactile simulation of materials in virtual reality mainly relies on complex and high-cost physical tactile feedback devices (such as force feedback devices or surface texture simulation devices). For example, the existing technology simulates the touch of materials through vibration motors or variable friction devices, but is limited by hardware cost, volume and the singleness of the simulated materials. Among them, technologies based on visual or tactile pseudo-feedback can also achieve the same effect. However, the current tactile simulation of materials in virtual reality has the following problems: physical tactile devices are expensive and bulky, making them difficult to popularize and apply; the range of material simulation is limited, and it is impossible to flexibly switch the touch of multiple materials through a single material interface; the multimodal feedback is not synergistic enough, and the real-time dynamic matching technology of auditory feedback and tactile feedback is not yet mature; the user's sense of immersion is insufficient, and there is a lack of cross-modal integration solutions for inducing tactile perception through hearing. Therefore, the current tactile simulation of materials in virtual reality is poor and difficult to meet user needs.

[0052] In order to solve one or more of the above-mentioned problems, the present invention obtains the real-time pressure received by the pressure sensor and determines the pressure zone of the real-time pressure; selects the target function according to the pressure zone and the real-time pressure, and calculates the target volume gain value according to the target function; selects the target audio type and the target frequency modulation rate according to the pressure zone, and calculates the target center frequency according to the real-time pressure, the target frequency modulation rate and the target center frequency adjustment function; changes the volume according to the current volume gain value, changes the frequency of the target audio type according to the target center frequency, and obtains the target sound according to the results after the volume change and the results after the frequency change.

[0053] The virtual reality material perception simulation method based on dynamic auditory feedback described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the virtual reality material perception simulation method based on dynamic auditory feedback includes the following steps:

[0054] Step S10: Acquire the real-time pressure received by the pressure sensor, and determine the pressure zone of the real-time pressure.

[0055] Specifically, the real-time pressure acquired by the pressure sensor is the pressure of the finger on the corresponding touch interface. In one embodiment of the present invention, the corresponding pressure sensors are two OLUDA AT8501 load sensors, which detect finger pressure in real time (0.1g resolution). Furthermore, in the present invention, the touch interface can be a 3D-printed PLA board, the size of which can be customized based on actual conditions. In one embodiment of the present invention, the size is 250×212×8mm.

[0056] PLA (polylactic acid) is one of the most common and widely used 3D printing materials. Made from renewable resources such as corn starch or sucrose, it contains no harmful substances and is therefore environmentally friendly. PLA does not emit pungent odors or harmful gases during the printing process. Its excellent layer adhesion and surface quality make it suitable for creating items such as decorations, models, and prototypes. Its low melting temperature and high fluidity allow for printing at low temperatures, eliminating the need for a special heated bed. Objects printed with PLA can be given a smooth, delicate appearance through surface treatment.

[0057] In this invention, the physical tactile sensation provided by the PLA board remains unchanged; the surface remains constant, and the user is always touching the same PLA board. The simulated material is achieved by altering the acoustic feedback. For example, if a wooden board is seen in VR, the program automatically switches to outputting the sound of the board. This dynamic sound, processed by the VR material perception simulation method based on dynamic auditory feedback, influences human perception, giving the user the sensation of touching the board—a pseudo-tactile sensation.

[0058] Furthermore, the acquiring of the real-time pressure received by the pressure sensor and determining the pressure zone of the real-time pressure specifically includes:

[0059] Acquire the real-time pressure received by the pressure sensor and the preset first pressure zone and second pressure zone;

[0060] The pressure zones of the real-time pressure are determined according to the real-time pressure, wherein the pressure zones include a first pressure zone and a second pressure zone.

[0061] Specifically, in the present invention, the received real-time pressure is divided into a first pressure zone and a second pressure zone, i.e., a light pressure zone and a heavy pressure zone. The pressure range of each zone is preset and is set according to specific circumstances. i.e., the range of the first pressure zone is [F l min , F th ), the range of the second pressure zone is [F th , F max The corresponding pressure zone can be determined by the real-time pressure.

[0062] Step S20: selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function.

[0063] Specifically, in the present invention, adaptive amplitude modulation is set, that is, the gain value of the volume is set; that is, according to the pressure input of the finger, the algorithm will be executed to output the volume corresponding to the material (that is, the volume changes in real time according to the change in the finger pressure), but the frequency will not be changed intentionally; the sound gain (volume) is controlled by a quadratic piecewise function to achieve a natural transition between the two sound tracks of light pressure and heavy pressure.

[0064] Furthermore, selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function, specifically includes:

[0065] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function;

[0066] When the pressure zone is the first pressure zone, the second quadratic piecewise function is selected as the objective function, and the target volume gain value is calculated according to the corresponding objective function.

[0067] Specifically, in the present invention, for different pressure zones, this case adopts different quadratic piecewise functions as objective functions to calculate the target volume gain value; wherein, when the user's finger presses lightly, that is, in the first pressure zone, the gain corresponding to the light pressure segment is adopted, and when the user's finger presses heavily, that is, in the second pressure zone, the gain corresponding to the heavy pressure segment is adopted.

[0068] Furthermore, when the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0069] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function;

[0070] Substituting the real-time pressure into the first quadratic piecewise function to obtain a corresponding target volume gain value;

[0071] Among them, the first quadratic piecewise function is:

[0072]

[0073] G l (F) represents the target volume gain value under the first pressure partition, ranging from 0 to 1, F represents the real-time pressure, the unit is gram or Newton, a1, a2, b1, b2, c1 and c2 represent the coefficients of the first quadratic piecewise function, F l min Indicates the starting threshold of the first pressure zone, F trans Indicates the starting point of the crossover transition area between the first pressure zone and the second pressure zone, Fth Indicates the pressure threshold at which the second pressure zone begins to dominate.

[0074] Specifically, to create overlapping control areas and prevent auditory discontinuity, the four pressure limits must satisfy the increasing order, i.e., F l min <F trans <F th <F max .

[0075] Furthermore, when the pressure zone is the first pressure zone, selecting the second quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0076] When the pressure zone is the second pressure zone, selecting a second quadratic piecewise function as the objective function;

[0077] Substituting the real-time pressure into the second quadratic piecewise function to obtain a corresponding target volume gain value;

[0078] Among them, the second quadratic piecewise function is:

[0079]

[0080] G h (F) represents the target volume gain value under the second pressure partition, ranging from 0 to 1, F represents the real-time pressure, a3, a4, b3, b4, c3 and c4 represent the coefficients of the second quadratic piecewise function, Indicates the starting threshold of the second pressure zone, F max Indicates the maximum pressure threshold of the system, F th Indicates the pressure threshold at which the second pressure zone begins to dominate, G sat Indicates the upper limit of saturation gain.

[0081] Among them, G sat Indicates the upper limit of saturation gain, used to protect hearing.

[0082] In addition, for the coefficients of the first quadratic piecewise function and the coefficients of the second quadratic piecewise function in the present invention, the present invention sets them accordingly, as shown in Table 1 below, where the light pressure range is the first pressure partition and the heavy pressure range is the second pressure partition.

[0083] Table 1 Piecewise function empirical coefficients

[0084]

[0085] Step S30: selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function.

[0086] Specifically, in the present invention, the target audio type and target frequency modulation rate are selected according to the pressure zone, wherein the target audio type is divided into the first pressure channel audio and the second pressure channel audio, and each pressure channel audio corresponds to a bandpass filter, and the selection of the target audio type is controlled by controlling the switch of the bandpass filter. In the present invention, when the real-time pressure is lower than the threshold F th , only light pressure audio (i.e., the first pressure channel) plays; if it exceeds the threshold, heavy pressure audio (i.e., the second pressure channel) plays. The corresponding center frequency is adjusted to achieve a change in timbre, simulating a change in "tactile depth."

[0087] Furthermore, selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function specifically includes:

[0088] When the pressure zone is the first pressure zone, select the first pressure channel audio as the target audio type, select the first pressure zone center frequency decrease rate as the target frequency modulation rate, substitute the real-time pressure and the first pressure zone center frequency decrease rate into the target center frequency adjustment function, and calculate the corresponding target center frequency;

[0089] When the pressure zone is the second pressure zone, the second pressure channel audio is selected as the target audio type, and the center frequency decrease rate of the second pressure zone is selected as the target frequency modulation rate. The corresponding target center frequency is calculated by substituting the real-time pressure and the center frequency decrease rate of the second pressure zone into the target center frequency adjustment function.

[0090] Specifically, in the present invention, when selecting the target audio type, the following gain control (on / off mechanism) is adopted:

[0091]

[0092] Among them, G(F) is the target audio type, G base For a fixed output volume value, when the real-time pressure is lower than the threshold F th , only light pressure audio, that is, the first pressure channel audio, is played; if it is higher than the threshold, heavy pressure audio, that is, the second pressure channel audio, is played.

[0093] Then, in the present invention, the target frequency modulation rate is selected according to the corresponding pressure partition. For the first pressure partition, the first pressure partition center frequency decrease rate α (light)Expressed as:

[0094]

[0095] Among them, f max is the highest center frequency at the lowest pressure, f min is the lowest center frequency before reaching the heavy pressure; and where f min Need to be less than f max In this formula, the denominator is the pressure span and the numerator is the frequency span, so α (light) is the frequency decrease rate.

[0096] For the second pressure zone, the center frequency decrease rate α of the second pressure zone (heavy) Expressed as:

[0097]

[0098] Among them, the present invention sets the heavy pressure, that is, the center frequency of the second pressure zone continues to decrease (or restarts from f max starts to decrease), depends on the filter design.

[0099] For different pressure zones, the target audio type is selected accordingly. Then, the real-time pressure and target frequency modulation rate are substituted into the target center frequency adjustment function to calculate the corresponding target center frequency. The target center frequency adjustment function is:

[0100] f c (F) = f max -α(FF regime_min );

[0101] Among them, f c (F) is the center frequency of the bandpass filter (the "position" of the timbre), α is the target FM rate (in Hz / g), and F regime_min is the minimum pressure value in the current mode, that is, the minimum pressure value in the second pressure zone or the minimum pressure value in the first pressure zone.

[0102] In the present invention, as the pressure increases, the center frequency gradually decreases and the sound becomes deeper, simulating the intuitive hearing feeling of "the heavier the pressure, the deeper the friction sound". th The frequency spectrum is changed dynamically to achieve the frequency change of the material texture. Through this process, the present invention linearly controls the change of the filter center frequency according to the finger pressure, thereby changing the timbre of the sound. For example, when pressing lightly, the sound is sharp (more high frequency); the harder you press, the sound becomes deeper (high frequency is weakened and low frequency is enhanced).

[0103] Step S40: changing the volume according to the current volume gain value, changing the frequency of the target audio type according to the target center frequency, and obtaining the target sound according to the results after the volume change and the frequency change.

[0104] Specifically, in the present invention, a dynamic volume gain and frequency are output simultaneously according to the finger pressure input (ie, the volume and frequency are output in real time according to the change in the finger pressure).

[0105] Furthermore, the volume is changed according to the current volume gain value, the frequency of the target audio type is changed according to the target center frequency, and the target sound is obtained according to the result after the volume change and the result after the frequency change, which specifically includes:

[0106] Controlling the volume to change according to the current volume gain value to obtain a result after the volume change;

[0107] Performing frequency change on the target audio type according to the target center frequency to obtain a result after the frequency change;

[0108] Substitute the result after the volume change and the result after the frequency change into the final output formula to obtain the target sound.

[0109] Specifically, the volume is adjusted according to the current volume gain value to obtain the volume change result; the frequency of the target audio type is changed according to the target center frequency to obtain the frequency change result. These results are then substituted into the following final output formula:

[0110] y(F)=G l (F)*H l (f c (F))+G h (F)*H h (f c (F));

[0111] Among them, y(F) is the target sound, H l (f c (F)) is the audio signal after the first pressure channel audio is filtered by the bandpass filter under light pressure, H h (f c (F)) is the audio signal after the second pressure channel audio is filtered by a bandpass filter under heavy pressure.

[0112] The above process yields the target sound perceived by the user and provides feedback on the specific instance. Specifically, in this invention, finger pressure simultaneously controls: volume (gain), which determines the sound's "intensity," and frequency (filter center), which determines the sound's "texture." The final output is a weighted combination of the two audio channels. This achieves three perceptual goals: low-frequency enhancement (the greater the pressure, the deeper the sound); smooth timbre transitions to avoid auditory "jumps"; and a safe volume limit, with the output never exceeding 70dBSPL, to protect the user's hearing.

[0113] This invention can sense a variety of materials (such as clay, ABS plastic, wood boards, etc.) in VR by touching a single PLA board and listening to dynamically changing sounds, with an accuracy rate of up to 82.3%; it also improves the response speed and consistency of tactile feedback; the dynamic sounds processed by the algorithm can significantly enhance the user's sense of immersion.

[0114] Furthermore, in the present invention, a contact microphone can be provided to collect the sound of scratching the PLA board and detect finger movement. The contact microphone can be provided as two AKG C411 PP microphones.

[0115] Furthermore, in one implementation of the present invention, the corresponding hardware is as follows: Figure 2 As shown, two load cells and contact microphones are mounted under the PLA board to detect pressure input and finger movement. They are fixed to the table with two clamps.

[0116] like Figure 3 As shown, it is a schematic diagram of the processing process of an embodiment, showing the overall layout of the PLA board, pressure sensor and audio feedback control. It can be seen that the final sound can be adjusted through the corresponding pressure sensor, thereby realizing the tactile simulation of purple sand.

[0117] like Figure 4 As shown, in one embodiment of the present invention, a pressure sensor and a contact microphone can be set to correspond to Figure 4 , and apply the method of the present invention.

[0118] The present invention obtains the real-time pressure received by the pressure sensor and determines the pressure zone of the real-time pressure; selects a target function according to the pressure zone and the real-time pressure, and calculates a target volume gain value according to the target function; selects a target audio type and a target frequency modulation rate according to the pressure zone, and calculates a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function; changes the volume according to the current volume gain value, changes the frequency of the target audio type according to the target center frequency, and obtains the target sound according to the result after the volume change and the result after the frequency change. The present invention collects the pressure of the user's finger through a pressure sensor, combines the corresponding volume adjustment method and frequency adjustment method, and dynamically adjusts the playback parameters of the pre-recorded material scratching sound, thereby using a material and a corresponding target sound to allow the user to produce an auditory illusion and a perceptual illusion, thereby achieving the effect of simulating the tactile sensation of multiple materials in VR.

[0119] Further, if Figure 5 As shown, based on the above-mentioned virtual reality material perception simulation method based on dynamic auditory feedback, the present invention also provides a virtual reality material perception simulation system based on dynamic auditory feedback, wherein the virtual reality material perception simulation system based on dynamic auditory feedback includes:

[0120] A pressure zone determination module 51 is configured to obtain the real-time pressure received by the pressure sensor and determine the pressure zone of the real-time pressure;

[0121] a first calculation module 52, configured to select a target function according to the pressure partition and the real-time pressure, and calculate a target volume gain value according to the target function;

[0122] A second calculation module 53 is configured to select a target audio type and a target frequency modulation rate according to the pressure zone, and calculate a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function;

[0123] The result output module 54 is used to change the volume according to the current volume gain value, change the frequency of the target audio type according to the target center frequency, and obtain the target sound according to the results after the volume change and the results after the frequency change.

[0124] Further, if Figure 6 As shown, based on the above-mentioned virtual reality material perception simulation method and system based on dynamic auditory feedback, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 6 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0125] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the terminal's hard drive or memory. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 20 may include both the terminal's internal storage unit and an external storage device. The memory 20 is used to store application software installed on the terminal and various types of data, such as program code installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or is about to be output. In one embodiment, the memory 20 stores a virtual reality material perception simulation program 40 based on dynamic auditory feedback. This virtual reality material perception simulation program 40 based on dynamic auditory feedback can be executed by the processor 10, thereby implementing the virtual reality material perception simulation method based on dynamic auditory feedback of the present invention.

[0126] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program code or process data stored in the memory 20, such as executing the virtual reality material perception simulation method based on dynamic auditory feedback.

[0127] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0128] In one embodiment, when the processor 10 executes the virtual reality material perception simulation program 40 based on dynamic auditory feedback in the memory 20 , the steps of the above method for virtual reality material perception simulation based on dynamic auditory feedback are implemented.

[0129] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a virtual reality material perception simulation program based on dynamic auditory feedback, and when the virtual reality material perception simulation program based on dynamic auditory feedback is executed by a processor, the following steps are implemented:

[0130] Acquire the real-time pressure received by the pressure sensor, and determine the pressure zone of the real-time pressure;

[0131] selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function;

[0132] selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function;

[0133] The volume is changed according to the current volume gain value, the frequency of the target audio type is changed according to the target center frequency, and the target sound is obtained according to the results after the volume change and the results after the frequency change.

[0134] The acquiring of the real-time pressure received by the pressure sensor and determining the pressure zone of the real-time pressure specifically includes:

[0135] Acquire the real-time pressure received by the pressure sensor and the preset first pressure zone and second pressure zone;

[0136] The pressure zones of the real-time pressure are determined according to the real-time pressure, wherein the pressure zones include a first pressure zone and a second pressure zone.

[0137] The step of selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function, specifically includes:

[0138] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function;

[0139] When the pressure zone is the first pressure zone, the second quadratic piecewise function is selected as the objective function, and the target volume gain value is calculated according to the corresponding objective function.

[0140] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0141] When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function;

[0142] Substituting the real-time pressure into the first quadratic piecewise function to obtain a corresponding target volume gain value;

[0143] Among them, the first quadratic piecewise function is:

[0144]

[0145] G l(F) represents the target volume gain value under the first pressure partition, F represents the real-time pressure, a1, a2, b1, b2, c1 and c2 represent the coefficients of the first quadratic piecewise function, F l min Indicates the starting threshold of the first pressure zone, F trans Indicates the starting point of the crossover transition area between the first pressure zone and the second pressure zone, F th Indicates the pressure threshold at which the second pressure zone begins to dominate.

[0146] Wherein, when the pressure zone is the first pressure zone, selecting the second quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function specifically includes:

[0147] When the pressure zone is the second pressure zone, selecting a second quadratic piecewise function as the objective function;

[0148] Substituting the real-time pressure into the second quadratic piecewise function to obtain a corresponding target volume gain value;

[0149] Among them, the second quadratic piecewise function is:

[0150]

[0151] G h (F) represents the target volume gain value under the second pressure partition, F represents the real-time pressure, a3, a4, b3, b4, c3 and c4 represent the coefficients of the second quadratic piecewise function, Indicates the starting threshold of the second pressure zone, F max Indicates the maximum pressure threshold of the system, F th Indicates the pressure threshold at which the second pressure zone begins to dominate, G sat Indicates the upper limit of saturation gain.

[0152] The step of selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function specifically includes:

[0153] When the pressure zone is the first pressure zone, select the first pressure channel audio as the target audio type, select the first pressure zone center frequency decrease rate as the target frequency modulation rate, substitute the real-time pressure and the first pressure zone center frequency decrease rate into the target center frequency adjustment function, and calculate the corresponding target center frequency;

[0154] When the pressure zone is the second pressure zone, the second pressure channel audio is selected as the target audio type, and the center frequency decrease rate of the second pressure zone is selected as the target frequency modulation rate. The corresponding target center frequency is calculated by substituting the real-time pressure and the center frequency decrease rate of the second pressure zone into the target center frequency adjustment function.

[0155] The step of changing the volume according to the current volume gain value, changing the frequency of the target audio type according to the target center frequency, and obtaining the target sound according to the result after the volume change and the result after the frequency change specifically includes:

[0156] Controlling the volume to change according to the current volume gain value to obtain a result after the volume change;

[0157] Performing frequency change on the target audio type according to the target center frequency to obtain a result after the frequency change;

[0158] Substitute the result after the volume change and the result after the frequency change into the final output formula to obtain the target sound.

[0159] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0160] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0161] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for simulating virtual reality material perception based on dynamic auditory feedback, characterized in that: The virtual reality material perception simulation method based on dynamic auditory feedback includes: Acquire the real-time pressure received by the pressure sensor, and determine the pressure zone of the real-time pressure; selecting an objective function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the objective function; selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function; The volume is changed according to the current volume gain value, the frequency of the target audio type is changed according to the target center frequency, and the target sound is obtained according to the results after the volume change and the results after the frequency change.

2. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 1, characterized in that: The acquiring the real-time pressure received by the pressure sensor and determining the pressure zone of the real-time pressure specifically includes: Acquire the real-time pressure received by the pressure sensor and the preset first pressure zone and second pressure zone; The pressure zones of the real-time pressure are determined according to the real-time pressure, wherein the pressure zones include a first pressure zone and a second pressure zone.

3. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 2, characterized in that: The selecting a target function according to the pressure partition and the real-time pressure, and calculating a target volume gain value according to the target function, specifically includes: When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function; When the pressure zone is the first pressure zone, the second quadratic piecewise function is selected as the objective function, and the target volume gain value is calculated according to the corresponding objective function.

4. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 3, characterized in that: When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function, specifically includes: When the pressure zone is the first pressure zone, selecting the first quadratic piecewise function as the objective function; Substituting the real-time pressure into the first quadratic piecewise function to obtain a corresponding target volume gain value; Among them, the first quadratic piecewise function is: G l (F) represents the target volume gain value under the first pressure partition, F represents the real-time pressure, a1, a2, b1, b2, c1 and c2 represent the coefficients of the first quadratic piecewise function, F l min Indicates the starting threshold of the first pressure zone, F trans Indicates the starting point of the crossover transition area between the first pressure zone and the second pressure zone, F th Indicates the pressure threshold at which the second pressure zone begins to dominate.

5. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 3, characterized in that: When the pressure zone is the first pressure zone, selecting the second quadratic piecewise function as the objective function, and calculating the target volume gain value according to the corresponding objective function, specifically includes: When the pressure zone is the second pressure zone, selecting a second quadratic piecewise function as the objective function; Substituting the real-time pressure into the second quadratic piecewise function to obtain a corresponding target volume gain value; Among them, the second quadratic piecewise function is: G h (F) represents the target volume gain value under the second pressure partition, F represents the real-time pressure, a3, a4, b3, b4, c3 and c4 represent the coefficients of the second quadratic piecewise function, Indicates the starting threshold of the second pressure zone, F max Indicates the maximum pressure threshold of the system, F th Indicates the pressure threshold at which the second pressure zone begins to dominate, G sat Indicates the upper limit of saturation gain.

6. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 2, characterized in that: The step of selecting a target audio type and a target frequency modulation rate according to the pressure partition, and calculating a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function specifically includes: When the pressure zone is the first pressure zone, select the first pressure channel audio as the target audio type, select the first pressure zone center frequency decrease rate as the target frequency modulation rate, substitute the real-time pressure and the first pressure zone center frequency decrease rate into the target center frequency adjustment function, and calculate the corresponding target center frequency; When the pressure zone is the second pressure zone, the second pressure channel audio is selected as the target audio type, and the center frequency decrease rate of the second pressure zone is selected as the target frequency modulation rate. The corresponding target center frequency is calculated by substituting the real-time pressure and the center frequency decrease rate of the second pressure zone into the target center frequency adjustment function.

7. The method for simulating virtual reality material perception based on dynamic auditory feedback according to claim 1, characterized in that: The step of changing the volume according to the current volume gain value, changing the frequency of the target audio type according to the target center frequency, and obtaining the target sound according to the result after the volume change and the result after the frequency change specifically includes: Controlling the volume to change according to the current volume gain value to obtain a result after the volume change; Performing frequency change on the target audio type according to the target center frequency to obtain a result after the frequency change; Substitute the result after the volume change and the result after the frequency change into the final output formula to obtain the target sound.

8. A virtual reality material perception simulation system based on dynamic auditory feedback, characterized in that: The virtual reality material perception simulation system based on dynamic auditory feedback includes: a pressure zone determination module, configured to obtain the real-time pressure received by the pressure sensor and determine the pressure zone of the real-time pressure; a first calculation module, configured to select an objective function according to the pressure partition and the real-time pressure, and calculate a target volume gain value according to the objective function; a second calculation module, configured to select a target audio type and a target frequency modulation rate according to the pressure partition, and calculate a target center frequency according to the real-time pressure, the target frequency modulation rate, and the target center frequency adjustment function; The result output module is used to change the volume according to the current volume gain value, change the frequency of the target audio type according to the target center frequency, and obtain the target sound according to the results after the volume change and the frequency change.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a virtual reality material perception simulation program based on dynamic auditory feedback stored in the memory and executable on the processor. When the virtual reality material perception simulation program based on dynamic auditory feedback is executed by the processor, the steps of the virtual reality material perception simulation method based on dynamic auditory feedback are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a virtual reality material perception simulation program based on dynamic auditory feedback. When the virtual reality material perception simulation program based on dynamic auditory feedback is executed by a processor, the steps of the virtual reality material perception simulation method based on dynamic auditory feedback are implemented as described in any one of claims 1 to 7.

Images