Immersive art design interaction method and system in virtual reality scene

Abstract

The application discloses an immersive artistic design interaction method and system in a virtual reality scene, and belongs to the technical field of electric digital data processing. The method first acquires a space coordinate tracking sequence of an interaction terminal and a software material benchmark physical parameter; determines an interaction contact area through space mapping and collision detection, and extracts a center pressing depth and a surface deformation curvature; then, respectively solves a basic elastic component, a dynamic damping component and a vibration gain component, and performs vector superposition coupling to generate continuously updated dynamic tactile driving parameters; finally, encapsulates and synchronously issues the dynamic tactile driving parameters and a visual rendering frame hardware timestamp to a tactile device to trigger synchronous output of corresponding tactile driving signals. The application constructs a continuous mapping mechanism from a software physical deformation feature to dynamic tactile, realizes timing synchronization and dynamic linkage of visual rendering and tactile feedback, and improves material reality, interaction immersion and three-dimensional modeling experience in a virtual reality artistic creation process.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to an immersive art design interaction method and system in virtual reality scenarios. Background Technology

[0002] With the rapid development of virtual reality technology, immersive art is undergoing a profound transformation from two-dimensional screen presentation to three-dimensional spatial experience. In this transformation, interaction is not only a bridge connecting users and the virtual world, but also a core element in building immersion and aesthetic experience.

[0003] In the current technology, the process of virtual reality immersive art design interaction begins with the digitization and semantic annotation of art assets. This involves constructing geometric and material attributes through 3D modeling and assigning interactive semantic tags to scene elements to clarify their interactive identity. Next, spatial mapping and interactive topology construction are implemented. The virtual interaction boundaries are defined using spatial coordinate systems and collision volumes. Logical relationships between elements are established through visual programming or scripts, establishing interactive triggering rules for gaze projection and spatial tracking. Finally, event-driven logic and state machine binding are implemented. An event-driven mechanism accurately maps user input actions (such as gesture tracking and controller operations) to state changes in art forms, such as triggering dynamic flows of material lighting and shadows. The process involves generating and evolving particle effects; then, multi-sensory real-time rendering and feedback output are performed. The real-time rendering pipeline, combined with post-processing technology, calculates and outputs high-frame-rate visual images. Spatial audio technology dynamically calculates the three-dimensional sound field based on the user's head orientation and the location of the sound source. At the same time, interactive events are converted into vibration signals and fed back to the tactile device, achieving a synchronous perception loop of visual, auditory, and tactile multimodalities. Finally, in the optimization and runtime iteration phase, performance monitoring tools are used to analyze system load. Techniques such as detail level switching and instantiated rendering are used to ensure smooth interaction. Combined with recorded user interaction behavior logs, the interaction threshold is dynamically adjusted and artistic expression deviations are corrected, thereby completing the continuous iteration and closed loop of the entire interaction process.

[0004] However, in the process of implementing the inventive technical solution in the embodiments of this application, it was found that the above-mentioned technology has at least the following technical problems:

[0005] Flat design is limited by the inherent lack of tactile dimension in two-dimensional media, while the integrity of the virtual reality three-dimensional space creation experience highly depends on the cross-modal perception loop of "vision-touch". However, existing virtual reality interaction technology lacks real-time dynamic tactile calculation and mapping when dealing with non-rigid soft art materials. Its tactile feedback mostly relies on preset discrete static vibration waveforms, which cannot establish a continuous mapping relationship with the dynamic physical parameters such as three-dimensional pressing depth and deformation curvature generated by soft interaction. This means that although it can render the visual data of soft deformation in real time, it cannot synchronously reverse calculate and output corresponding dynamic damping and other tactile feedback signals. This serious disconnect between continuous visual deformation and discrete tactile vibration causes cross-modal sensory conflict, causing users to fall into the perceptual illusion of "seeing three-dimensional and touching two-dimensional". As a result, immersive art design interaction in virtual reality scenarios cannot truly realize the depth perception and physical shaping of three-dimensional materials, seriously breaking the immersive experience loop of art interaction. Summary of the Invention

[0006] To address the technical problem in existing technologies where the lack of real-time dynamic tactile processing and continuous mapping mechanisms for non-rigid soft materials leads to a cross-modal sensory disconnect between continuous visual deformation and discrete tactile vibration, thus disrupting the closed loop of immersive art interaction experiences in virtual reality, this invention provides an immersive art design interaction method and system for virtual reality scenarios. The technical solution is as follows:

[0007] On the one hand, an immersive art design interaction method in a virtual reality scene is provided. This method includes: acquiring the spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene, as well as the three-dimensional mesh model and reference physical parameters of the non-rigid soft art material, including stiffness coefficient, viscosity coefficient, and vibration conversion coefficient; performing spatial mapping and collision detection on the spatial coordinate tracking sequence and the three-dimensional mesh model to determine the interactive contact area containing multiple mesh vertices; extracting the set of three-dimensional displacement vectors of the mesh vertices within the interactive contact area relative to the initial topological state, and calculating the maximum magnitude of the three-dimensional displacement vector set to determine the center pressing depth; simultaneously, calculating the ratio of the angle between the normal vectors of adjacent mesh vertices within the interactive contact area to the spatial distance between vertices to obtain the local discrete curvature, and statistically analyzing the average value of the local discrete curvature within the interactive contact area as the surface deformation curvature; and then calculating the center pressing depth... The basic elastic force component is calculated using the degree and stiffness coefficients as input variables through a preset basic elastic force mapping function. The pressing speed is obtained by calculating the partial derivative of the center pressing depth over time, and the dynamic damping component is calculated by combining the pressing speed with the viscosity coefficient. The transient deformation rate is obtained by calculating the partial derivative of the surface deformation curvature over time, and the vibration gain component is calculated by using the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter. The basic elastic force component, dynamic damping component, and vibration gain component are vector-coupled to generate continuously updated dynamic haptic drive parameters. In response to the graphics rendering pipeline completing the software visual rendering update of the current frame, the hardware timestamp of the rendering frame is extracted. The dynamic haptic drive parameters and the hardware timestamp are encapsulated into a haptic control data packet and sent to the haptic feedback device to trigger the haptic feedback device to synchronously output the corresponding haptic drive signal according to the hardware timestamp.

[0008] On the other hand, an immersive art design interaction system in a virtual reality scene is provided, including: an interactive data acquisition module, a deformation feature extraction module, a tactile drive mapping module, and a visual-tactile synchronous output module; the interactive data acquisition module is used to acquire the spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene, as well as the three-dimensional mesh model and reference physical parameters of the non-rigid soft art material, including stiffness coefficient, viscosity coefficient, and vibration conversion coefficient; the deformation feature extraction module is used to perform spatial mapping and collision detection between the spatial coordinate tracking sequence and the three-dimensional mesh model to determine the interactive contact area containing multiple mesh vertices; extract the set of three-dimensional displacement vectors of the mesh vertices in the interactive contact area relative to the initial topological state, and calculate the maximum value of the modulus of the three-dimensional displacement vector set to determine the center pressing depth; at the same time, calculate the ratio of the angle between the normal vectors of adjacent mesh vertices in the interactive contact area to the spatial distance between vertices to obtain the local discrete curvature, and statistically analyze the average local discrete curvature in the interactive contact area. The mean value is used as the surface deformation curvature; the haptic drive mapping module is used to take the center pressing depth and stiffness coefficient as input variables, and calculate the basic elastic force component through a preset basic elastic force mapping function; calculate the partial derivative of the center pressing depth with time to obtain the pressing speed, and combine the pressing speed with the viscosity coefficient to calculate the dynamic damping component; calculate the partial derivative of the surface deformation curvature with time to obtain the transient deformation rate, and use the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter to calculate the vibration gain component; the basic elastic force component, dynamic damping component and vibration gain component are vector-superimposed and coupled to generate continuously updated dynamic haptic drive parameters; the visual-touch synchronization output module is used to respond to the graphics rendering pipeline to complete the software visual rendering update of the current frame, extract the hardware timestamp of the rendering frame; encapsulate the dynamic haptic drive parameters and the hardware timestamp into a haptic control data packet and send it to the haptic feedback device to trigger the haptic feedback device to synchronously output the corresponding haptic drive signal according to the hardware timestamp.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0010] This invention addresses the lack of real-time dynamic tactile computation and continuous mapping mechanisms for non-rigid soft materials in existing technologies. Compared to the limitations of existing technologies that rely on preset discrete waveforms for triggered feedback, it achieves the following significant advantages: Specifically, firstly, at the level of interactive feature extraction, this invention breaks through the single-dimensional triggering limitations of existing technologies, constructing a tactile computation foundation based on physical driving and precise extraction of multi-dimensional deformation. Existing technologies often ignore the physical constitutive properties of materials, relying solely on a single collision event or pressure depth to trigger preset vibrations. This invention, however, transforms virtual interaction from "event-triggered" to "physically driven" by introducing benchmark physical parameters such as stiffness coefficient and viscosity coefficient. Simultaneously, in collision detection, it not only extracts the center pressure depth reflecting macroscopic pressure intensity but also innovatively extracts the surface deformation curvature of the interactive contact area. This feature extraction method accurately captures the topological changes of the surface micro-bumps and wrinkles of the soft material after it is subjected to force, which expands the quantification dimension of interactive features from one dimension to multiple dimensions. This provides high-fidelity input variables for the subsequent establishment of continuous and delicate tactile mapping, breaking the perceptual illusion of "touching a flat surface" from the source of data.

[0011] Secondly, at the tactile mapping mechanism level, this invention constructs a continuous mapping mechanism coupling static damping and dynamic vibration gain, effectively replacing the discrete static vibration waveform of existing technologies. Addressing the deficiency of existing technologies in establishing continuous mappings with dynamic physical parameters such as three-dimensional pressing depth and deformation curvature, this invention calculates the basic elastic force component based on pressing depth and stiffness coefficient, and the dynamic damping component based on pressing speed and viscosity coefficient, accurately restoring the viscoelastic constitutive structure of the soft body. Furthermore, it innovatively utilizes the transient deformation rate of the deformation curvature's time-dependent derivative, modulating the vibration gain component as a high-frequency dynamic compensation using the vibration conversion coefficient as a weight. This superimposed coupling of "basic elasticity + dynamic damping + vibration gain" transforms the tactile signal from a rigid discrete vibration into a composite physical force that continuously fluctuates in real time with deformation speed and depth. It successfully reverse-engineers the dynamic physical characteristics generated by soft body interaction into continuous tactile driving parameters, highly restoring the unique elastic and viscous physical tactile sensation of non-rigid soft bodies.

[0012] Finally, at the cross-modal synchronization level, this invention proposes a visual-tactile frame-level synchronization alignment mechanism, effectively eliminating cross-modal sensory conflicts and closing the loop of the immersive experience. In existing technologies, the severe disconnect between continuous visual deformation and discrete tactile vibration easily leads to sensory misalignment for users. This invention extracts hardware timestamps in response to the graphics rendering pipeline and synchronously distributes continuously updated dynamic tactile driving parameters and visual rendering frame encapsulation, achieving precise frame-level alignment of tactile output and visual rendering on the timeline. This mechanism effectively eliminates the time difference between visual and tactile signals, allowing users to simultaneously feel matching dynamic damping feedback in their hands at the same instant they visually perceive the soft deformation. This successfully constructs a cross-modal perception loop of "visual-tactile," allowing users to truly escape the disconnect between "seeing three-dimensional objects and touching flat surfaces," achieving deep perception and physical shaping of three-dimensional materials in virtual reality scenes, and completely closing the loop of an immersive art interaction experience. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating the immersive art design interaction method in a virtual reality scene provided in this application embodiment;

[0015] Figure 2 This is a schematic diagram of the structure of an immersive art design interaction system in a virtual reality scene provided in an embodiment of this application. Detailed Implementation

[0016] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0017] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] With the continuous evolution of virtual reality technology, immersive art design is gradually leaping from traditional two-dimensional planar media to three-dimensional space. In the traditional two-dimensional planar design system, creators are naturally limited by the physical properties of the medium and cannot obtain tactile sensory feedback; however, in virtual reality three-dimensional space creation, the integrity of the interactive experience highly depends on the cross-modal sensory loop of "vision-touch". Especially for the shaping of non-rigid soft art materials (such as virtual clay, slime, etc.), the realism and real-time nature of tactile feedback directly determine the creator's depth of perception of the material's physical properties and the precision of shaping, which is a core element to ensure the immersiveness of art creation.

[0020] However, existing virtual reality interaction technologies suffer from a severe lack of real-time dynamic haptic calculation and continuous mapping mechanisms when dealing with non-rigid soft art materials. Current mainstream haptic feedback solutions largely rely on preset discrete static vibration waveforms, meaning they only invoke fixed vibration patterns when interactive events occur (such as collision triggers or penetration thresholds). This event-triggered discrete feedback mechanism cannot establish a mapping relationship with continuously changing dynamic physical parameters such as 3D pressing depth and surface deformation curvature generated in real time during soft body interaction. Although existing graphics rendering pipelines can render visual data of soft body deformation in real time and continuously based on physics engines, haptic feedback systems cannot synchronously reverse-calculate and output corresponding dynamic damping and other haptic feedback signals.

[0021] This severe disconnect between visually continuous deformation and tactile discrete vibration triggers a strong cross-modal sensory conflict, causing users to fall into the perceptual illusion of "seeing three-dimensionality while touching a flat surface." While creators can visually observe the continuous elastic deformation of soft materials, they can only feel stiff, fragmented planar vibrations, unable to perceive the damping changes and viscous rebound during the material deformation process. This not only prevents immersive art design interactions in virtual reality scenarios from truly achieving depth perception and physical shaping of three-dimensional materials, but also seriously disrupts the closed loop of immersive art interaction experiences.

[0022] Based on this, there is an urgent need to propose a virtual reality immersive art design interaction scheme that can establish a dynamic tactile real-time calculation and continuous mapping mechanism for non-rigid soft art materials, so as to realize the real-time coupling between the three-dimensional deformation characteristics of soft materials and tactile feedback parameters, and further realize the synchronous linkage between visual rendering and tactile output, thereby improving the material realism, spatial immersion and three-dimensional shaping experience in the virtual reality art interaction process.

[0023] like Figure 1The diagram shown is a flowchart of an immersive art design interaction method in a virtual reality scene provided in an embodiment of this application, based on... Figure 1 It should be understood that the embodiments of the present invention achieve dynamic coupling and real-time linkage between the deformation process of the soft material and the tactile feedback output by constructing the motion trajectory of the interactive terminal, the three-dimensional mesh model of the non-rigid soft art material, and the correlation mechanism between physical parameters. By continuously calculating the center pressing depth, surface deformation curvature, and transient deformation rate in the interactive contact area, and combining the stiffness coefficient and viscosity coefficient to generate dynamic tactile driving parameters, the tactile feedback can adaptively change with the real-time deformation state of the soft material, thereby overcoming the limitations of the prior art where tactile feedback relies on a fixed discrete vibration template and cannot truly reflect the dynamic physical characteristics of the soft material. At the same time, the present invention further establishes a synchronization mechanism between visual rendering and tactile feedback through hardware timestamps, so that the tactile driving signal can maintain temporal consistency with the visual deformation process of the soft material, effectively reducing the feedback delay between vision and touch and cross-modal sensory conflict, and avoiding the fragmented experience of "visual stereoscopic but tactile planar" for users during virtual reality interaction. Therefore, this invention not only improves the realistic tactile expression of virtual soft art materials, but also enhances the immersive, spatial and three-dimensional shaping experience in the virtual reality art design process, thereby perfecting the immersive art interaction loop of "visual-tactile" integration in virtual reality scenarios.

[0024] The first step of the immersive art design interaction method in virtual reality scenarios establishes a correlation between the motion trajectory of the interactive terminal and the physical properties of the soft art material, thereby achieving a unified modeling of user interaction behavior and the physical characteristics of virtual materials. Compared with the existing technology that only relies on simple touch events for feedback triggering, this invention can continuously perceive the interactive state of the soft material at the spatial motion level, providing a reliable data foundation for subsequent dynamic deformation analysis and real-time tactile calculation, thus improving the realism and physical consistency of virtual art material interaction.

[0025] The first step involves acquiring the spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene, as well as the 3D mesh model and baseline physical parameters of the non-rigid soft art material. Specific implementation methods include:

[0026] S101-1, Obtain the spatial coordinate tracking sequence of the interactive terminal: Using the spatial positioning module of the virtual reality system (e.g., an inside-out tracking system based on an external laser positioning base station or a built-in inertial measurement unit and visual camera), the three-dimensional position data of the interactive terminal (e.g., a VR controller or force feedback data glove) in the virtual reality world coordinate system is collected in real time at a preset physical sampling frequency (e.g., 90Hz or 120Hz). The three-dimensional position data obtained by continuous sampling along the time axis are arranged in timestamp order to generate a spatial coordinate tracking sequence. Preferably, to eliminate hardware noise from the positioning sensor and minor hand tremors, the spatial coordinate tracking sequence is input into a low-pass filter (such as a Kalman filter) for smoothing and noise reduction to obtain stable and continuous terminal motion trajectory data.

[0027] S101-2, Obtain the 3D mesh model and initial topology of the non-rigid soft art material: Load the 3D mesh model corresponding to the currently interactive non-rigid soft art material from the virtual scene's storage medium. This 3D mesh model consists of a vertex set. The model is composed of an edge set and a face set connecting vertices, where each vertex contains its initial 3D spatial coordinates and vertex normal vector. Specifically, at the initial moment when the model is loaded and has not been subjected to any external forces, the system traverses and extracts the initial position coordinates and initial normal vectors of all mesh vertices in the model, and persists them in memory as the initial topological state, so as to serve as the zero-position reference for subsequent calculation of the angle offset between the relative displacement vector and the normal vector of the mesh vertices in the interaction contact area.

[0028] S101-3, Obtain the baseline physical parameters of non-rigid soft art materials: Based on the type identifier of the current virtual soft art material, query and read the baseline physical parameters bound to the 3D mesh model from the material attribute configuration table of the physics engine. The baseline physical parameters include at least the stiffness coefficient. and viscosity coefficient Among them: stiffness coefficient : Characterizes the mechanical properties of soft materials against elastic deformation, used as a scaling factor in subsequent steps to map pressure depth to damping force, determining the magnitude of basic tactile resistance; viscosity coefficient This parameter characterizes the mechanical properties of soft materials as they resist deformation rate due to internal friction during dynamic deformation. It serves as a weighting modulation parameter for dynamic vibration gain in subsequent steps, determining the intensity of viscous sensation and elastic vibration in haptic feedback. These parameters are known constants that reside in system memory throughout the lifecycle of the 3D mesh model, available for real-time later use.

[0029] Based on the above, a spatial coordinate tracking sequence is obtained through high-frequency sampling and filtering, and the initial topological state of the mesh vertices is strictly recorded. This provides a reference benchmark with temporal continuity and spatial accuracy for the accurate calculation of the "three-dimensional displacement vector set" and "transient deformation rate" in subsequent steps, avoiding tactile calculation distortion caused by tracking data jitter or loss of initial state.

[0030] The second step of the immersive art design interaction method in virtual reality scenarios involves accurately identifying the interactive contact area and extracting dynamic deformation characteristics of the soft material, such as the pressing depth and surface curvature, to achieve a refined analysis of the local stress state and continuous deformation process of the soft material. Compared to existing technologies that use fixed trigger rules or single pressure values ​​for feedback control, this step can more accurately reflect the dynamic changes of the soft material during pressing, stretching, bending, and rebounding, thereby enhancing the user's realistic perception of the thickness, elasticity, and three-dimensional shape changes of the virtual material.

[0031] In the second step, the spatial coordinate tracking sequence is spatially mapped and collision detected with the 3D mesh model to determine the interactive contact area containing multiple mesh vertices, and the subsequent feature extraction and parameter correction process is carried out. Specific implementation methods include:

[0032] S102-1, Collision Detection and Contact Region Determination Based on Hierarchical Bounding Box Tree: During the initialization phase of the 3D mesh model, a hierarchical bounding box tree (such as an AABB tree) is constructed based on its triangular facet topology, dividing the space of the mesh model into nodes with hierarchical relationships. In the real-time interaction loop, the current terminal spatial coordinates in the spatial coordinate tracking sequence are extracted as interaction probe points. The probe points are intersected with the nodes of the hierarchical bounding box tree from top to bottom, and nodes that do not intersect are eliminated to filter out a set of potential candidate mesh faces. Subsequently, for each triangular facet in the candidate mesh facet set, the shortest distance from the interaction probe point to the triangular facet is calculated (if the projection of the probe point in the normal direction of the facet falls inside the facet, it is the vertical projection distance; if it falls outside the facet, it is the geometric shortest distance to the edge of the facet). Triangular facests with a shortest distance less than a preset contact threshold (such as 0.5mm) are extracted, and the vertices of these triangular facests are grouped together to form an interaction contact region.

[0033] S102-2, Extraction of Center Press Depth and Hard-Bottom Truncation Penalty Mechanism: Extract the difference between the current frame's 3D spatial coordinates of each mesh vertex within the interactive contact area and its 3D spatial coordinates in the initial topology state cached in the first step, generating a set of 3D displacement vectors; traverse this set to calculate the magnitude of each displacement vector, and extract the maximum magnitude as the center press depth. Subsequently, the local maximum deformation tolerance of the current interactive contact area is read from the material property configuration table. (Characterizing the physical depth threshold that allows the soft material to undergo elastic deformation); Determining the center pressing depth Is it greater than :like Then As input values ​​for subsequent calculations; if This triggers a truncation and penalty mechanism, forcibly truncating the center pressure depth value used in the calculation. At the same time, the stiffness coefficient of the current material is amplified according to a preset penalty multiplier (e.g., 3 times). The amplified stiffness coefficient is then used to update the subsequent reference damping components.

[0034] S102-3, Surface Deformation Curvature Calculation Based on Neighborhood Normal Vector Angle and Distance: For any target mesh vertex within the interactive contact area, extract the ring of neighboring mesh vertices directly connected to it through mesh edges based on mesh topological connectivity (such as half-edge data structure); calculate the spatial angle between the current normal vector of the target mesh vertex and the current normal vector of each ring of neighboring vertices, and calculate the spatial Euclidean distance between the target mesh vertex and each ring of neighboring vertices; divide the calculated spatial angle by the corresponding spatial Euclidean distance to obtain the local discrete curvatures belonging to the target mesh vertex; traverse all mesh vertices within the interactive contact area, summarize all local discrete curvatures and calculate their average value, and output the average value as the overall surface deformation curvature of this frame.

[0035] S102-4, Generation and Synthesis of Tangential Damping Based on Normal Pressure and Anisotropic Friction Tensor: While extracting the three-dimensional displacement vector set, the spatial coordinate difference between two adjacent frames of the interactive terminal is extracted and divided by the frame interval time to obtain the three-dimensional spatial velocity vector; a local tangential plane coordinate system is constructed using the average normal vector of the interactive contact area, and the three-dimensional spatial velocity vector is projected onto this tangential plane to obtain the two-dimensional tangential sliding velocity vector. Read the anisotropic friction tensor of the two-dimensional tangential plane from the reference physical parameters. ; Obtain the projection magnitude of the basic elastic force component output by the current haptic drive mapping module onto the normal direction of the local tangential plane as the normal force N; Combine the normal force N and the tangential sliding velocity vector The reverse unit vector and the anisotropic friction tensor The basic elastic force component output by the current tactile drive mapping module is obtained and multiplied in the normal direction of the local tangential plane to generate a two-dimensional tangential friction damping vector that conforms to Coulomb's law of friction. This vector is then back-projected onto the three-dimensional world coordinate system to generate a three-dimensional tangential friction damping component. Finally, it is superimposed and synthesized with the normal tactile parameter vector.

[0036] It should be understood that in existing technologies, excessive force on software can easily lead to visual penetration and a lack of tactile feedback. This invention, by setting a local maximum deformation tolerance, visually prevents excessive deformation and non-physical penetration of the model; tactilely, by truncating depth and introducing a stiffness penalty multiplier to amplify the reference damping, the user instantly feels extremely strong "bottoming resistance" when excessively pressed, replicating the mechanical response of pressing a real soft object to the bottom and improving the realism of physical interaction.

[0037] Furthermore, the sliding friction of real artistic materials (such as wood grain, leather, and textured clay) varies in different directions. This invention introduces a two-dimensional anisotropic friction tensor for the tangential plane, dynamically extracting the directional friction coefficient based on the direction of the tangential sliding velocity to generate tangential friction damping, which is then superimposed on the normal tactile parameter vector. This mechanism allows users to feel not only the resistance of the pressing depth when pressing and sliding, but also the texture friction that changes with the sliding direction, greatly enriching the dimensions of tactile feedback and achieving a deep shaping of the physical texture of three-dimensional material surfaces.

[0038] The third step of the immersive art design interaction method in virtual reality scenarios involves constructing a dynamic haptic real-time solution mechanism. This mechanism couples and analyzes static deformation with dynamic deformation processes, transforming haptic feedback from traditional discrete preset vibration modes to a continuous dynamic response mode. Compared to existing haptic feedback methods with fixed frequencies and amplitudes, this step dynamically adjusts damping and vibration output based on the real-time deformation state of the soft material. This allows the haptic feedback to realistically correspond to the physical changes of the virtual material, effectively improving the sensitivity, detail, and physical realism of the haptic feedback, and avoiding the disconnect between continuous visual deformation and single haptic vibration.

[0039] In step three, the center pressing depth and stiffness coefficient are used as input variables, and the reference damping component is calculated through a preset reference damping mapping function; the transient deformation rate is obtained by calculating the partial derivative of the surface deformation curvature over time, and the vibration gain component is calculated using the transient deformation rate as the independent variable and the viscosity coefficient as the weighted modulation parameter; the vibration gain component is superimposed and coupled to the reference damping component as a dynamic compensation term to generate continuously updated dynamic tactile driving parameters. Specific implementation methods include:

[0040] S103-1, Calculation of fundamental elastic components based on nonlinear hardening mapping: The center compression depth extracted in the preceding steps is... and stiffness coefficient As input variables, the following calculations are performed using a preset baseline damping mapping function: First, the center is used as an input variable, and the calculations are performed using a preset basic elasticity mapping function: First, the center is pressed to a certain depth. and stiffness coefficient As input variables, the following operations are performed using a preset benchmark damping mapping function: a power function operation is performed on the base variable to obtain the depth deformation variable; subsequently, the depth deformation variable is compared with the stiffness coefficient. As an input variable, the following operation is performed through the preset reference damping mapping function: First, the center is multiplied to obtain the basic resistance value as the basic elastic force component (dimension N, Newton).

[0041] S103-2, Calculation of Dynamic Damping and Vibration Gain Components: First, calculate the partial derivative of the center pressing depth with time to obtain the pressing velocity. Multiply the pressing velocity by the viscosity coefficient in the reference physical parameters to obtain the dynamic damping component (mechanical dimension: N). Second, extract the surface deformation curvature of the current and previous visual rendering frames to calculate the discretized transient deformation rate. Perform nonlinear compression mapping: introduce a preset anti-zero offset. ( For extremely small real numbers greater than 0, such as 0.001; a logarithmic modulation function is used to... Perform nonlinear compression mapping: introduce a preset anti-zero offset. ( For extremely small real numbers greater than 0, such as 0, a rate modulation variable is obtained by nonlinear compression mapping. ; rate modulation variable The vibration gain component is generated by multiplying the vibration conversion coefficient (a unit of uniformity, denoted as N·s) preset in the reference physical parameters. .

[0042] S103-3, Generation of dynamic haptic drive parameters based on gating judgment and smoothing filtering: a preset static deformation threshold. (This threshold is a real number greater than 0, used to define the minimum rate limit for macroscopic dynamic deformation of the software); determine the discretized transient deformation rate calculated in S103-2 above. Is it greater than the static deformation threshold? :like If no drastic dynamic surface deformation is detected, the basic elastic force component and the dynamic damping component are directly vector-added, and the output is the dynamic haptic drive parameter for the current frame; if The determination was accompanied by a significant abrupt change in curvature, and the basic elastic force component, dynamic damping component, and vibration gain component were combined. Numerical addition coupling is performed to obtain composite haptic parameters; then the parameters are input to a first-order low-pass smoothing filter to remove step high-frequency noise, and finally output continuously updated dynamic haptic driving parameters.

[0043] Based on the above, it should be noted that existing technologies typically use a linear spring model (damping = stiffness × depth) to calculate the pressing force, which does not conform to the "hardening" physical characteristic of real soft materials (such as clay and silicone) where resistance increases dramatically with deeper pressing. This invention introduces a material hardening constant with an exponent greater than 1 for power function calculation, resulting in a non-linear exponential increase in reference damping with pressing depth. This accurately reproduces the physical resistance variation law when pressing deep into soft materials, avoiding the distortion of "too hard for light pressing and too soft for heavy pressing" caused by linear models.

[0044] It should also be noted that real software experiences only resistance and no vibration when statically subjected to pressure. This invention sets a static deformation threshold, outputting only reference damping at low speeds, thus preventing minute calculation fluctuations from being mismapped as vibrations and generating noise that causes a "static numbness." When vibration compensation is triggered by high-speed deformation, the low-pass smoothing filter algorithm effectively eliminates the digital step caused by inter-frame data difference calculations, ensuring that the driving parameters received by the haptic device are continuous and smooth. This prevents mechanical stuttering or harsh noise caused by signal abrupt changes in the actuator, guaranteeing the smoothness of the immersive experience.

[0045] The fourth step of the immersive art design interaction method in virtual reality scenarios establishes a temporal synchronization mechanism between visual rendering and tactile feedback, achieving synchronized output between tactile feedback signals and visual deformation processes. Compared to existing technologies where visual and tactile feedback are independent and prone to time delays and misalignments, this solution effectively reduces the time difference between continuous visual changes and tactile feedback, alleviates cross-modal sensory conflicts, and avoids the perceptual disconnect of "seeing three-dimensional objects and touching flat surfaces," thereby enhancing the immersion, spatial realism, and three-dimensional material shaping experience in the virtual reality art design process.

[0046] In the fourth step, in response to the completion of the software visual rendering update of the current frame by the graphics rendering pipeline, the hardware timestamp of the rendered frame is extracted; the dynamic haptic drive parameters and the hardware timestamp are encapsulated into a haptic control data packet and sent to the haptic feedback device. Specific implementation methods include:

[0047] S104-1, Timestamp Capture Based on Vertical Synchronization Signal: Monitors the vertical synchronization signal at the end of the graphics rendering pipeline; when the graphics processor completes the visual deformation rendering update of the non-rigid software in the current frame, and at the instant of submitting the frame buffer and waiting for the display to refresh, an interrupt response is triggered to read the high-precision hardware timestamp of the current moment. (For example, using microsecond-level absolute time obtained from the system's high-precision event timer) to ensure that the timestamp is strictly aligned with the actual display time of the visual image; at the same time, the hardware timestamp of the previous visual rendering frame is retrieved from the cache. .

[0048] S104-2, Visual Inter-Frame Period Calculation and Subframe Interval Division: Calculate the hardware timestamp difference between the current frame and the previous frame as the visual inter-frame period. ; Obtain the inherent hardware refresh cycle of the haptic feedback device ;by The number of subframes that can be divided is calculated using the interpolation step size. (i.e., rounding down), thus dividing the visual inter-frame period into... The subframes are divided into equally spaced subframe intervals.

[0049] S104-3, Generation of haptic subframe parameter sequence based on linear interpolation: Obtaining dynamic haptic driving parameters output from the previous visual rendering frame. Dynamic haptic drive parameters calculated with the current visual rendering frame ;exist Within the interval of subframes, for the first... Subframes (of which) From 1 to (integers), calculate their interpolation weights. Then, the first is generated through linear interpolation. Parameters of each haptic subframe ; Traversal From 1 to , generated by The haptic subframe parameter sequence; simultaneously, based on the start timestamp With the interpolation step size, assign a corresponding absolute interpolation timestamp to each subframe parameter. .

[0050] S104-4, Data Encapsulation and Precise Synchronization Based on Timestamps: The haptic subframe parameter sequence and its corresponding interpolated timestamp sequence are sequentially encapsulated into a haptic control data packet queue according to chronological order; this data packet queue is written in batches to the internal circular buffer of the haptic feedback device through a high-bandwidth, low-latency interface; the microcontroller of the haptic feedback device maintains a local hardware clock, and by comparing the local clock with the interpolated timestamps parsed from the data packets, when the corresponding time point is reached, the corresponding subframe parameters are sequentially extracted from the circular buffer and converted into analog drive signals for output to the actuator; if the local clock is ahead of the interpolated timestamp, the output is temporarily suspended; if the local clock is behind, the outdated data packets are discarded, and the system directly jumps to the subframe parameter output corresponding to the current timestamp to prevent cumulative delay in haptic feedback.

[0051] It should be further explained that existing technologies mostly employ simple queuing and distribution at the software level, which, due to operating system scheduling delays, easily leads to a misalignment where "the deformation is seen first, followed by the tactile sensation." This invention, by directly extracting the hardware timestamp of the graphics rendering pipeline completion moment as a synchronization anchor, binds the playback time of tactile data to the vertical synchronization signal of the visual image on the underlying hardware clock. This effectively eliminates the uncertain delays caused by software scheduling, achieving millisecond-level strict synchronization of visual and tactile sensations across modalities, thus avoiding dizziness caused by sensory misalignment for the user.

[0052] like Figure 2 The diagram shown is a structural schematic of an immersive art design interaction system in a virtual reality scene provided in an embodiment of this application. (Refer to...) Figure 2The system includes: an interactive data acquisition module, a deformation feature extraction module, a haptic drive mapping module, and a visual-touch synchronization output module. The interactive data acquisition module is used to acquire the spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene, as well as the 3D mesh model and reference physical parameters of the non-rigid soft art material. The reference physical parameters include stiffness coefficient, viscosity coefficient, and vibration conversion coefficient.

[0053] The deformation feature extraction module is used to perform spatial mapping and collision detection between the spatial coordinate tracking sequence and the 3D mesh model to determine the interactive contact area containing multiple mesh vertices; extract the set of 3D displacement vectors of mesh vertices in the interactive contact area relative to the initial topological state, and calculate the maximum value of the modulus of the 3D displacement vector set to determine the center pressing depth; at the same time, calculate the ratio of the angle between the normal vectors of adjacent mesh vertices in the interactive contact area to the spatial distance between vertices to obtain the local discrete curvature, and statistically analyze the average value of the local discrete curvature in the interactive contact area as the surface deformation curvature.

[0054] The haptic drive mapping module takes the center pressing depth and stiffness coefficient as input variables and calculates the basic elastic force component through a preset basic elastic force mapping function; it calculates the partial derivative of the center pressing depth with time to obtain the pressing speed, and combines the pressing speed with the viscosity coefficient to calculate the dynamic damping component; it calculates the partial derivative of the surface deformation curvature with time to obtain the transient deformation rate, and uses the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter to calculate the vibration gain component; it then performs vector superposition coupling of the basic elastic force component, the dynamic damping component, and the vibration gain component to generate continuously updated dynamic haptic drive parameters.

[0055] The visual-touch synchronization output module is used to respond to the software visual rendering update of the current frame completed by the graphics rendering pipeline, extract the hardware timestamp of the rendering frame, encapsulate the dynamic haptic drive parameters and the hardware timestamp into a haptic control data packet and send it to the haptic feedback device to trigger the haptic feedback device to synchronously output the corresponding haptic drive signal according to the hardware timestamp.

[0056] Building upon the above, it is necessary to further elaborate that the four modules of this invention are closely linked to data flow and physical logic, forming a deeply coupled data flow link of "spatial input - geometric quantization - physical mapping - sensory closed loop." Specifically, the interactive data acquisition module and the deformation feature extraction module work together to accurately translate the original spatial data into "center pressing depth" and "surface deformation curvature," which have macroscopic force and microscopic morphological significance. This breaks through the limitations of traditional single collision detection and provides high-fidelity geometric input for tactile generation. The deformation feature extraction module and the tactile drive mapping module work together to generate basic elastic force by combining depth with stiffness, and to generate dynamic damping and curvature by combining depth change rate with viscosity coefficient. The rate of change combined with the vibration conversion coefficient generates vibration gain, realizing a continuous dynamic mapping from geometric deformation to composite physical tactile sensation, overcoming the tactile discontinuity caused by discrete waveforms. The tactile driving mapping module works in conjunction with the visual-tactile synchronization output module, using hardware timestamp anchoring and subframe interpolation technology to convert continuously calculated tactile parameters into execution signals that are time-continuous and strictly aligned with visual frames, eliminating timing conflicts and stuttering between the computational domain and the perceptual domain. Finally, the four modules work together in a closed loop, enabling visual deformation and hand tactile sensation to achieve a high degree of unity in spatial position, deformation amplitude, and temporal rhythm during user interaction, realizing a realistic reproduction of depth perception and physical shaping of non-rigid soft three-dimensional materials.

[0057] Overall, this invention addresses the pain point of lacking real-time dynamic tactile processing and continuous mapping for non-rigid soft materials in existing technologies. It constructs a complete technical chain from multi-dimensional physical deformation feature extraction to continuous dynamic tactile processing, and finally to frame-level synchronization of visual and tactile sensation. By introducing stiffness and viscosity physical parameters, this invention accurately extracts the center pressing depth and surface deformation curvature, and innovatively calculates them as continuous dynamic tactile driving parameters coupled with reference damping and vibration gain, effectively breaking the limitations of existing technologies that rely on preset discrete waveforms. Simultaneously, it achieves precise alignment of tactile sensation and visual rendering by relying on hardware timestamps, fundamentally eliminating cross-modal sensory conflicts. This invention effectively solves the perceptual illusion of "seeing three-dimensional objects while touching a flat surface," enabling users to obtain highly realistic dynamic tactile feedback that is strictly synchronized with visual deformation in virtual reality art creation. It truly realizes deep perception and physical shaping of non-rigid soft materials, completely closing the loop of the interactive experience of immersive art design, and has significant application value in promoting the development of virtual reality interactive technology towards high fidelity and cross-modal integration.

[0058] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0059] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0060] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units, located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0061] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0062] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the solution, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0063] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An immersive art design interaction method in a virtual reality scenario, characterized in that, Includes the following steps: The spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene is obtained, as well as the three-dimensional mesh model and reference physical parameters of the non-rigid soft art material. The reference physical parameters include stiffness coefficient, viscosity coefficient and vibration conversion coefficient. Spatial coordinate tracking sequences are spatially mapped and collision detected with a 3D mesh model to determine interactive contact areas containing multiple mesh vertices. A set of 3D displacement vectors of mesh vertices relative to the initial topological state within the interactive contact area is extracted, and the maximum magnitude of the 3D displacement vector set is calculated to determine the center pressing depth. Simultaneously, the ratio of the angle between the normal vectors of adjacent mesh vertices within the interactive contact area to the spatial distance between vertices is calculated to obtain the local discrete curvature, and the average value of the local discrete curvature within the interactive contact area is statistically analyzed as the surface deformation curvature. Using the center pressing depth and stiffness coefficient as input variables, the basic elastic force components are calculated through a preset basic elastic force mapping function; The partial derivative of the center pressing depth with time is used to obtain the pressing speed. The pressing speed is combined with the viscosity coefficient to calculate the dynamic damping component. The transient deformation rate is obtained by calculating the partial derivative of the surface deformation curvature with time. The vibration gain component is calculated by using the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter. The basic elastic force component, dynamic damping component, and vibration gain component are vector-coupled to generate continuously updated dynamic tactile drive parameters. In response to the completion of software visual rendering updates for the current frame by the graphics rendering pipeline, extract the hardware timestamp of the rendered frame. The dynamic haptic drive parameters and hardware timestamps are encapsulated into a haptic control data packet and sent to the haptic feedback device to trigger the haptic feedback device to synchronously output the corresponding haptic drive signal according to the hardware timestamp.

2. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The step of spatially mapping and collision detection between the spatial coordinate tracking sequence and the 3D mesh model to determine the interactive contact area containing multiple mesh vertices specifically includes: Construct a hierarchical bounding box tree structure for the 3D mesh model; Intersection test between the current terminal spatial coordinates in the spatial coordinate tracking sequence and the hierarchical bounding box tree structure is performed to filter out the candidate mesh patch set; Calculate the vertical projection distance from the current terminal spatial coordinates to each candidate mesh patch, extract the vertices of the patches whose vertical projection distance is less than a preset contact threshold, and construct the interactive contact area from the patch vertices.

3. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The reference physical parameters also include the two-dimensional tangential anisotropic friction tensor; After extracting the set of three-dimensional displacement vectors of the mesh vertices relative to the initial topological state within the interactive contact region, the following is also included: Extract the spatial movement trajectory of the interactive terminal, project the spatial movement trajectory onto the tangential plane of the interactive contact area, and obtain the tangential sliding velocity vector; Based on the two-dimensional anisotropic friction tensor of the tangential plane, the direction of the tangential sliding velocity vector is identified and the corresponding directional friction coefficient is extracted. The projection of the basic elastic force component in the normal direction of the interactive contact area is obtained as the normal force. The directional friction coefficient, the normal force, and the unit vector in the opposite direction of the tangential sliding velocity vector are multiplied to generate the tangential friction damping component. The tangential friction damping component is vector-superimposed with the dynamic haptic actuation parameters to synthesize and update the final dynamic haptic actuation parameters.

4. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The process of determining the center pressure depth then includes: Determine whether the center pressing depth is greater than the local maximum deformation tolerance of the 3D mesh model in the interactive contact area; When the center compression depth is greater than the local maximum deformation tolerance, the calculated value of the center compression depth is truncated to the value of the local maximum deformation tolerance. At the same time, the stiffness coefficient is amplified according to the preset penalty ratio, and the reference damping component is updated.

5. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The calculation of the ratio of the angle between the normal vectors of adjacent mesh vertices within the interactive contact area to the spatial distance between vertices is used to obtain the local discrete curvature, and the average value of the local discrete curvature within the interactive contact area is calculated as the surface deformation curvature. Specifically, this includes: Extract the target mesh vertices within the interactive contact area, as well as the adjacent mesh vertices that have a direct topological connection with the target mesh vertex; Calculate the spatial angle between the normal vector of the target mesh vertex and the normal vector of each adjacent mesh vertex, and obtain the spatial distance between the target mesh vertex and each adjacent mesh vertex; Divide the spatial angle by the corresponding spatial distance to obtain the local discrete curvatures belonging to the vertex of the target mesh; Traverse all mesh vertices within the interactive contact area, summarize all local discrete curvatures and calculate their average value, and use this average value as the surface deformation curvature.

6. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The process of using the center pressing depth and stiffness coefficient as input variables and calculating the basic elastic force components through a preset basic elastic force mapping function specifically includes: The depth deformation variable is obtained by performing a power function operation with the center pressing depth as the base variable, and the exponent term of the power function operation is configured to be a material hardening constant greater than one. The foundation resistance value is obtained by multiplying the depth deformation variable and the stiffness coefficient, and the foundation resistance value is used as the foundation elastic force component.

7. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The transient deformation rate is obtained by calculating the partial derivative of the surface deformation curvature over time. Using the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter, the vibration gain component is calculated, specifically including: Extract the surface deformation curvature between the current visual rendering frame and the previous visual rendering frame, calculate the absolute value of the difference between the two, and divide it by the time interval between adjacent rendering frames to obtain the discretized transient deformation rate. A logarithmic modulation function is used to perform nonlinear compression mapping on the transient deformation rate. The logarithm of the sum of the transient deformation rate and the preset zero offset is taken to obtain the rate modulation variable. The vibration gain component is generated by multiplying the rate modulation variable with the vibration conversion coefficient.

8. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The process of vector-coupled superposition of the basic elastic force component, dynamic damping component, and vibration gain component to generate continuously updated dynamic tactile actuation parameters specifically includes: Determine whether the transient deformation rate is greater than the preset static deformation threshold; If it is not greater than the static deformation threshold, the sum of the basic elastic force component and the dynamic damping component is directly output as the dynamic tactile driving parameter. If the deformation exceeds the static deformation threshold, the basic elastic force component, dynamic damping component, and vibration gain component are numerically added together. After applying a low-pass smoothing filter algorithm to the addition result, the output is the dynamic tactile driving parameter.

9. The immersive art design interaction method in a virtual reality scene as described in claim 1, characterized in that, The step of encapsulating dynamic haptic driving parameters and hardware timestamps into a haptic control data packet and sending it to the haptic feedback device also includes: Calculate the hardware timestamp difference between the current visual rendering frame and the previous visual rendering frame, and use it as the visual frame period. Within the visual frame interval, the hardware refresh cycle of the haptic feedback device is used as the interpolation step. Linear interpolation is performed based on the dynamic haptic driving parameters of the previous frame and the dynamic haptic driving parameters of the current frame to generate a haptic subframe parameter sequence. The haptic subframe parameter sequence and the corresponding interpolation timestamp are encapsulated into a haptic control data packet and sent to the haptic feedback device.

10. An immersive art design interaction system in a virtual reality scene, employing the immersive art design interaction method in a virtual reality scene as described in any one of claims 1-9, characterized in that, include: Interactive data acquisition module, deformation feature extraction module, tactile drive mapping module, and visual-tactile synchronous output module; The interactive data acquisition module is used to acquire the spatial coordinate tracking sequence of the interactive terminal in the virtual reality scene, as well as the three-dimensional mesh model and reference physical parameters of the non-rigid soft art material. The reference physical parameters include stiffness coefficient, viscosity coefficient and vibration conversion coefficient. The deformation feature extraction module is used to perform spatial mapping and collision detection between the spatial coordinate tracking sequence and the three-dimensional mesh model to determine the interactive contact area containing multiple mesh vertices; extract the set of three-dimensional displacement vectors of the mesh vertices in the interactive contact area relative to the initial topological state, calculate the maximum value of the modulus of the three-dimensional displacement vector set to determine the center pressing depth; at the same time, calculate the ratio of the angle between the normal vectors of adjacent mesh vertices in the interactive contact area to the spatial distance between vertices to obtain the local discrete curvature, and statistically analyze the average value of the local discrete curvature in the interactive contact area as the surface deformation curvature; The tactile drive mapping module is used to take the center pressing depth and stiffness coefficient as input variables and calculate the basic elastic force components through a preset basic elastic force mapping function; The partial derivative of the center pressing depth with time is used to obtain the pressing speed. The pressing speed is combined with the viscosity coefficient to calculate the dynamic damping component. The transient deformation rate is obtained by calculating the partial derivative of the surface deformation curvature over time. The vibration gain component is calculated by using the transient deformation rate as the independent variable and the vibration conversion coefficient as the weighted modulation parameter. The basic elastic force component, dynamic damping component and vibration gain component are vector-superimposed and coupled to generate continuously updated dynamic tactile driving parameters. The visual-touch synchronization output module is used to extract the hardware timestamp of the rendering frame in response to the completion of software visual rendering update of the current frame by the graphics rendering pipeline. The dynamic haptic drive parameters and hardware timestamps are encapsulated into a haptic control data packet and sent to the haptic feedback device to trigger the haptic feedback device to synchronously output the corresponding haptic drive signal according to the hardware timestamp.

Images