A real-time morphing method, system, storage medium and electronic device

Abstract

This application discloses a real-time deformation method, system, storage medium, and electronic device, relating to the field of virtual application technology. This solution introduces speed-modulated force calculation, density-adaptive deformation, and adjustments to the mesh complexity of the virtual sculpture model based on the interaction force and speed of the user object. This allows for real-time, adaptive adjustment of the virtual sculpture model's mesh complexity, thereby presenting a highly realistic physical deformation effect—i.e., visual realism—while ensuring real-time performance. Furthermore, through the topological structure, material property definition, and real-time processing optimization of the virtual sculpture model's mesh, this solution ensures that the model does not tear or distort after deformation and maintains texture consistency. This enables dynamic and local adjustments to the accuracy of the virtual sculpture model during sculpting using VR sculpting software, based on the real-time force intended by the user object.

Description

Technical Field

[0001] This application relates to the field of virtual application technology, and more specifically, to a real-time deformation method, system, storage medium, and electronic device. Background Technology

[0002] Virtual Reality (VR) sculpting software is an application that uses virtual reality technology to allow users to intuitively create, sculpt, and shape digital sculptures in a three-dimensional virtual space. Through VR headsets and controllers, it digitizes the traditional sculpting process, allowing users to seemingly mold virtual clay by hand in real air, thus breaking the limitations of physical materials and enabling free-form creation.

[0003] In the current process of sculpting using VR sculpting software, users face a dilemma: while low-poly models run smoothly, they cannot sculpt fine details; while high-poly models can display details, they quickly lead to a drop in frame rate and operation lag, destroying the immersive experience. This prevents users from dynamically and locally adjusting the model's precision based on their real-time operational intentions when sculpting with VR sculpting software. Summary of the Invention

[0004] In view of this, this application discloses a real-time deformation method, system, storage medium, and electronic device, aiming to present a highly realistic physical deformation effect during the sculpting process using VR sculpting software. It also dynamically and locally adjusts the precision of the virtual sculpture model based on the real-time force intention of the object being sculpted.

[0005] To achieve the above objectives, the disclosed technical solution is as follows:

[0006] The first aspect of this application discloses a method for real-time mesh deformation, the method comprising:

[0007] When an intersection between the manipulated object and the virtual sculpture model is detected, the force intent data packet of the manipulated object is obtained through collision detection.

[0008] Based on the force intention data packet, simulate the mechanical response data of the virtual sculpture model in the real world;

[0009] Deformation calculations are performed on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model;

[0010] If the visual deformation feedback signal does not meet the preset smoothing conditions, an optimization request signal is generated.

[0011] The optimization request signal triggers an adaptive adjustment of the mesh complexity of the virtual sculpture model.

[0012] A second aspect of this application discloses a real-time deformation system, the system comprising:

[0013] The first acquisition unit is used to acquire the force intention data packet of the operation object through collision detection when it is detected that the operation object intersects with the virtual sculpture model.

[0014] The simulation unit is used to simulate the mechanical response data of the virtual sculpture model in the real world based on the force intention data packet.

[0015] The second acquisition unit is used to perform deformation calculation on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model;

[0016] The generation unit is used to generate an optimization request signal if the visual deformation feedback signal does not meet the preset smoothing conditions.

[0017] An adaptive adjustment unit is used to trigger adaptive adjustment of the mesh complexity of the virtual sculpture model based on the optimization request signal.

[0018] A third aspect of this application discloses a storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides is controlled to perform the real-time deformation method as described in any one of the first aspects.

[0019] The fourth aspect of this application discloses an electronic device including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the real-time deformation method as described in any of the first aspects.

[0020] As described in the above technical solution, when the interaction between the object being manipulated and the virtual sculpture model is detected, the integration of collision detection, force analysis, deformation calculation, dynamic retopology, and real-time rendering enables the virtual sculpture model to adaptively deform and subdivide its mesh based on the posture and speed of the virtual hand upon contact. This solution introduces speed-modulated force calculation, density-adaptive deformation, and adjustments to the mesh complexity of the virtual sculpture model in real-time and adaptively based on the force and speed of the interaction between the object and the virtual sculpture model. This ensures highly realistic physical deformation effects, i.e., visual realism, while maintaining real-time performance. Furthermore, through the topological structure, material property definition, and real-time processing optimization of the virtual sculpture model's mesh, this solution ensures that the model will not tear or distort after deformation and maintains texture consistency. This allows for dynamic and localized adjustment of the virtual sculpture model's precision during the interaction process using VR sculpting software, such as during the sculpting process, based on the real-time force intention of the object being manipulated. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a real-time deformation method disclosed in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure of a real-time deformation system disclosed in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of the electronic device disclosed in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] As the background technology indicates, while low-polygon models run smoothly, they cannot sculpt fine details; while high-polygon models can display details, they quickly lead to a drop in frame rate and operational lag, disrupting the immersive experience. This prevents users from dynamically and locally adjusting the model's precision based on their real-time operational intentions when sculpting with VR software.

[0028] To address the aforementioned issues, this application discloses a real-time deformation method, system, storage medium, and electronic device. When an interaction between an object and a virtual sculpture model is detected, the method integrates collision detection, force analysis, deformation calculation, dynamic retopology, and real-time rendering. This allows the model to adaptively deform and subdivide its mesh based on the virtual hand's posture and speed upon contact. This solution introduces speed-modulated force calculation, density-adaptive deformation, and real-time adaptive adjustment of the virtual sculpture model's mesh complexity based on the force and speed of the interaction between the object and the virtual sculpture model. This achieves a highly realistic physical deformation effect, i.e., visual realism, while ensuring real-time performance. Furthermore, through the topological structure, material property definition, and real-time processing optimization of the virtual sculpture model's mesh, this solution ensures that the model does not tear or distort after deformation and maintains texture consistency. This enables dynamic and local adjustments to the virtual sculpture model's precision based on the user's real-time operational intentions when the object interacts with the VR sculpting software, such as during the object's sculpting process. Specific implementation details are provided in the following embodiments.

[0029] refer to Figure 1 The image shows a real-time deformation method disclosed in an embodiment of this application. The real-time deformation method mainly includes the following steps:

[0030] S101: When an intersection between the object being manipulated and the virtual sculpture model is detected, the force intent data packet of the object being manipulated is obtained through collision detection.

[0031] In S101, when the interaction between the operation object (such as an artist) and the virtual sculpture model is detected (such as when the operation object is sculpting the virtual sculpture model), it is necessary to capture the intention of the operation object, that is, capture the transition from physical gestures to digital signals.

[0032] When the system detects an object sculpting a virtual sculpture model, it needs to accurately and seamlessly capture which finger the object is using, in what posture, at what position on the virtual sculpture model, and with what force. This process discretizes continuous physical world information into computer-processable data. Therefore, it requires intent capture through collision detection, that is, converting physical gestures into digital signals to obtain the object's force intent data packet.

[0033] The collision detection method is a detection method that continuously checks whether the virtual hand collider (such as two small spheres bound to the tips of the index finger and thumb) intersects with the clay model collider (simplified low-poly convex hull).

[0034] The specific process of obtaining the force intent data packet of the operation object through collision detection is shown in A1-A2.

[0035] A1: When it is detected that the object is sculpting the virtual sculpture model, the input signal of the intersection between the object and the virtual sculpture model, the virtual reality (VR) tracking data, and the collision detection data of the physical engine between the object and the virtual sculpture model are obtained.

[0036] The collision detection data includes at least the pinch force value, contact data, and the instantaneous velocity vector of the controller.

[0037] Input signals include, but are not limited to, controller input signals.

[0038] VR tracking data can be obtained through pose tracking technology using VR headsets and controllers.

[0039] A2: Based on VR tracking data, controller input signals, and collision detection results, obtain the force intention data packet of the manipulated object.

[0040] In the specific application scenario of VR sculpture, VR tracking data, controller input signals and collision detection results of the physics engine are systematically integrated and synchronized to form a unified force intention data package.

[0041] The specific process of obtaining the force intent data packet of the manipulated object based on virtual tracking data, input signals, and collision detection results is shown in B1-B8.

[0042] B1: Acquire the virtual head-mounted display signal of the target object and the controller signal generated when the target object holds the controller.

[0043] In B1, when the object being operated is detected to be performing a pinching action, the controller's capacitive sensor or analog trigger detects the pressure change and generates a continuous voltage signal. This continuous voltage signal is the controller signal generated when the object is gripping the controller.

[0044] B2: Input the virtual head-mounted display signal and the controller signal as controller input signals to the virtual system and run it.

[0045] B3: During the operation of the virtual system, virtual tracking data is acquired at a preset frequency; the virtual tracking data includes at least controller position information, controller rotation information, and finger posture data.

[0046] When the user puts on a Virtual Reality (VR) headset and holds the controller, the headset's inside-out tracking camera and the controller's internal Inertial Measurement Unit (IMU) begin to operate continuously.

[0047] VR headsets include, but are not limited to, high-end VR headsets (Meta Quest Pro).

[0048] The preset frequency can be 90 times per second, 95 times per second, etc. The preset frequency is determined according to the actual situation and is not specifically limited in this application.

[0049] For example, when a VR system (such as the Oculus VR API or OpenXR) is running, it outputs a stable pose data stream at a frequency of 90 times per second. This pose data stream includes at least controller position data, controller rotation data, and finger pose data. A precise pose data stream is fundamental to all spatial computations, enabling the virtual hand to stably appear in the correct position.

[0050] The controller position data is a three-dimensional coordinate (X, Y, Z) based on the origin of the VR tracking space.

[0051] The controller rotation data is a quaternion (Qx, Qy, Qz, Qw) representing the orientation.

[0052] For controllers that support gesture tracking, the system will directly output finger posture data of a virtual hand skeleton; for standard controllers, it will map the pinching state of the thumb and index finger through predefined animations.

[0053] The controller position data and controller rotation data are the data sources for calculating the instantaneous velocity vector (v_vector). Stable controller position data and controller rotation data can avoid calculation errors in the instantaneous velocity vector and improve the accuracy of subsequent force calculations.

[0054] B4: Obtain the pressure changes and contact point data structure when the operation object intersects with the virtual sculpture model.

[0055] It should be noted that in order to obtain the pressure changes and contact point data structure when the manipulated object intersects with the virtual sculpture model, a collider query is required. Specifically, this involves continuously checking whether the virtual hand collider (such as two small spheres attached to the tips of the index finger and thumb) intersects with the clay model collider (a simplified low-polygon convex hull). This process is accomplished by the engine's physics engine (such as PhysX).

[0056] B5: Determine the kneading force value based on pressure changes.

[0057] In B5, the controller signal is converted into a digital value by the controller firmware and normalized by the VR runtime to a pinch force value in the range of [0.0, 1.0].

[0058] Here, 0.0 represents complete release; 1.0 represents maximum pressure; the pinching force value is represented by p. The pinching force value output in this step will be used as one of the core parameters and can be directly used to calculate the following posture force component (F_pose).

[0059] Transforming the subjective intention of the object being manipulated into an objective, programmable scalar is a prerequisite for achieving the different effects produced by actions such as "handling gently" and "gripping firmly".

[0060] B6: Extract contact data from the contact point data structure where the operation object intersects with the virtual sculpture model.

[0061] The contact data includes at least the coordinates of the contact point and the normal to the contact surface.

[0062] When an intersection (i.e., collision) is detected between the manipulated object and the virtual sculpture model, the physics engine generates a ContactPoint data structure, determines the contact point through the ContactPoint data structure, and extracts the contact data from the ContactPoint data structure.

[0063] The contact point coordinates refer to the three-dimensional location where the collision occurs on the model surface. The contact point coordinates are represented by P_contact. P_contact is used to determine the vertex extent of the deformation effect as described below.

[0064] The contact surface normal is the unit normal vector of the model's colliding surface at the contact point coordinates. The contact surface normal is represented by N_surface, which is the directional reference for the resultant force vector F.

[0065] B7: Obtain the position information of the controller in the previous frame and the position information in the current frame, and calculate the displacement based on the position information of the previous frame and the position information in the current frame.

[0066] The position information of the previous frame is represented by P_previous. P_contact is used to determine the vertex range affected by deformation in the following sections.

[0067] The displacement is calculated using formula (1) based on the position information of the previous frame and the position information of the current frame.

[0068] ΔS=P_current-P_previous(1);

[0069] Where ΔS is the displacement; P_current is the position information of the current frame; and P_previous is the position information of the previous frame.

[0070] B8: Calculate the instantaneous velocity vector of the controller using displacement and frame time.

[0071] The frame time is represented by ΔT. The frame time is typically 1 / 90 of a second.

[0072] Specifically, the instantaneous velocity vector of the controller is calculated using formula (2).

[0073] v_vector=ΔS / ΔT(2;

[0074] Where v_vector is the instantaneous velocity vector.

[0075] Formula (2) can also be transformed into v_vector = (P_current - P_previous) / ΔT.

[0076] The instantaneous velocity vector is the basis for the velocity scalar (v) and the direction of force (i.e., the resultant force vector F). The velocity scalar (v) is the magnitude of v_vector.

[0077] It should be noted that P_current and N_surface define the origin and initial direction of the deformation. v_vector quantifies the dynamic characteristics of the interaction.

[0078] In practical applications, such as the "SenseClay" VR sculpting application, overly complex collision objects, like those with 90Hz collision detection, can become a performance bottleneck. Tracking system jitter can also lead to excessive noise in the v_vector. Therefore, optimization is necessary. Specifically, this optimization involves creating an extremely simplified convex hull collider for the high-poly sculpture; applying a simple low-pass filter to the calculated v_vector to smooth out high-frequency noise and prevent jitter in subsequent force calculations; ensuring that pose data, collision data, and input data are processed at the same timestamp; and ensuring data synchronization to avoid drift caused by data latency. In terms of user experience, the user is unaware of this step; the system responds immediately when the user reaches out and pinches the clay, making the entire process natural and smooth.

[0079] Without the above optimizations, users will experience virtual hand tremors, clipping issues, intermittent collision detection, or unresponsive force feedback. This disrupts the immersive experience, making users aware of the tremors, clipping, and inconsistent collision detection, thus distracting them during the virtual sculpting process and reducing their user experience.

[0080] S102: Based on the force intention data packet, simulate the mechanical response data of the virtual sculpture model in the real world.

[0081] In S102, intelligent force application simulates the mechanical response data of the virtual sculpture model in the real world, that is, simulating the response of real materials based on the force intention data package. Specifically, when the object interacts with the virtual clay at different speeds and forces, the system needs to simulate the mechanical response data corresponding to different materials in the real world (such as soft clay, hard terracotta, etc.). The core of this step is to establish a material brain, so that actions such as rapid tapping and slow kneading of the object can produce distinctly different deformation driving signals that conform to physical intuition.

[0082] In this step, the force mapping software module continuously receives the aforementioned force intention data packet, and inputs the instantaneous velocity vector, pinching force value, and contact point normal from the force intention data packet as input data streams into the force mapping software module.

[0083] Specifically, based on the force intention data package, the process of simulating the mechanical response data of the virtual sculpture model in the real world is shown in C1-C6.

[0084] C1: Calculate the magnitude of the velocity vector based on the instantaneous velocity vector.

[0085] In C1, the force mapping software module calculates a scalar value representing the speed of motion based on the input data stream, namely the velocity vector magnitude, which is represented by v=||v_vector||.

[0086] C2: Obtain mass, exponential decay function, attitude sensitivity coefficient, stiffness coefficient, and attitude factor.

[0087] In C2, when the material selected by the operation object to sculpt the virtual sculpture model is obtained, the preset parameters corresponding to the material are retrieved from the internal database; among them, the preset parameters include at least mass, exponential decay function, posture sensitivity coefficient, stiffness coefficient, damping coefficient and posture factor.

[0088] Materials can be selected from the material library via the SenseClay system's UI, such as soft clay. The system then loads a set of preset parameters bound to that material from its internal database.

[0089] Here, mass is represented by m, which represents the inertia of the model and is set to 1.0 (relative value).

[0090] The damping coefficient is represented by λ. Soft clay has a relatively high λ value (e.g., 0.8) to simulate its energy absorption and resistance to rebound. The damping coefficient defines the material's stiffness and perceived energy absorption. Physically, in the impact force formula F_impact=m*v*e^(-λ*v), λ controls the nonlinear attenuation of force by velocity.

[0091] The specific explanation of how λ controls the nonlinear attenuation intensity of force due to velocity is as follows:

[0092] Increasing the λ value, for example from 0.3 to 1.0, makes the material feel softer and more "fleshy," like clay or wet clay. Even when slapped hard and quickly, the impact force is absorbed in large quantities, and the deformation is gentle and saturated, without producing excessive bouncing or splashing.

[0093] Lowering λ, for example from 1.0 to 0.1, makes the material feel harder and more "elastic," like hard clay or elastic putty. It is sensitive to speed, and rapid operation will produce a strong impact, with obvious deformation and possibly strong rebound. Therefore, λ controls how the material responds to "speed," determining whether the material is "stable and heavy" or "sensitive and bouncy."

[0094] The stiffness coefficient is represented by k_s. The stiffness coefficient is used to define the model's ability to resist deformation; soft clay has a lower stiffness coefficient.

[0095] The posture sensitivity coefficient is represented by k_p. The posture sensitivity coefficient is used to control the contribution of the kneading motion to the final force.

[0096] By incorporating art direction into physical simulation through preset parameters, this forms the technical basis for achieving different material textures.

[0097] The visual and interactive style of the entire deformation system is determined by performing the following deformation calculations using preset parameters (the specific calculation process is C3-C6 below).

[0098] C3: Calculate the instantaneous impact force based on mass, exponential decay function, and velocity vector magnitude.

[0099] If the existing simple linear model F_impact=m*v is used, the calculated force will be very large when the object moves its hand quickly, resulting in excessive and unrealistic deformation that does not conform to the energy loss of the material under high-speed collision. Therefore, the force mapping module uses an algorithm formula containing nonlinear terms to calculate the impact force component. The algorithm formula containing nonlinear terms is shown in formula (3).

[0100] F_impact=m*v*e^(-λ*v)(3);

[0101] Where F_impact is the instantaneous impact force; e^(-λ*v) is the numerical decay function; when the magnitude v of the velocity vector is very small, that is, v is close to 1, the force is mainly determined by m*v; when v is very large, that is, v rapidly approaches 0, thus suppressing the growth of force, simulating the energy absorption and damping effect under high-speed collision. v=||v_vector||.

[0102] F_impact=m*v*e^(-λ*v) is an empirical model invented to simulate nonlinear material damping in VR. It quantifies abstract material properties (such as "softness") into specific, adjustable algorithm parameters (m, λ, k_s, k_p) and integrates them into real-time force calculation.

[0103] Formula (3) is not a standard formula in classical physics, but an empirical model invented to simulate a specific material feel in VR. Its innovation lies in combining material properties (λ) with motion dynamics (v) through an exponential decay term to simulate nonlinear damping effects.

[0104] Formula (3) ensures the rationality of the physical simulation and the controllability of the art. It makes the transition between operations such as "tap" and "hit" smooth and realistic, rather than a simple linear amplification.

[0105] Furthermore, F_impact calculated by formula (3) is the main component of the final resultant force, which will directly determine the initial intensity and depth of deformation. Formula (3) is a key converter that elevates low-level sensor data to a high-level, physically meaningful "creative intent". The system uses formula (3) to distinguish between "patting" and "kneading" operations, as well as to simulate different materials.

[0106] To facilitate understanding of formula (3), a practical application scenario analysis is presented here, as follows:

[0107] In the SenseClayVR Sculpture app, the following is how it works:

[0108] Because the exponential operation e^(-λ*v) is computationally more expensive than linear operations, it can lead to performance overhead when multiple collision calculations are required in each frame.

[0109] Solution:

[0110] Lookup table: A lookup table can be pre-computed to map v to the value of e^(-λ*v), converting the more performance-intensive exponential operation into a fast memory access.

[0111] Shader computation: This computation and vertex shading weights can be moved to the GPU shader for execution, especially when multiple touch points need to be processed in parallel.

[0112] User experience and artistic control of the manipulated object:

[0113] Successful results: When the material selected is soft clay, it feels soft, easily deformed, and has a slow rebound; when "hard clay" is selected, it feels hard, requires more force to deform, and has a fast rebound. Different sculpting effects can be achieved intuitively by changing the force and speed of the gestures.

[0114] Workflow optimization: SenseClayVR provides a debug panel for manipulated objects, allowing real-time adjustment of parameters such as λ and k_p, with immediate visual feedback of deformation changes in VR. This makes creating new virtual materials (such as "elastic clay" or "slippery mud") a highly visualized and iterative process.

[0115] C4: Calculate the postural force generated by the kneading action based on the postural sensitivity coefficient and postural factor.

[0116] In C4, the force mapping software module simultaneously calculates the force components generated by the pinching action, that is, the posture force generated by the pinching action. Specifically, as shown in formula (4).

[0117] F_pose=k_p*p(4)

[0118] Where F_pose is the pose force generated by the pinching action; k_p is the pose sensitivity coefficient; and p is the pinching force.

[0119] Formula (4) is a linear mapping.

[0120] C5: The resultant force scalar is calculated based on the instantaneous impact force and the posture force.

[0121] In C5, the instantaneous impact force and the posture force are summed to obtain the resultant force scalar. Specifically, it is shown in formula (5).

[0122] F_magnitude=F_impact+F_pose(5);

[0123] Among them, F_magnitude is the resultant force scalar, that is, the magnitude of the resultant force; F_impact is the instantaneous impact force, which is the force generated due to the speed of hand movement (collision). The instantaneous impact force simulates the momentum effect of physical collision; F_pose is the pose force.

[0124] Formula (5) integrates the physical instantaneous impact force with the posture force representing the user's intention to manipulate the object within the same mechanical model. Formula (5) is the superposition of the "physical collision effect" and the "active manipulation intention of the object". For example, quickly slapping an object (high F_impact) and slowly squeezing it (high F_pose) can both produce a large resultant force, but the texture of the force is different. Formula (5) unifies them into a usable value.

[0125] F_pose ensures that even when the hand movement speed is zero (v=0), the object can still be shaped by the pinching action. The formula for calculating F_pose is shown in formula (6).

[0126] F_pose=k·p*p_e(6)

[0127] Here, F_pose is a scalar representing the magnitude of the force generated by the gesture of the manipulated object (such as the pressure of a pinch, hand posture, etc.); k is a proportionality coefficient or stiffness coefficient, an adjustable constant used to amplify or reduce the input signal to a suitable range of physical force; p is a pose factor, a raw input signal from the VR controller (such as a gamepad trigger or capacitive sensor), typically between 0 and 1, representing the degree of the user's finger pinch or the intensity of a specific gesture; p_e is a pose enhancement factor or another pose parameter. This may represent another dimension of gesture input (such as different combinations of fingers), or a coefficient used to adjust the influence of p.

[0128] The pose force F_pose in formula (6) is proportional to the "active control intention" of the object being manipulated. Even if the object's hand moves very slowly, as long as the object grips the object tightly (large p value), a significant force can be generated to simulate "pinching" or "grabbing" the object.

[0129] In formula (6), k_p defines the precision of gesture control.

[0130] Physical function of k_p: In formula (6), k_p amplifies the input signal p from the controller (such as the kneading force).

[0131] Adjusting the artistic meaning of k_p:

[0132] Increasing k_p amplifies the control over gestures, allowing even slight pinching of the object to produce significant deformation, making it suitable for fine carving. This is similar to using a very sharp and sensitive carving knife for intricate carving.

[0133] Lowering k_p reduces the control over gestures, requiring a firmer grip on the object to produce the same deformation, making it suitable for shaping large blocks. For example, it provides a more stable and less prone-to-accidental operation when using a heavy scraper.

[0134] k_p controls the material's sensitivity to "gestures," determining whether the tool is "sharp and precise" or "stable and heavy."

[0135] The significance and value of real-time adjustment of k_p:

[0136] Adjusting these parameters in real time within the SenseClay debug panel revolutionizes the workflow for manipulating objects:

[0137] WYSIWYG material creation:

[0138] The objects being manipulated are no longer pre-defined abstract numbers that are then compiled and tested by the program. Instead, it's like mixing clay in the real world; parameters are adjusted while the material changes are felt firsthand in VR. This instant feedback makes material creation highly intuitive and exploratory.

[0139] Achieving precise art direction:

[0140] Different works require different textures. Sculpting a cartoon character might require bouncy clay (low λ, medium k_p), while sculpting a realistic head requires thick, heavy clay (high λ, high k_p). Real-time parameter tuning allows the manipulated object to be quickly customized to best match the artistic style for each project.

[0141] Among them, the bouncy clay of cartoon characters:

[0142] The range of low λ (viscosity coefficient) is:

[0143] λ: 0.20~0.40;

[0144] The range of k_p (plastic coefficient) is:

[0145] k_p: 0.40~0.60;

[0146] Features: Quick deformation and rebound, bouncy feel, non-sticky when shaped, suitable for rounded cartoon shapes.

[0147] The realistic portrait features a thick layer of clay.

[0148] The range of high λ is:

[0149] λ: 0.70~0.90;

[0150] k_p: 0.70~0.90;

[0151] Features: Smooth and heavy deformation, no rebound, solid shaping, and fits the texture of clay in realistic sculpture.

[0152] Optimize user experience:

[0153] For the object being manipulated, these parameters determine the feel of the VR sculpture. By adjusting these parameters in real time, developers can ensure that everyone, from beginners to professional artists, receives a controllable, comfortable, and expressive interactive experience, avoiding overly stiff or limp manipulation.

[0154] By adjusting parameters in real time, the system essentially hands over control of the physical simulation to the object being manipulated. It transforms technical parameters from a black box into an intuitive set of material sculpting tools, allowing the artistic creativity of the object to directly and dynamically drive the technical expression. This is the core manifestation of the SenseClay system's power and innovation.

[0155] C6: Calculate the target force vector that drives the deformation of the virtual object based on the resultant force scalar and contact data, and use the target force vector as the mechanical response data of the virtual sculpture model in the real world.

[0156] In C6, the resultant force scalar is multiplied by the contact surface normal to obtain the final target force vector that drives the deformation of the virtual object, as shown in formula (7).

[0157] F=F_magnitude*N_surface(7);

[0158] Where F is the target force vector, a physical quantity with magnitude and direction; F_magnitude is the resultant force scalar; N_surface is the contact surface normal, and N_surface is the unit vector perpendicular to the model surface at the contact point.

[0159] Formula (7) is a vector composition. The magnitude of the resultant force calculated by Formula (7) is assigned a direction, thus forming a complete target force vector that can be used for physical calculations.

[0160] In formula (7), the force has a direction. In most real interactions, the force perpendicular to the surface is most likely to cause compression deformation. Therefore, multiplying the calculated F_magnitude by N_surface gives an F with the correct direction (perpendicular to the contact surface) and appropriate size. F will be directly input into the next deformation calculation to determine how the object is "pushed" and deformed.

[0161] Finally, vectorization ensures that the force is applied along the normal direction of the model surface, which is the basis for generating reliable extrusion deformation.

[0162] The goal of the entire C1-C6 process is to generate a target force vector that drives the deformation of the virtual object. The specific steps to obtain the target force vector are as follows:

[0163] 1. Collect input:

[0164] The system obtains user gesture data (p, p_e) from the sensors and motion velocity (used to calculate F_impact) and contact surface normal (N_surface) from collision detection.

[0165] 2. Calculate separately:

[0166] The force representing the user's intent is calculated using the formula F_pose=k·p*pe.

[0167] Calculate the force representing the physical collision using other formulas (such as F_impact=m*v*e^(-λ*v) mentioned above);

[0168] 3. Merging and Targeting:

[0169] Adding the two forces together gives the total magnitude, i.e., the resultant force scalar: F_magnitude = F_impact + F_pose;

[0170] Assign the correct direction to the target force vector F: F = F_magnitude * N_surface.

[0171] The final target force vector F is a high-quality input signal that conforms to both physical laws (considering collision speed and material damping) and user operation intentions (considering grip strength), and is used to generate realistic object deformation in the future.

[0172] S103: Perform deformation calculations on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model.

[0173] In S103, when the object being manipulated applies force to the virtual clay, the deformation engine module generates visual deformation feedback, i.e., a visual deformation feedback signal.

[0174] It should be noted that the system needs to determine whether the deformation is smooth enough under the current mesh precision, that is, whether it meets the preset smoothing conditions. If the deformation does not meet the preset smoothing conditions under the current mesh precision, an optimization request signal is generated. The deformation quality is evaluated through the optimization request signal to complete the topology optimization.

[0175] The optimization process involves data preparation and vertex selection, vertex displacement calculation, and deformation quality assessment.

[0176] Data preparation and vertex selection:

[0177] After the deformation engine module receives the target force vector and contact point coordinates from the force intent data packet, it reads the current mesh data of the virtual sculpture model from the graphics processing unit (GPU) or memory. The current mesh data includes an array of vertex positions and an array of triangle indices.

[0178] Determine the spherical influence region: The system creates a spherical influence region centered on P_contact, based on a preset influence radius (e.g., positively correlated with the magnitude of the force); iterates through all vertices of the spherical influence region, selecting vertices located within the spherical influence region to form a set of vertices to be deformed. The preset influence radius is represented by R.

[0179] It should be noted that the entire virtual sculpture model is not deformed for performance reasons. Processing only the vertices of the spherical influence area ensures real-time performance, while setting the influence radius to be related to the force aligns with physical intuition: the greater the force, the wider the influence range.

[0180] Vertex displacement calculation (involving decay weight calculation, displacement calculation, and vertex position update):

[0181] Attenuation weight calculation: For each vertex V_i in the set of vertices to be deformed, calculate the distance d_i from each vertex V_i to the contact point, and calculate an attenuation weight w_i based on d_i. Usually, a linear or Gaussian attenuation function is used so that the vertices farther away from the contact point have a smaller degree of deformation.

[0182] Displacement calculation: For each vertex V_i, calculate its displacement vector ΔV_i. Here, a simplified model based on Hooke's law is used to calculate the displacement. The simplified model based on Hooke's law is shown in formula (8):

[0183] ΔV_i=(F / k_s)*w_i*N_surface(8);

[0184] Where ΔV_i is the displacement vector of each vertex V_i; k_s is the stiffness coefficient read from the material library; N_surface is the contact surface normal; and w_i is the attenuation weight.

[0185] Vertex position update:

[0186] The displacement vector is added to the original position of the vertex, as shown in formula (9):

[0187] V_i_new=V_i_old+ΔV_i(9);

[0188] Where Vi_i_new is the updated vertex position; Vi_i_old is the vertex position before the update.

[0189] The vertex displacement formula can be derived from ΔV_i, as shown in formula (8) above.

[0190] Formula (8) is a variation of Hooke's Law (F=kx) and incorporates vertex weight masking, a technique commonly used in computer graphics. It is applied to real-time VR interaction and seamlessly integrated with the complex force model from the preceding steps.

[0191] This virtual sculpture model is computationally efficient and produces intuitive and reasonable visual deformations. Furthermore, the attenuation weights in the virtual sculpture model ensure a smooth transition between deformed and undeformed areas, avoiding abrupt boundaries.

[0192] The result of this step is the deformed mesh, which is the direct object of topology optimization. Furthermore, the accuracy of the vertex displacement calculation directly determines the value of subsequent optimizations.

[0193] Deformation quality assessment:

[0194] Problem identification: On low-polygon meshes, large deformations can cause triangles to be overstretched, resulting in sharp edges on the surface that appear rough and unrealistic. The system needs to automatically detect this situation.

[0195] Evaluation algorithm execution: Activate the detail evaluator module. Traverse the set of vertices to be deformed through the detail evaluator module and perform the following operations:

[0196] 1) Calculate the displacement magnitude ||ΔV_i|| at each vertex;

[0197] 2) Calculate the average value δ_avg and the maximum value δ_max of the displacement amplitude ||ΔV_i|| of each vertex;

[0198] 3) Analyze the rate of change of the triangle area within the deformation region or the angle between the normals of adjacent vertices. If the rate of change of the triangle area within the deformation region or the angle between the normals of adjacent vertices exceeds a certain threshold (set according to the actual situation, this application does not make a specific limitation), it indicates that the mesh quality is declining, and a mesh quality index is generated.

[0199] The average and maximum values ​​are used to evaluate the visual quality of deformation in real time and generate a generative optimization request signal that drives key downstream decisions.

[0200] δ transforms the visual "roughness" into a calculable, objective number. It provides the system with the ability to perceive its own state and is key to achieving intelligent optimization.

[0201] Without the δ signal, there would be no basis for intelligent decision-making, degenerating into blind subdivision. Therefore, the calculated δ value will be used as a core input parameter, directly passed to the topology management core to decide whether and how to trigger dynamic subdivision.

[0202] Generate optimization request signal:

[0203] By combining δ_avg, δ_max, and mesh quality metrics, a scalar visual deformation feedback signal is generated using a weighting function. A higher visual deformation feedback signal value indicates that the current mesh is less capable of representing the current deformation with high quality.

[0204] The visual deformation feedback signal is shown in formula (10):

[0205] δ=a*δ_avg+b*δ_max(10);

[0206] Where δ is the visual deformation feedback signal; a is the average value weighting coefficient; δ_avg is the average value of the displacement amplitude ||ΔV_i|| of each vertex; b is the maximum value weighting coefficient; and δ_max is the maximum value of the displacement amplitude ||ΔV_i|| of each vertex.

[0207] The value of 'a' determines the proportion of the average deformation degree (δ_avg) of the entire deformation area in the visual deformation feedback signal. Its function is to allow the system to "perceive" the prevalence and extent of deformation. If the value of 'a' is set large, the system will pay more attention to whether a large deformation has generally occurred across the entire area. For example, when the entire palm is gently pressed against clay, although the indentation at each point is not deep, the area is large. A large value of 'a' can sensitively capture this "large-area gentle deformation" and trigger optimization.

[0208] `b` is an adjustable maximum value weighting coefficient (usually a number between 0 and 1). `b` determines the proportion of the deformation degree (i.e., δ_max) at the most severe point within the deformation region in `δ`. `b` allows the system to perceive the extremes and local peaks of deformation. If the value of `b` is set large, the system will be particularly sensitive to drastic local deformations. For example, pressing clay hard with a fingertip will create a very deep but small-area indentation. A large value of `b` ensures that this "small-area depth deformation" is identified and optimized in a timely manner, avoiding the formation of rough, deep-hole edges.

[0209] Since using δ_avg or δ_max alone has its drawbacks, two coefficients, a and b, are needed to consider both artistic and technical aspects.

[0210] 1. The drawbacks of using only the average value (assuming only a, b=0) are as follows:

[0211] It will ignore severely damaged areas. For example, in a scene where most areas are only slightly stretched, a small area (such as the tip of a wrinkle) may be severely torn, but the overall average may still be low, causing the system not to optimize this most important area. Artistic consequence: The most noticeable and hardest-to-see damage will not be repaired.

[0212] 2. The drawback of using only the maximum value (assuming only b and a=0) is as follows:

[0213] It can be overly sensitive to noise or individual vertex anomalies. A tiny calculation error or jitter in a single vertex of a virtual sculpture model can generate a very high \(\de1ta_{\text{max}}\), causing the system to frequently trigger unnecessary optimizations and waste performance. This leads to system instability, potentially wasting resources on unnecessary detail additions and causing frame rate drops. Therefore, combining a and b is an intelligent design that balances "global" and "local," "stability" and "sensitivity."

[0214] Adjusting each volume:

[0215] For materials that require an overall smooth effect (such as sculpting rounded objects), you can increase a and decrease b; for materials that need to show sharp details or are concerned about local tearing (such as sculpting hair or cracks), you can increase b and decrease a.

[0216] In most cases, a and b will maintain a balance, such as a=0.7 and b=0.3, to ensure that the system is both stable and responsive.

[0217] a and b provide practical guidance for the objects being operated on:

[0218] In system debugging panels like SenseClay, 'a' and 'b' are provided directly to artists as sliders or parameters for adjustment.

[0219] The parameter name may be displayed as:

[0220] Average deformation weight;

[0221] Peak weight;

[0222] Or a more artistic overall smoothness and detail sensitivity.

[0223] The adjustment effect is as follows:

[0224] When 'a' is increased, the system tends to increase mesh subdivision when shaping large areas, making the overall area of ​​large deformation smoother. When 'b' is increased for the object being manipulated, the system reacts more violently to the brush tips, forceful stabbing, and other operations on the object, generating extremely high-density details in deep pits or sharp areas.

[0225] a and b are not merely parameters in a mathematical formula; they are two "tuning knobs" assigned to the object being manipulated. By adjusting a and b, the object can directly control the "personality" of this deformation monitor, whether it focuses more on large-area, gradual changes or on more intense, localized shaping. This allows the system to perfectly adapt to its specific artistic style and technical requirements, which is a key aspect of the solution's user-friendliness and innovation.

[0226] The specific process of performing deformation calculations on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model is shown in D1-D9.

[0227] D1: Create a spherical influence area based on contact data and a preset influence radius.

[0228] In D1, a spherical influence region is created based on the contact point coordinates and the preset influence radius.

[0229] D2: Select all vertices from the spherical influence region and form a set of vertices to be deformed based on all vertices in the spherical influence region.

[0230] The set of vertices to be deformed is determined for the precise target range. Topological subdivision will strictly limit this range to the spherical influence area, avoiding unnecessary calculations and thus improving efficiency.

[0231] D3: For each vertex in the set of vertices to be deformed, calculate the distance from each vertex to the contact data.

[0232] In D3, for each vertex in the set of vertices to be deformed, calculate the distance from each vertex to the coordinates of the contact point.

[0233] D4: Calculate the attenuation weight based on distance.

[0234] D5: Calculate the displacement vector of each vertex based on the target force vector, stiffness coefficient, attenuation weight, and contact data.

[0235] In D5, the displacement vector of each vertex is calculated based on the target force vector, stiffness coefficient, attenuation weight, and contact surface normal.

[0236] D6: Update the vertex positions based on the displacement vector to obtain the updated vertex positions.

[0237] D7: Calculate the displacement magnitude of each vertex after updating its position based on the displacement vector of each vertex.

[0238] D8: Calculate the average and maximum displacement magnitude of the vertex after each updated position.

[0239] D9: Generate visual deformation feedback signals for the virtual sculpture model based on the average value, maximum value, and weighting function.

[0240] The execution process and principle of D1-D9 are consistent with the execution process and principle of data preparation, vertex filtering, and vertex displacement calculation mentioned above, and can be referred to accordingly. They will not be repeated here.

[0241] The updated vertex positions Vi_i_new are submitted to the graphics API (such as OpenGL or DirectX), which is called submitting a deformation operation and updating the vertex buffer object (VBO). The GPU will then use the new vertex data to render the next frame, and the artist will immediately see the deformation effect.

[0242] Because the geometry has changed, the vertex normals are no longer accurate. Therefore, the system calls the engine's normal recalculation function to recalculate the normals, ensuring that the lighting and shadows are correct, so that the deformation looks three-dimensional and realistic.

[0243] The calculated δ signal is packaged together with F and v and passed to the decision module, which then decides whether and how to trigger dynamic subdivision.

[0244] Vertex position updates and normal recalculation are standard graphics operations that ensure visual accuracy. Data transfer is the foundation of communication between modular systems.

[0245] Deformation quality assessment process: Deformation calculation and real-time quality analysis are coupled in the same step.

[0246] Visual deformation feedback signal δ: Defines a specific, quantifiable parameter to characterize the stress of deformation on mesh quality.

[0247] Closed-loop design of computation-evaluation: This design thinking is a system-level innovation that enables the system to have the ability to self-awareness and seek help from downstream systems.

[0248] It is the sole execution step that generates visual deformation and is the cornerstone of the entire technical solution's visual performance. Simultaneously, the generated delta signal is the key switch that triggers intelligent behavior.

[0249] Analysis of practical application scenarios:

[0250] In the SenseClay VR Sculpture application, the deformation and quality monitoring functions as follows:

[0251] Performance and Implementation:

[0252] Challenge: Calculating deformation per vertex on the CPU may not be possible within a single frame (approximately 11ms) for models with more than 100,000 vertices.

[0253] Solution:

[0254] By writing the vertex displacement calculation in the Compute Shader, the GPU's parallel computing capabilities are utilized for GPU acceleration, enabling the processing of millions of vertices instantly.

[0255] Level of Detail (LOD): When calculating the delta signal, the LOD version of the model can be used first for calculation, and then the deformation result can be mapped back to the high-level model.

[0256] User experience:

[0257] Successful state: The object is pressed into the clay, creating a smooth indentation. When the system detects that the object is applying enough force, it understands the object's intention, and the details of the indentation instantly become richer and smoother, making the entire process seamless and responsive.

[0258] Failure status: Deformation delay, or flickering or tearing in the deformation area. The evaluation system is either insensitive, causing even very coarse deformations to fail to trigger optimization, or oversensitive, causing even slight touches to trigger dense subdivisions, resulting in performance lag.

[0259] S104: If the visual deformation feedback signal does not meet the preset smoothing conditions, generate an optimization request signal.

[0260] Specifically, if the visual deformation feedback signal does not meet the preset smoothing conditions, the process of generating an optimization request signal is shown in F1-F4.

[0261] F1: The decision score is calculated based on the visual deformation feedback signal, the target force vector, the velocity vector magnitude, and the preset weight coefficients.

[0262] F2: In the spherical influence area, compare the decision score with the dynamic threshold.

[0263] The dynamic threshold should be set according to the actual situation, and this application does not impose specific restrictions.

[0264] F3: If the decision score is greater than the dynamic threshold, it is determined that the visual deformation feedback signal does not meet the preset smoothing conditions.

[0265] F4: Generate an optimization request signal if the visual deformation feedback signal does not meet the preset smoothing conditions.

[0266] If the decision score is less than or equal to the dynamic threshold, the visual deformation feedback signal is determined to meet the preset smoothing conditions, the process terminates, and the system maintains the current state to save computing power.

[0267] The execution process and principle of F1-F4 are consistent with those of the above-mentioned deformation quality assessment, and can be referred to accordingly. They will not be repeated here.

[0268] In practical applications, the process of generating optimization request signals is as follows:

[0269] Step 1: Multimodal Decision Making—Determining “When” Requires Details:

[0270] Data aggregation: The topology management core module receives real-time data streams from upstream.

[0271] Visual deformation feedback signal δ;

[0272] Force F (F represents the “intensity” of the interaction) and velocity (v represents the dynamic characteristic).

[0273] Comprehensive Decision Making: Decisions are made based on a multi-factor decision-making algorithm and delta (δ). The logic is as follows:

[0274] If (δ is already very large, or although the deformation is moderate, the applied force is very strong or the speed is extremely fast, resulting in a strong impact), then more details are needed;

[0275] Multi-factor decision algorithm execution: The multi-factor decision algorithm integrates multiple inputs into a unified decision score. For example, a weighted sum is used, as shown in the decision score formula (11):

[0276] Decision_Score=α*F+β*v+γ*δ(11);

[0277] Wherein, Decision_Score is the decision score; α, β, and γ are pre-calibrated weighting coefficients, where α is used to balance the influence of force in decision-making; β is used to balance the influence of speed in decision-making; γ is used to balance the influence of the degree of deformation in decision-making; and δ is the visual deformation feedback signal.

[0278] The decision score formula integrates signals from different dimensions (force, velocity) in physical interaction with geometric state signals (degree of deformation) into a scalar decision value.

[0279] A single indicator is prone to misjudgment. For example, a slowly formed deep pit (high δ, low v) and a shallow mark formed by a rapid slap (low δ, high v) may both require additional details. This multi-factor model makes decision-making smarter and more robust.

[0280] The calculated Decision_Score is the sole, quantitative overall controller for all subsequent operations (whether to trigger, intensity subdivision).

[0281] Step 2: Dynamic threshold determination to decide whether to execute immediately:

[0282] Obtain environmental status: The system reads the base mesh density D0 of the current deformed region (e.g., the initial average number of triangles per square unit area of ​​the region).

[0283] Calculating the dynamic threshold: Instead of using a fixed threshold, the system calculates a dynamic threshold T_dynamic. The logic is that if the region is already relatively dense (D0 is high), then its ability to withstand deformation is stronger, and the threshold for triggering further subdivision should be increased accordingly.

[0284] Judgment: Compare Decision_Score with T_dynamic. Only when Decision_Score > T_dynamic will the subdivision operation be triggered. Otherwise, the process terminates, and the system maintains its current state to save computing power. The dynamic threshold formula is shown in formula (12):

[0285] T_dynamic=T_base+κ*D0(12);

[0286] Here, T_dynamic is the dynamic threshold, which is the actual decision threshold obtained by adjusting the base threshold (T_base) based on the base grid density D0 of the current region. Due to the addition of the term k × D0, when the region grid is already relatively dense (D0 is large), the dynamic threshold will increase accordingly, making it more difficult to trigger subdivision, thus avoiding unnecessary calculations in already sufficiently fine regions. T_base is the base threshold, a preset, fixed constant representing the minimum decision score required for the system to trigger subdivision operation without considering the current grid density. T_base can be understood as the system's default sensitivity benchmark. D0 is the base grid density of the current deformed region. κ is the gain coefficient, which controls the degree of influence of DO on T_dynamic. If the κ value is large, even if DO only increases slightly, the dynamic threshold will be significantly raised, and the system will become very conservative, making it difficult to trigger further subdivision in high-density regions. If the κ value is small, the influence of DO on the dynamic threshold is small, and the system can still relatively easily trigger subdivision in high-density regions.

[0287] The dynamic threshold formula introduces the concept of "dynamic," enabling the system's sensitivity to adaptively adjust based on the model's current geometric state, avoiding unnecessary over-subdivision in areas already possessing high detail. This is the core valve for achieving on-demand allocation and performance optimization. It prevents ineffective and performance-intensive subdivision calculations in areas with slight interactions or already high detail.

[0288] The result of whether Decision_Score > T_dynamic directly determines whether the expensive mesh subdivision algorithm will be called in this frame, and is a key safety lock for controlling performance overhead.

[0289] Step 3: Local Execution—Implementing "How" to Add Details:

[0290] Defining the operating region: The system strictly restricts operations to the triangular facets affected by the defined set of vertices to be deformed. This ensures the locality of subdivision.

[0291] Selecting and calling subdivision algorithms: The system calls mature subdivision surface algorithms from the graphics library, such as Catmull-Clark (for quadrilateral-dominated meshes) or Loop (for triangular meshes).

[0292] Controlling the subdivision intensity: The Decision_Score determines not only whether to subdivide, but also how much to subdivide. The system dynamically determines the number of subdivision iterations based on the Decision_Score. For example:

[0293] If the Decision_Score just exceeds the threshold, only one subdivision may be performed; if the Decision_Score is very large (the artist is working extremely hard), two or three subdivisions may be performed.

[0294] Each subdivision increases the number of faces by approximately four times. This variable number of subdivision iterations, driven by decision scores, is a concrete implementation of "dynamic density."

[0295] The computational resources (the increased number of triangles) are precisely matched to the intensity of the artist's interaction. The more effort is applied, the more detail is obtained, aligning with real-world creative intuition.

[0296] Output a completely new 3D mesh with increased density only in certain areas. This new mesh replaces the corresponding parts of the original mesh and is the direct carrier of the final visual result presented to the artist.

[0297] Step 4: Data Inheritance and Output

[0298] Deformation state transfer: The new vertices generated by subdivision do not know that they should be in a "concave" state. The system performs deformation calculations on the mechanical response data, that is, the deformation displacement calculated by the above formula (8) is interpolated from the old mesh to each vertex of the new mesh. This is usually achieved by barycentric coordinate interpolation or by a weighted average of nearest neighbor search based on spatial location.

[0299] Generate the final mesh: The new local high-precision mesh inherited from the deformation data is stitched together with the rest of the unchanged parts of the model to generate a complete new model containing different levels of detail.

[0300] Output to the rendering pipeline: The final mesh is submitted to the graphics rendering engine. Optionally, a subtle haptic feedback from a handle is triggered as a physical confirmation of the "level of detail enhancement." Without this step, the subdivided model would "bounce back" to its undeformed state, causing disastrous visual jumps and disrupting immersion. Therefore, this step ensures visual continuity.

[0301] The completion of this step marks the end of a full "perception-decision-execution-feedback" cycle. The high-quality mesh it outputs is the final artwork that the artist sees in the VR headset, inspiring their creative process.

[0302] Decision-making mechanism based on multi-factor physical context: Decision_Score=α*F+β*v+γ*δ.

[0303] Adaptive dynamic threshold system: T_dynamic = T_base + κ * D0.

[0304] Variable subdivision intensity control driven by decision scores (e.g., controlling the number of subdivision iterations).

[0305] S105: Trigger the execution of adaptive adjustment of the mesh complexity of the virtual sculpture model based on the optimization request signal to complete the dynamic mesh subdivision operation.

[0306] When the system detects that the current mesh precision is no longer able to elegantly represent the creative intent of the object being manipulated (i.e., the deformation is coarse), it automatically, locally, and to an appropriate degree, increases the geometric details of the model, making the deformation instantly smooth and refined, and the whole process is transparent and imperceptible to the object being manipulated.

[0307] The practical application scenarios of dynamic mesh subdivision operations are analyzed as follows:

[0308] In the SenseClay VR sculpting application, the on-demand additive manufacturing and dynamic mesh subdivision function works as follows:

[0309] Performance and implementation challenges:

[0310] Challenge: Mesh subdivision, especially multiple iterations, is a computationally intensive operation. Executing it synchronously on the main thread can cause a sharp drop in frame rate, resulting in stuttering.

[0311] Solution:

[0312] Asynchronous computation: The subdivision algorithm is run asynchronously on the GPU using a separate worker thread or a Compute Shader. The rendering loop continues to use the old mesh, and after subdivision is complete, it seamlessly switches to the new mesh in the next frame. The artist is completely unaware of the computation process.

[0313] Detailed boundary handling: The boundaries of locally subdivided regions need to transition smoothly with the surrounding low-precision mesh; otherwise, seams will appear. This requires additional mesh stitching algorithms and is a key aspect of engineering implementation.

[0314] User experience:

[0315] Successful state (magical effect): The artist is immersed in the creation. When the artist vigorously carves out an eye wrinkle, there are no loading icons or lags; only the clay under the brushstrokes naturally and instantly emerges with enough detail to represent the subtle indentation.

[0316] Parameter tuning: Adjusting the weighting coefficients (α, β, γ) and the threshold (T_base) is crucial. This determines the system's "personality"—whether it's responsive (prone to detail) or conservative (more performance-efficient). SenseClay provides advanced users with a tuning panel that allows adjusting these parameters like adjusting brush hardness.

[0317] This solution aims to enhance the realism of virtual hand-model interactions in VR games. By integrating collision detection, force analysis, deformation calculation, dynamic retopology, and real-time rendering, the model adaptively deforms and subdivides its mesh based on the virtual hand's posture and speed upon contact. The core innovation lies in introducing velocity-modulated force calculation, density-adaptive deformation, and real-time adaptive adjustment of the mesh complexity based on the intensity and speed of the interaction. This ensures highly realistic physical deformation effects while maintaining real-time performance, thus enhancing the visual realism of the deformation. Emphasis is placed on the topological structure of the model mesh, material property definition, and real-time processing optimization to ensure that the model does not tear or distort after deformation and maintains texture consistency.

[0318] In this embodiment, when an intersection between the operating object and the virtual sculpture model is detected, such as when the operating object sculpts the virtual sculpture model, the model undergoes adaptive deformation and mesh subdivision based on the virtual hand's posture and speed upon contact by integrating collision detection, force analysis, deformation calculation, dynamic retopology, and real-time rendering. This solution introduces speed-modulated force calculation, density-adaptive deformation, and real-time adaptive adjustment of the virtual sculpture model's mesh complexity based on the force and speed of the interaction between the operating object and the virtual sculpture model. This achieves a highly realistic physical deformation effect, i.e., visual realism, while ensuring real-time performance. Furthermore, through the topological structure, material property definition, and real-time processing optimization of the virtual sculpture model's mesh, this solution ensures that the model does not tear or distort after deformation and maintains texture consistency. This enables dynamic and localized adjustment of the virtual sculpture model's precision based on the user's real-time operational intentions during the sculpting process using VR sculpting software.

[0319] Based on the above embodiments Figure 1 The present application discloses a real-time deformation method and a corresponding real-time deformation system, such as... Figure 2 As shown, the real-time deformation system includes:

[0320] The first acquisition unit 201 is used to acquire the force intention data packet of the operation object through collision detection when it is detected that the operation object intersects with the virtual sculpture model.

[0321] The simulation unit 202 is used to simulate the mechanical response data of the virtual sculpture model in the real world based on the force intention data packet;

[0322] The second acquisition unit 203 is used to perform deformation calculation on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model;

[0323] The generation unit 204 is used to generate an optimization request signal if the visual deformation feedback signal does not meet the preset smoothing conditions.

[0324] The adaptive adjustment unit 205 is used to trigger the adaptive adjustment of the mesh complexity of the virtual sculpture model according to the optimization request signal, so as to complete the dynamic mesh subdivision operation.

[0325] Furthermore, the first acquisition unit 201 includes:

[0326] The first acquisition module is used to acquire the input signal where the operation object and the virtual sculpture model intersect, the virtual tracking data, and the collision detection data of the physical engine of the operation object and the virtual sculpture model when it is detected that the operation object is sculpting the virtual sculpture model.

[0327] The second acquisition module is used to obtain the force intention data packet of the manipulated object based on the virtual tracking data, input signal and collision detection data.

[0328] Furthermore, the collision detection data includes at least the pinch force value, contact data, and the instantaneous velocity vector of the controller. The second acquisition module includes:

[0329] The first acquisition submodule is used to acquire the pressure changes and contact point data structure when the operation object and the virtual sculpture model intersect;

[0330] The determination submodule is used to determine the kneading force value based on pressure changes;

[0331] The extraction submodule is used to extract contact data from the contact point data structure where the operation object intersects with the virtual sculpture model; the contact data includes at least the contact point coordinates and the contact surface normal.

[0332] The second acquisition submodule is used to acquire the position information of the controller in the previous frame and the position information in the current frame, and to calculate the displacement based on the position information of the previous frame and the position information in the current frame.

[0333] The calculation submodule is used to calculate the instantaneous velocity vector of the controller using displacement and frame time.

[0334] Furthermore, the simulation unit 202 includes:

[0335] The first calculation module is used to calculate the magnitude of the velocity vector based on the instantaneous velocity vector;

[0336] The third acquisition module is used to acquire quality, exponential decay function, posture sensitivity coefficient and posture factor;

[0337] The second calculation module is used to calculate the instantaneous impact force based on the mass, the exponential decay function, and the magnitude of the velocity vector.

[0338] The third calculation module is used to calculate the postural force generated by the kneading action based on the postural sensitivity coefficient and postural factor.

[0339] The fourth calculation module is used to calculate the resultant force scalar based on the instantaneous impact force and the posture force;

[0340] The fifth calculation module is used to calculate the target force vector that drives the deformation of the virtual object based on the resultant force scalar and contact data, and to use the target force vector as the mechanical response data of the virtual sculpture model in the real world.

[0341] Furthermore, the second acquisition unit 203 includes:

[0342] A module is created to generate a spherical influence area based on contact data and a preset influence radius.

[0343] The filtering module is used to filter out all vertices from the spherical influence region and form a set of vertices to be deformed based on all vertices in the spherical influence region;

[0344] The sixth calculation module is used to calculate the distance from each vertex to the contact data for each vertex in the set of vertices to be deformed;

[0345] The seventh calculation module is used to calculate the attenuation weight based on the distance;

[0346] The eighth calculation module is used to calculate the displacement vector of each vertex based on the target force vector, stiffness coefficient, attenuation weight, and contact data.

[0347] The update module is used to update the vertex positions based on the displacement vector to obtain the updated vertex positions;

[0348] The ninth calculation module is used to calculate the displacement magnitude of each vertex after its updated position based on the displacement vector of each vertex.

[0349] The tenth calculation module is used to calculate the average and maximum displacement magnitude of the vertex after each updated position;

[0350] The first generation module is used to generate visual deformation feedback signals for the virtual sculpture model based on the average value, the maximum value, and the weighting function.

[0351] Furthermore, the generating unit 204 includes:

[0352] The eleventh calculation module is used to calculate the decision score based on the visual deformation feedback signal, the target force vector, the velocity vector magnitude, and the preset weight coefficients.

[0353] The comparison module is used to compare the decision score with the dynamic threshold in the spherical influence area;

[0354] The determination module is used to determine whether the visual deformation feedback signal does not meet the preset smoothing conditions if the decision score is greater than the dynamic threshold.

[0355] The second generation module is used to generate an optimization request signal when the visual deformation feedback signal does not meet the preset smoothing conditions.

[0356] In this embodiment, when an intersection between the operating object and the virtual sculpture model is detected, such as when the operating object sculpts the virtual sculpture model, the model undergoes adaptive deformation and mesh subdivision based on the virtual hand's posture and speed upon contact by integrating collision detection, force analysis, deformation calculation, dynamic retopology, and real-time rendering. This solution introduces speed-modulated force calculation, density-adaptive deformation, and real-time adaptive adjustment of the virtual sculpture model's mesh complexity based on the force and speed of the interaction between the operating object and the virtual sculpture model. This achieves a highly realistic physical deformation effect, i.e., visual realism, while ensuring real-time performance. Furthermore, through the topological structure, material property definition, and real-time processing optimization of the virtual sculpture model's mesh, this solution ensures that the model does not tear or distort after deformation and maintains texture consistency. This enables dynamic and localized adjustment of the virtual sculpture model's precision based on the user's real-time operational intentions during the sculpting process using VR sculpting software.

[0357] This application also provides a storage medium that includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to perform the real-time deformation method described above.

[0358] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 3As shown, it specifically includes a memory 301 and one or more instructions 302, wherein one or more instructions 302 are stored in the memory 301 and configured to be executed by one or more processors 303 to perform the above-mentioned real-time deformation method.

[0359] The steps in the methods of the various embodiments of this application can be adjusted, combined, and deleted according to actual needs. It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0360] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0361] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A real-time deformation method, characterized in that, The method includes: When an intersection between the manipulated object and the virtual sculpture model is detected, the force intent data packet of the manipulated object is obtained through collision detection. Based on the force intention data packet, simulate the mechanical response data of the virtual sculpture model in the real world; Deformation calculations are performed on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model; If the visual deformation feedback signal does not meet the preset smoothing conditions, an optimization request signal is generated. The optimization request signal triggers an adaptive adjustment of the mesh complexity of the virtual sculpture model.

2. The method according to claim 1, characterized in that, When an intersection between the manipulated object and the virtual sculpture model is detected, the force intention data packet of the manipulated object is obtained through collision detection, including: When it is detected that the object being manipulated is sculpting the virtual sculpture model, the input signal where the object and the virtual sculpture model intersect, the virtual tracking data, and the collision detection data of the physical engine between the object and the virtual sculpture model are acquired. Based on the virtual tracking data, the input signal, and the collision detection data, a force intention data packet of the manipulated object is obtained.

3. The method according to claim 2, characterized in that, The collision detection data includes at least the pinching force value, contact data, and instantaneous velocity vector. The step of obtaining the force intent data packet of the manipulated object based on the virtual tracking data, the input signal, and the collision detection data includes: Acquire the pressure changes and contact point data structure when the manipulated object intersects with the virtual sculpture model; The kneading force value is determined based on the pressure change. Extract the contact data where the operation object intersects with the virtual sculpture model from the contact point data structure; Obtain the position information of the controller in the previous frame and the position information in the current frame, and calculate the displacement based on the position information of the previous frame and the position information in the current frame; The instantaneous velocity vector of the controller is calculated using the displacement and frame time.

4. The method according to claim 3, characterized in that, The step of simulating the mechanical response data of the virtual sculpture model in the real world based on the force intention data packet includes: Calculate the magnitude of the velocity vector based on the instantaneous velocity vector; Obtain the quality, exponential decay function, pose sensitivity coefficient, and pose factor; The instantaneous impact force is calculated based on the mass, the exponential decay function, and the magnitude of the velocity vector. The postural force generated by the kneading action is calculated based on the postural sensitivity coefficient and the postural factor. The resultant force scalar is calculated based on the instantaneous impact force and the posture force. The target force vector driving the deformation of the virtual object is calculated based on the resultant force scalar and the contact data, and the target force vector is used as the mechanical response data of the virtual sculpture model in the real world.

5. The method according to claim 3, characterized in that, The process of performing deformation calculations on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model includes: A spherical influence area is created based on the contact data and the preset influence radius; All vertices are selected from the spherical influence region, and a set of vertices to be deformed is formed based on all vertices in the spherical influence region; For each vertex in the set of vertices to be deformed, calculate the distance from each vertex to the contact data; Calculate the attenuation weight based on the distance; The displacement vector of each vertex is calculated based on the target force vector, stiffness coefficient, attenuation weight, and contact data. The vertex positions are updated based on the displacement vector to obtain the updated vertex positions. Calculate the displacement magnitude of each vertex after updating its position based on the displacement vector of each vertex; Calculate the average and maximum values ​​of the displacement amplitude; Based on the average value, the maximum value, and the weighting function, a visual deformation feedback signal for the virtual sculpture model is generated.

6. The method according to claim 5, characterized in that, If the visual deformation feedback signal does not meet the preset smoothing conditions, generating an optimization request signal includes: The decision score is calculated based on the visual deformation feedback signal, the target force vector, the velocity vector magnitude, and the preset weighting coefficients. Within the spherical influence region, the decision score is compared with the dynamic threshold; If the decision score is greater than the dynamic threshold, it is determined that the visual deformation feedback signal does not meet the preset smoothing condition; If the visual deformation feedback signal does not meet the preset smoothing conditions, an optimization request signal is generated.

7. A real-time deformation system, characterized in that, The system includes: The first acquisition unit is used to acquire the force intention data packet of the operation object through collision detection when it is detected that the operation object intersects with the virtual sculpture model. The simulation unit is used to simulate the mechanical response data of the virtual sculpture model in the real world based on the force intention data packet. The second acquisition unit is used to perform deformation calculation on the mechanical response data to obtain the visual deformation feedback signal of the virtual sculpture model; The generation unit is used to generate an optimization request signal if the visual deformation feedback signal does not meet the preset smoothing conditions. An adaptive adjustment unit is used to trigger adaptive adjustment of the mesh complexity of the virtual sculpture model based on the optimization request signal.

8. The system according to claim 7, characterized in that, The first acquisition unit includes: The first acquisition module is used to acquire the input signal where the operation object and the virtual sculpture model intersect, the virtual tracking data, and the collision detection data of the physical engine of the operation object and the virtual sculpture model when it is detected that the operation object is sculpting the virtual sculpture model. The second acquisition module is used to obtain the force intention data packet of the operation object based on the virtual tracking data, the input signal and the collision detection data.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides is controlled to perform the real-time deformation method as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 6.

Images