3D animation virtual interactive display method

Abstract

The invention relates to the technical field of 3D imaging, and discloses a 3D animation virtual interactive display method, which comprises the following steps of: S1, describing the pose of a virtual object in a three-dimensional space by utilizing a Lie group theory, and introducing Lie algebra to represent tiny disturbance to finish registration between the virtual object and a real scene; s2, estimating the state of the virtual object described in the step S1 by adopting an extended Kalman-Buch filter; and S3, on the basis of the pose of the virtual object determined in the step S1, discretizing a contact area by using a nonlinear finite element method, and calculating node displacement. The registration method based on the combination of the Lie group theory and the Lie algebra is adopted, accurate docking of the virtual object and the real scene is achieved, the high-precision registration effect of the virtual object and the real environment is achieved, compared with a traditional method depending on image processing and sensors, the technology can solve the problem of error accumulation in the registration process, and the registration accuracy is improved. And the stability and the accuracy of the virtual object in a complex environment are ensured.

Description

Technical Field

[0001] The present invention relates to the field of 3D imaging technology, and specifically to a 3D animation virtual interaction display method. Background Art

[0002] Existing 3D animation virtual interaction display methods mainly rely on traditional image processing and sensor data to achieve the registration of virtual objects and real scenes, and at the same time use simple haptic feedback control algorithms to simulate the interaction effect. In addition, the emotional feedback of virtual characters often adopts a fixed mode and cannot achieve real-time adjustment driven by user behavior. Generally speaking, the existing technology cannot simultaneously take into account high-precision registration, real haptic feedback, and emotional interaction. Thus, the solution of the present invention is introduced. The main body includes registration based on Lie groups and Lie algebras, haptic feedback regulation combining nonlinear finite elements and HJB optimal control, and a hybrid power system to achieve virtual character emotion regulation, aiming to solve the above deficiencies.

[0003] Existing technologies mostly use image processing and sensor data for registration, which is prone to error accumulation, has low registration accuracy, poor system stability, and virtual objects are prone to deviate from the target position in complex environments, directly affecting the authenticity of the interaction between virtual and reality.

[0004] Currently, most haptic feedback controls use simple algorithms, and the feedback adjustment strength cannot be self-adaptive. The feedback response is fixed and cannot match the user's dynamic operations in real time. The feedback effect is less natural and the interaction immersion is insufficient. Simple control methods cannot capture the non-linear characteristics of haptic mechanics.

[0005] The emotional feedback of virtual characters depends on preset modes and lacks real-time response to user behavior. The emotional expression is single and cannot be adjusted according to haptic feedback and user interaction. The result appears rigid and lacks emotional resonance. The existing solutions have obvious shortcomings in the multi-sensory interaction experience.

[0006] The above deficiencies are exactly the pain points that the present invention solves through innovative technical means. The present invention introduces advanced Lie group and Lie algebra theories to achieve precise registration, uses HJB optimal control to achieve dynamic haptic adjustment, and adopts a hybrid power system to adjust the emotional feedback of virtual characters, making up for the deficiencies of the existing technology in terms of accuracy, response, and interaction naturalness.

[0007] Therefore, those skilled in the art provide a 3D animation virtual interaction display method to solve the above-mentioned problems. Summary of the Invention

[0008] Aiming at the deficiencies of the existing technology, the present invention provides a 3D animation virtual interaction display method to solve the problems raised in the above background art.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: A 3D animation virtual interaction display method, comprising:

[0010] Step S1, using Lie group theory to describe the pose of a virtual object in three-dimensional space and introducing Lie algebra to represent small perturbations to complete the registration between the virtual object and the real scene;

[0011] Step S2, using an extended Kalman-Bucy filter to estimate the state of the virtual object described in Step S1;

[0012] Step S3, based on the pose of the virtual object determined in Step S1, using the nonlinear finite element method to discretize the contact area and calculate the node displacements, and then obtaining the stress field through an integration method to construct a tactile and force feedback simulation model;

[0013] Step S4, based on the tactile and force feedback simulation model constructed in Step S3, using the HJB optimal control theory to design a feedback control law to adjust the tactile feedback;

[0014] Step S5, combining the technical contents of Steps S1 to S4 to construct a hybrid power system to describe the emotional state of a virtual character, and the hybrid power system uses an adaptive control law to adjust the emotional feedback;

[0015] Step S6, integrating the results of Steps S1 to S5 to develop a virtual interaction system, and realizing real-time interaction between the virtual object, tactile feedback, virtual character emotional state and user behavior through sensor data acquisition.

[0016] Preferably, Step S1 further includes:

[0017] Step 1.1: Establish a pose representation of the virtual object based on the Lie group SE(3), that is:

[0018]

[0019] where, R1 is a rotation matrix, t1 is a translation vector, and T1 is an initial transformation matrix;

[0020] Step 1.2: Introduce Lie algebra se(3) for small perturbation modeling, and define the small perturbation of the pose as an element in Lie algebra se(3):

[0021] where, Δα1 is the rotation perturbation amount, Δβ1 is the translation perturbation amount, is the pose perturbation amount, and then use the exponential map to transform the Lie algebra to the Lie group to obtain the perturbed transformation:

[0022]

[0023] where, is an anti-symmetric matrix, and T′1 is the transformed matrix after perturbation;

[0024] Step 1.3: Combine the extended Kalman filter for state estimation of the virtual object. When performing registration, the motion state of the virtual object is described by the differential equation:

[0025]

[0026] where, is the state vector of the virtual object, represents the state transition equation of the system, is the noise influence matrix, is the standard Wiener process.

[0027] Preferably, in step S2, the extended Kalman-Bucy filter is used to estimate the state of the virtual object in step S1, aiming to optimize the position and attitude estimation of the virtual object through sensor data and the pose estimation results in the previous step.

[0028] Preferably, step S2 further includes:

[0029] Step 2.1: Describe the state evolution of the virtual object through a differential equation. The state of the virtual object consists of position and attitude, and the state vector is defined as:

[0030] where, is the state vector, R1 is the rotation matrix, t1 is the translation vector, and the evolution of the state is represented by the differential equation:

[0031] where, is the state transition matrix, is the noise propagation matrix, is the standard Wiener process;

[0032] Step 2.2: Predict the state of the virtual object through the state transition equation. Assume that at time τ1, the state of the virtual object is Then the prediction step is carried out through the following formula:

[0033]

[0034] where, is the white noise increment;

[0035] Step 2.3: After the prediction step, use the sensor measurement data for update. Assume that the measurement value obtained through the sensor is Z1, and the update step is carried out through the following Kalman gain formula:

[0036]

[0037] Among them, is the prediction error covariance matrix, H1 is the measurement matrix, and R1 is the measurement noise covariance matrix;

[0038] By calculating the Kalman gain K1, the state estimation of the virtual object is updated:

[0039]

[0040] The updated state is the best estimation result of the virtual object at the current moment;

[0041] Step 2.4: After updating the state of the virtual object, update the error covariance matrix The error covariance matrix reflects the uncertainty of the current state estimation:

[0042] Among them, I is the identity matrix, and the updated error covariance matrix provides the basis for the estimation accuracy in subsequent filtering steps;

[0043] Step 2.5: Combining the pose of the virtual object calculated in step S1, through the above Kalman filtering update process, the state of the virtual object will be optimized and corrected at each moment to ensure the dynamic matching of the virtual object with the real scene.

[0044] Preferably, step S3 further includes:

[0045] Step 3.1: Determine the contact area between the virtual object and the real scene. The contact area is discretized into elements in the finite element model by meshing, define the mesh node set and the finite element unit set, and discretize the contact area using the nonlinear finite element method;

[0046] The distribution of physical quantities between each node and adjacent nodes is described by the shape function to obtain the stiffness matrix K j and the load vector F j , and the matrix and vector are calculated by the following formulas:

[0047]

[0048] Among them, B is the strain-displacement matrix, C is the constitutive matrix of the material, N is the shape function matrix, f is the external force vector, and dA is the integration region;

[0049] Step 3.2: Calculate the node displacements by solving the equilibrium equation of the discretized finite element system. The equilibrium equation is: KU = F,

[0050] Among them, U is the displacement of all nodes, F is the external force, and K is the stiffness matrix. After solving for the node displacements, the stress field is calculated using the displacement data. The stress field σ(x) is obtained through the following formula: σ(x) = CBU,

[0051] where B is the strain-displacement matrix and C is the constitutive matrix of the material;

[0052] Step 3.3: By means of an integration method, the calculated stress field is combined with the displacement information to establish a tactile and force feedback simulation model. The force feedback model calculates the total force F exerted on the virtual object when it contacts the real object through integration total : F total = ∫ Γ σ(x)dA,

[0053] where Γ is the contact surface, σ(x) is the stress field, and dA is the integration region.

[0054] Preferably, step S4 further includes:

[0055] Step 4.1: In step S3, the displacements and stress fields in the contact area between the virtual object and the user are calculated by means of the nonlinear finite element method. To establish the tactile feedback model, it is necessary to define the state variables of the system; and according to the calculation results of step S3, the inputs of the tactile feedback are mainly related to the displacements, stresses, and torques at the contact points in the contact area;

[0056] Step 4.2: To design the optimal tactile feedback control law, the HJB optimal control theory is adopted. The core of the HJB optimal control theory is to obtain the optimal control input by solving the Hamilton-Jacobi equation. The cost function J of the system is defined as:

[0057]

[0058] where L(x(t), u(t)) is the immediate cost function, λ(x(t)) is the state constraint function, x(t) is the system state vector, u(t) is the system control input, t is time, and J is the cost function;

[0059] According to the HJB optimal control theory, the Hamiltonian of the system is expressed as:

[0060] H(x, u, p) = L(x, u) + p T f(x, u),

[0061] where L(x, u) is the immediate cost function, f(x, u) is the system dynamics function, H(x, u, p) is composed of the instantaneous loss and the dot product of the adjoint variable and the dynamics function, and p T is the transpose matrix of the adjoint variable p.

[0062] Preferably, step S4 further includes:

[0063] Step 4.3: To obtain the optimal control law, it is necessary to solve the HJB equation, and the HJB equation is:

[0064]

[0065] where V(x) is the value function, is the gradient of the value function with respect to the state, is the minimum operation over all feasible control input spaces, L(x, u) is the immediate cost function, and f(x, u) is the system dynamics function;

[0066] The calculation of the control input is as follows:

[0067] where, returns the set of control inputs u that minimizes the objective function.

[0068] Step 4.4: Once the optimal control input u * (x) is obtained through the HJB equation, it can be applied to the virtual interaction system;

[0069] Step 4.5: During the operation of the system, it will be adjusted according to the user's real-time feedback and environmental changes, and by collecting the user's interaction data, the design of the cost function L(x, u) will be further optimized.

[0070] Preferably, step S5 further includes:

[0071] Step 5.1: Based on steps S1 to S4, the emotional state of the virtual character is modeled by a hybrid dynamic system;

[0072] The emotional state θ(t) is represented as a vector, including the emotional variables of the virtual character, and the variables are described by the following state equation:

[0073] where θ(t) is the emotional state vector of the virtual character, u(t) is the external control input, y(t) is the user behavior feedback for interacting with the virtual object, and f θ is the function describing the change of the emotional state of the virtual character;

[0074] Step 5.2: To adjust the emotional feedback of the virtual character, a hybrid dynamic system model is used to describe the change of the emotional state. The hybrid dynamic system includes discrete events and continuous dynamics, which are used to describe the discrete change and continuous evolution of emotions;

[0075] The hybrid dynamic system model is represented in the following form:

[0076]

[0077] Among them, \(x(t)\) is the state variable of the system, \(u(t)\) is the external control input, \(A\) is the system matrix, \(B\) is the input matrix, and \(X\) is the state space.

[0078] Preferably, the step S5 further includes:

[0079] Step 5.3: The adaptive control law is used to adjust the emotional feedback of the virtual character, which can be adjusted according to the user's behavior or the interaction state of the virtual object. The core idea of the adaptive control is to dynamically adjust the control law according to the system state feedback;

[0080] The design of the control law is based on the dynamic equation of the virtual character's emotion. Let the emotional feedback control input be \(v(t)\), which is the adaptive control input for adjusting the emotional state. The goal is to minimize the cost function, and the cost function \(Y\) is defined as:

[0081]

[0082] Among them, \(Q\) is used to weigh the importance of the virtual character's emotional state, \(R\) is used to weigh the cost of the control input, \(\theta(t)\) is the emotional state vector of the virtual character, and \(u(t)\) is the external control input;

[0083] The adaptive adjustment rule of the control input \(v(t)\) is as follows:

[0084]

[0085] Among them, \(K(t)\) is the adaptive gain matrix, is the estimated gain matrix;

[0086] The adaptive algorithm gradually optimizes the control input by adjusting \(K(t)\) to achieve the expected change in the emotional state of the virtual character.

[0087] Preferably, the step S5 further includes:

[0088] Step 5.4: To ensure the stability and response speed of the emotional feedback, the Lyapunov stability theory is used to design the adaptive controller. Let \(v(\theta(t))\) be the Lyapunov function, which is a function of the emotional state \(\theta(t)\) and represents the "energy" of the system, and satisfies:

[0089] Through the appropriate Lyapunov function \(v(\theta(t))\), the stability of the emotional feedback system is ensured. The design goal of the adaptive controller is to ensure that the system can quickly return to the desired emotional state under disturbances or changes;

[0090] Step 5.5: Once the control law design is completed, the emotional feedback of the virtual character can be applied to the virtual interaction system;

[0091] Step 5.6: By collecting the interaction data of the user, evaluate the effect of the emotional feedback, and further optimize the parameters in the cost function Y and the adaptive control gain K(t) to improve the quality of the emotional feedback of the virtual character and the interaction experience.

[0092] The present invention provides a 3D animation virtual interaction display method. It has the following beneficial effects:

[0093] 1. The present invention adopts a registration method combining Lie group theory and Lie algebra to achieve the precise docking of virtual objects and real scenes, reaching a high-precision registration effect of virtual objects and real environments. Compared with traditional methods relying on image processing and sensors, this technology can solve the problem of error accumulation in the registration process and ensure the stability and accuracy of virtual objects in complex environments.

[0094] 2. The present invention uses the HJB optimal control theory to adjust the tactile feedback, significantly improving the feedback experience of users in virtual interaction. Compared with the simple tactile feedback control methods in the prior art, this solution can adjust the tactile feedback intensity between virtual objects and users in real time, making the mechanical response of virtual objects conform to real physical perception and enhancing the immersion of interaction.

[0095] 3. The present invention adjusts the emotional feedback of virtual characters through a hybrid power system to achieve the seamless integration of emotion and tactile feedback. Compared with traditional single feedback modes, this technology can dynamically adjust the emotional responses of virtual characters according to the behavior and emotional states of users, solve the limitation that traditional methods cannot accurately simulate emotional interaction, and enhance the realism and emotional resonance of virtual characters in user interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0097] To enable those skilled in the art to understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0098] The present invention will be described in detail below with reference to the accompanying drawings:

[0099] Embodiment:

[0100] Please refer to the attachedFigure 1 , embodiments of the present invention provide a 3D animation virtual interaction display method, including:

[0101] Step S1, using Lie group theory to describe the pose of a virtual object in three-dimensional space and introducing Lie algebra to represent small perturbations to complete the registration between the virtual object and the real scene;

[0102] Step 1.1: Establish a pose representation of the virtual object based on the Lie group SE(3), that is:

[0103]

[0104] where, R1 is a rotation matrix, t1 is a translation vector, and T1 is an initial transformation matrix;

[0105] Step 1.2: Introduce Lie algebra se(3) for small perturbation modeling, and define the small perturbation of the pose as an element in Lie algebra se(3):

[0106] where, Δα1 is the rotation perturbation amount, Δβ1 is the translation perturbation amount, is the pose perturbation amount, and then use the exponential map to convert the Lie algebra to the Lie group to obtain the perturbed transformation:

[0107]

[0108] where, is an anti-symmetric matrix, and T′1 is the perturbed transformation matrix;

[0109] Step 1.3: Combine the extended Kalman filter for state estimation of the virtual object. When performing registration, the motion state of the virtual object is described by a differential equation:

[0110]

[0111] where, is the state vector of the virtual object, represents the state transition equation of the system, is the noise influence matrix, is the standard Wiener process.

[0112] The advantage of Step 1.1 is that using Lie group representation can accurately capture the combined characteristics of rotation and translation, with concise mathematical expressions and easy subsequent operations, providing a solid foundation for the precise positioning of virtual objects in three-dimensional space;

[0113] The advantage of Step 1.2 is that the exponential map method can directly map small perturbations to actual transformations, effectively alleviating the problem of non-linear errors caused by the accumulation of small perturbations, and improving the registration accuracy and system robustness;

[0114] The benefit of Step 1.3 is that the extended Kalman filter can estimate the state of the virtual object in real time under noise interference, significantly improving the system tracking accuracy and response speed, and ensuring the smoothness and continuity of the virtual object movement during the registration process.

[0115] Step S2: Estimate the state of the virtual object described in Step S1 using the extended Kalman-Bucy filter;

[0116] In Step S2, the state of the virtual object in Step S1 is estimated by the extended Kalman-Bucy filter, aiming to optimize the position and attitude estimation of the virtual object through the sensor data and the pose estimation results in the previous step.

[0117] Step 2.1: Describe the state evolution of the virtual object through differential equations. The state of the virtual object consists of position and attitude, and the state vector is defined as:

[0118] where, is the state vector, R1 is the rotation matrix, t1 is the translation vector, and the evolution of the state is represented by the differential equation:

[0119] where, is the state transition matrix, is the noise propagation matrix, is the standard Wiener process;

[0120] Step 2.2: Predict the state of the virtual object through the state transition equation. Assume that at time τ1, the state of the virtual object is Then the prediction step is carried out through the following formula:

[0121]

[0122] where, is the white noise increment;

[0123] Step 2.3: After the prediction step, update using the sensor measurement data. Assume that the measurement value obtained through the sensor is Z1, and the update step is carried out through the following Kalman gain formula:

[0124]

[0125] where, is the prediction error covariance matrix, H1 is the measurement matrix, and R1 is the measurement noise covariance matrix;

[0126] By calculating the Kalman gain K1, update the state estimate of the virtual object:

[0127]

[0128] Updated state is the best estimated result of the virtual object at the current moment;

[0129] Step 2.4: After updating the state of the virtual object, update the error covariance matrix The error covariance matrix reflects the uncertainty of the current state estimate:

[0130] where I is the identity matrix, and the updated error covariance matrix provides the basis for the estimation accuracy in subsequent filtering steps;

[0131] Step 2.5: Combining the pose of the virtual object calculated in Step S1, through the above Kalman filter update process, the state of the virtual object will be optimized and corrected at each moment, ensuring the dynamic matching between the virtual object and the real scene.

[0132] The advantage of Step 2.1 is that using continuous differential equations to capture object motion can reflect state changes in detail, and it is closer to the actual dynamics than single sampling, with high accuracy;

[0133] The advantage of Step 2.2 is that it anticipates the next state in advance, reduces sudden errors, is simple and intuitive, and provides a reference for subsequent correction;

[0134] The advantage of Step 2.3 is that it fuses sensor data with the prediction result, corrects the deviation in real time, and is more accurate than simple prediction, making the state estimate closer to the real situation;

[0135] The advantage of Step 2.4 is that it dynamically reflects the uncertainty of the current state, adjusts the estimation accuracy, and provides a stable basis for subsequent filtering;

[0136] The advantage of Step 2.5 is that it realizes continuous correction, the state of the virtual object is always closely corresponding to the actual environment, and it solves the problems of state drift and inaccurate matching in traditional methods;

[0137] Overall summary, the extended Kalman-Bucy filter in the present invention realizes an effective combination of prediction and real-time update. The overall filtering process is smooth and continuous, can effectively resist noise interference, and makes the state estimation of the virtual object accurate and stable. Compared with traditional methods, this technology greatly improves the dynamic matching between the virtual object and the real scene, providing a solid foundation for subsequent 3D animation display and interaction.

[0138] Step S3, based on the pose of the virtual object determined in Step S1, use the nonlinear finite element method to discretize the contact area and calculate the node displacements, and then obtain the stress field through the integral method to construct a tactile and force feedback simulation model;

[0139] Step 3.1: Determine the contact area between the virtual object and the real scene. The contact area is discretized into elements in the finite element model by mesh division. Define the set of mesh nodes and the set of finite elements, and discretize the contact area using the nonlinear finite element method;

[0140] The distribution of physical quantities between each node and its adjacent nodes is described by the shape function, and the stiffness matrix K of each element is obtained j and the load vector F j , and the matrix and vector are calculated by the following formulas:

[0141]

[0142] where B is the strain-displacement matrix, C is the constitutive matrix of the material, N is the shape function matrix, f is the external force vector, and dA is the integration region;

[0143] Step 3.2: Calculate the node displacements by solving the equilibrium equation of the discretized finite element system. The equilibrium equation is: KU = F,

[0144] where U is the displacement of all nodes, F is the external force, and K is the stiffness matrix. After solving for the node displacements, calculate the stress field using the displacement data. The stress field σ(x) is obtained by the following formula: σ(x) = CBU,

[0145] where B is the strain-displacement matrix and C is the constitutive matrix of the material;

[0146] Step 3.3: Combine the calculated stress field and displacement information through an integration method to establish a tactile and force feedback simulation model. The force feedback model calculates the total force F total that the virtual object receives when contacting the real object through integration: F total = ∫ Γ σ(x)dA,

[0147] where Γ is the contact surface, σ(x) is the stress field, and dA is the integration region.

[0148] The advantage of Step 3.1 is that after using mesh discretization, it can finely capture local contact details, and is more targeted than traditional rough models, with accurate local response and facilitating subsequent calculations;

[0149] The advantage of Step 3.2 is that directly solving for the node displacements ensures fine and realistic simulation of force deformation, and the calculation of the stress field is closely connected, which can accurately reflect the internal physical changes in the contact area and improve the authenticity of the model response;

[0150] The benefit points calculation in step 3.3 ensures the global consistency of the feedback data, and the tactile and force feedback simulation model can reflect the overall force state when the virtual object contacts the real object in real time, overcoming the problems of single response and incomplete data in the traditional feedback model;

[0151] Generally speaking, by discretizing the contact area through nonlinear finite element method, solving the node displacements and stress fields, and then constructing a feedback model using the integral method, this technical solution can accurately simulate the mechanical response of virtual and real contacts. Compared with the traditional method, the model is detailed and the response is accurate, effectively improving the realism and dynamic interaction effect of tactile feedback and force feedback.

[0152] Step S4: Based on the tactile and force feedback simulation model constructed in step S3, the HJB optimal control theory is used to design a feedback control law to adjust the tactile feedback;

[0153] Step 4.1: In step S3, the displacements and stress fields of the contact area between the virtual object and the user are calculated by the nonlinear finite element method. To establish a tactile feedback model, the state variables of the system need to be defined; and according to the calculation results of step S3, the input of the tactile feedback is mainly related to the displacement, stress of the contact area and the moment of the contact point;

[0154] Step 4.2: To design the optimal tactile feedback control law, the HJB optimal control theory is adopted. The core of the HJB optimal control theory is to obtain the optimal control input by solving the Hamilton-Jacobi equation. The cost function J of the system is defined as:

[0155]

[0156] where, L(x(t), u(t)) is the instantaneous cost function, λ(x(t)) is the state constraint function, x(t) is the system state vector, u(t) is the system control input, t is the time, and J is the cost function;

[0157] According to the HJB optimal control theory, the Hamiltonian of the system is expressed as:

[0158] H(x, u, p) = L(x, u) + p T f(x, u),

[0159] where, L(x, u) is the instantaneous cost function, f(x, u) is the system dynamics function, H(x, u, p) is composed of the instantaneous loss and the dot product of the adjoint variable and the dynamics function, and p T is the transpose matrix of the adjoint variable p;

[0160] Step 4.3: To obtain the optimal control law, the HJB equation needs to be solved. The HJB equation is:

[0161]

[0162] Among them, V(x) is the value function, is the gradient of the value function with respect to the state, performs a minimization operation over all feasible control input spaces, L(x, u) is the immediate cost function, and f(x, u) is the system dynamics function;

[0163] The calculation of the control input is as follows:

[0164] Among them, returns the set of control inputs u that minimizes the objective function.

[0165] Step 4.4: Once the optimal control input u * (x) is obtained through the HJB equation, it can be applied to the virtual interaction system;

[0166] Step 4.5: During the operation of the system, it will be adjusted according to the user's real-time feedback and environmental changes, and by collecting the user's interaction data, the design of the cost function L(x, u) will be further optimized.

[0167] The benefit of Step 4.1 is that by accurately describing the system state variables, it ensures that the haptic feedback input is closely related to the actual situation of the physical contact area, can truly reflect the interaction between the object and the user, and enhance the accuracy of the feedback;

[0168] The benefit of Step 4.2 is that the HJB optimal control theory can accurately adjust the haptic feedback intensity when the virtual object interacts with the user. Different from traditional fixed feedback control methods, the HJB theory dynamically adjusts the feedback through an optimization process to ensure that the haptic feedback is both natural and in line with the actual mechanical response, enhancing the immersion of the interaction;

[0169] The benefit of Step 4.3 is that the solution of the HJB equation, by considering the relationship between the system dynamics, state, and control input, makes the feedback control precise and in line with physical laws, and can maximize the naturalness and realism of the response in the virtual object interaction;

[0170] The benefit of Step 4.4 is that the real-time application of the optimal control input makes the interaction process between the virtual object and the user smooth and natural. The haptic feedback is not fixed, but continuously optimized according to the system state and the user's real-time feedback, enhancing the dynamic sense of the interaction;

[0171] The benefit of Step 4.5 is that by continuously adjusting the cost function and the optimization process, the system can adapt to the user's behavior and emotional changes, provide personalized and responsive haptic feedback, and further enhance the realism and immersion of the virtual interaction;

[0172] Overall summary: By designing a tactile feedback control law through the HJB optimal control theory and adjusting the tactile intensity in combination with real-time user feedback and environmental data, more accurate and natural tactile responses can be achieved in virtual interactions, overcoming the limitations of traditional fixed feedback control, making the interaction between virtual objects and users more realistic, and enhancing the overall user experience, especially in complex interaction scenarios.

[0173] Step S5: Construct a hybrid power system to describe the emotional state of the virtual character by combining the technical content of steps S1 to S4. The hybrid power system uses an adaptive control law to adjust the emotional feedback.

[0174] Step 5.1: Based on steps S1 to S4, the emotional state of the virtual character is modeled by a hybrid power system.

[0175] The emotional state θ(t) is represented as a vector, which includes the emotional variables of the virtual character. The variables are described by the following state equations:

[0176] where θ(t) is the emotional state vector of the virtual character, u(t) is the external control input, y(t) is the user behavior feedback for interacting with the virtual object, and f θ is a function describing the change in the emotional state of the virtual character.

[0177] Step 5.2: To adjust the emotional feedback of the virtual character, a hybrid power system model is used to describe the change in the emotional state. The hybrid power system includes discrete events and continuous dynamics, which are used to describe the discrete changes and continuous evolution of emotions.

[0178] The hybrid power system model is expressed in the following form:

[0179]

[0180] where x(t) is the state variable of the system, u(t) is the external control input, A is the system matrix, B is the input matrix, and X is the state space.

[0181] Step 5.3: The adaptive control law is used to adjust the emotional feedback of the virtual character, which can be adjusted according to the user's behavior or the interaction state of the virtual object. The core idea of adaptive control is to dynamically adjust the control law based on the system state feedback.

[0182] The design of the control law is based on the dynamic equation of the virtual character's emotion. Let the emotional feedback control input be v(t), which is the adaptive control input for adjusting the emotional state. The goal is to minimize the cost function, and the cost function Y is defined as:

[0183]

[0184] Among them, Q is used to weigh the importance of the emotional state of the virtual character, R is used to weigh the cost of the control input, θ(t) is the emotional state vector of the virtual character, and u(t) is the external control input;

[0185] The adaptive adjustment rule of the control input v(t) is as follows:

[0186]

[0187] Among them, K(t) is the adaptive gain matrix, is the estimated gain matrix;

[0188] The adaptive algorithm gradually optimizes the control input by adjusting K(t) to achieve the expected change in the emotional state of the virtual character;

[0189] Step 5.4: To ensure the stability and response speed of the emotional feedback, the Lyapunov stability theory is used to design the adaptive controller. Let v(θ(t)) be the Lyapunov function, which is a function of the emotional state θ(t) and represents the "energy" of the system, and satisfies:

[0190] Through the appropriate Lyapunov function v(θ(t)), the stability of the emotional feedback system is ensured. The design goal of the adaptive controller is to ensure that the system can quickly return to the desired emotional state under perturbations or changes;

[0191] Step 5.5: Once the control law is designed, the emotional feedback of the virtual character can be applied to the virtual interaction system;

[0192] Step 5.6: By collecting the interaction data of the user, evaluate the effect of the emotional feedback, and further optimize the parameters in the cost function Y and the adaptive control gain K(t) to improve the quality of the emotional feedback of the virtual character and the interaction experience.

[0193] The benefit of Step 5.1 By introducing the model of the hybrid power system, the discrete changes and continuous evolution of the emotional state of the virtual character can be accurately captured, ensuring the immediacy and dynamics of the emotional feedback and user interaction;

[0194] The benefit of Step 5.2 The combination of discrete and continuous events ensures that the change of the emotional state can delicately reflect the user's behavior and can respond to the dynamic adjustment of the system, forming a natural and smooth emotional feedback;

[0195] The benefit of Step 5.3 The adaptive control law can intelligently adjust the emotional feedback of the virtual character according to different interaction situations, making the emotional expression of the virtual character personalized and accurate, and improving the user's interaction experience;

[0196] The benefits of step 5.4 The Lyapunov theory provides a method to ensure the stability of the system, avoid the out-of-control of the emotional feedback of the virtual character, and ensure the smooth and continuous emotional feedback during the interaction;

[0197] The benefits of step 5.5 The adaptive controller adjusts the emotional feedback in real time during virtual interaction, ensuring that the emotional expression of the virtual character is consistent with the changes in the user's behavior and situation, and improving the naturalness and immersion of the interaction;

[0198] The benefits of step 5.6 Dynamically collect interaction data and optimize and adjust the system, making the emotional feedback of the virtual character accurate and meeting the user's needs, and improving the user experience and the quality of emotional interaction;

[0199] Overall summary, by using a hybrid power system to model the emotional state of the virtual character and combining an adaptive control law to adjust the emotional feedback, the present invention can achieve precise control of the virtual character's emotions. Compared with the traditional static feedback method, the design of the hybrid power system makes the emotional feedback of the virtual character natural, personalized and stable. In addition, using the Lyapunov stability theory and adaptive control optimization technology ensures the stability and accuracy of the virtual character's emotional state in various dynamic interaction environments, thereby significantly improving the user experience and interaction quality of the virtual interaction system.

[0200] Step S6, integrate the results of steps S1 to S5 to develop a virtual interaction system, and realize the real-time interaction between the virtual object, tactile feedback, the emotional state of the virtual character and the user's behavior through sensor data acquisition.

[0201] By integrating the state estimation of the virtual object, tactile feedback, the real-time interaction between the emotional state of the virtual character and the user's behavior, a comprehensive and dynamic interaction experience can be provided. The system can respond in real time to the user's behavior and emotional changes, and interact seamlessly with the objects and characters in the virtual environment.

[0202] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A 3D animation virtual interactive display method, characterized in that: include: Step S1, using Lie group theory to describe the position and posture of the virtual object in the three-dimensional space and introducing Lie algebra to represent small perturbations to complete the registration between the virtual object and the real scene; Step S2, using an extended Kalman-Bush filter to estimate the state of the virtual object described in step S1; Step S3, based on the virtual object posture determined in step S1, the contact area is discretized using a nonlinear finite element method and node displacements are calculated, and then the stress field is obtained by an integral method to construct a tactile and force feedback simulation model; Step S4, based on the tactile and force feedback simulation model constructed in step S3, a feedback control law is designed using the HJB optimal control theory to adjust the tactile feedback; Step S5, combining the technical contents of steps S1 to S4 to construct a hybrid power system to describe the emotional state of the virtual character, wherein the hybrid power system adopts an adaptive control law to adjust the emotional feedback; Step S6, integrating the results of steps S1 to S5 to develop a virtual interaction system, and realizing real-time interaction between virtual objects, tactile feedback, emotional states of virtual characters and user behaviors through sensor data collection.

2. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S1 further comprises: Step 1.1: Establish the pose representation of the virtual object based on Lie group SE (3), that is: Among them, R1 is the rotation matrix, t1 is the translation vector, and T1 is the initial transformation matrix; Step 1.2: Introduce Lie algebra se(3) to model small perturbations, and define small perturbations of posture as elements in Lie algebra se(3): Among them, Δα1 is the rotation perturbation, Δβ1 is the translation perturbation, is the position perturbation, and then the exponential mapping is used to transform the Lie algebra to the Lie group, and the perturbed transformation is obtained: in, is an antisymmetric matrix, T′1 is the transformation matrix after perturbation; Step 1.3: Combine the extended Kalman filter to estimate the state of the virtual object. When performing the registration, the motion state of the virtual object Described by the differential equation: in, is the state vector of the virtual object, represents the state transition equation of the system, is the noise impact matrix, is a standard Wiener process.

3. A 3D animation virtual interactive display method according to claim 1, characterized in that: In step S2, the state of the virtual object in step S1 is estimated by an extended Kalman-Bush filter, with the aim of optimizing the position and attitude estimation of the virtual object through sensor data and the position and attitude estimation results in the previous steps.

4. A 3D animation virtual interactive display method according to claim 3, characterized in that: The step S2 further comprises: Step 2.1: Describe the state evolution of the virtual object through differential equations. The state of the virtual object consists of position and posture. The state vector is defined as: in, is the state vector, R1 is the rotation matrix, t1 is the translation vector, and the evolution of the state is expressed by the differential equation: in, is the state transfer matrix, is the noise propagation matrix, is the standard Wiener process; Step 2.2: Predict the state of the virtual object through the state transfer equation. Assume that at time τ1, the state of the virtual object is The prediction step is then performed using the following formula: in, is the white noise increment; Step 2.3: After the prediction step, the sensor measurement data is used for updating. Assuming that the measurement value obtained by the sensor is Z1, the update step is performed using the following Kalman gain formula: in, is the prediction error covariance matrix, H1 is the measurement matrix, and R1 is the measurement noise covariance matrix; Update the state estimate of the virtual object by calculating the Kalman gain K1: Updated status is the best estimation result of the virtual object at the current moment; Step 2.4: After updating the virtual object state, update the error covariance matrix The error covariance matrix reflects the uncertainty of the current state estimate: Among them, I is the unit matrix, and the updated error covariance matrix is Provides a basis for estimation accuracy for subsequent filtering steps; Step 2.5: Combined with the virtual object posture calculated in step S1, through the above Kalman filter update process, the state of the virtual object will be optimized and corrected at each moment to ensure the dynamic matching of the virtual object and the real scene.

5. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S3 further comprises: Step 3.1: Determine the contact area between the virtual object and the real scene, discretize the contact area into elements in the finite element model through mesh division, define the mesh node set and the finite element unit set, and discretize the contact area using the nonlinear finite element method; The distribution of physical quantities between each node and adjacent nodes is described by shape functions, and the stiffness matrix K of each unit is obtained. j and load vector F j , matrices and vectors are calculated using the following formulas: Where B is the strain-displacement matrix, C is the constitutive matrix of the material, N is the shape function matrix, f is the external force vector, and dA is the integration area; Step 3.2: Calculate the node displacement by solving the equilibrium equation of the discretized finite element system. The equilibrium equation is: KU = F, Among them, U is the displacement of all nodes, F is the external force, and K is the stiffness matrix. After solving the node displacement, the stress field is calculated using the displacement data. The stress field σ(x) is obtained by the following formula: σ(x) = CBU, Among them, B is the strain-displacement matrix, C is the constitutive matrix of the material; Step 3.3: Combine the calculated stress field with the displacement information through the integration method to establish a tactile and force feedback simulation model. The force feedback model calculates the total force F exerted on the virtual object when it contacts the real object through integration. total : F total =∫ Γ σ(x)dA, Where Γ is the contact surface, σ(x) is the stress field, and dA is the integration area.

6. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S4 further comprises: Step 4.1: In step S3, the displacement and stress field of the contact area between the virtual object and the user are calculated by the nonlinear finite element method. In order to establish a tactile feedback model, it is necessary to define the state variables of the system; and according to the calculation results of step S3, the input of tactile feedback is mainly related to the displacement and stress of the contact area and the torque of the contact point; Step 4.2: To design the optimal tactile feedback control law, the HJB optimal control theory is used. The core of the HJB optimal control theory is to obtain the optimal control input by solving the Hamilton-Jacobi equation. The cost function J of the system is defined as: Where L(x(t),u(t)) is the instantaneous cost function, λ(x(t)) is the state constraint function, x(t) is the system state vector, u(t) is the system control input, t is time, and J is the cost function; According to the HJB optimal control theory, the Hamiltonian of the system is expressed as: H(x,u,p)=L(x,u)+p T f(x,u), Among them, L(x,u) is the instantaneous cost function, f(x,u) is the system dynamics function, H(x,u,p) is composed of the instantaneous loss and the dot product of the adjoint variable and the dynamics function, p T is the transposed matrix of the adjoint variable p.

7. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S4 further comprises: Step 4.3: To obtain the optimal control law, it is necessary to solve the HJB equation, which is: Where V(x) is the value function, is the gradient of the value function with respect to the state, It is a minimum operation in all feasible control input spaces, L(x,u) is the instantaneous cost function, and f(x,u) is the system dynamics function; The control input is calculated as follows: in, It returns the set of control inputs u that minimize the objective function. Step 4.4: Once the optimal control input u is obtained through the HJB equation * (x), can be applied to virtual interactive systems; Step 4.5: During the operation of the system, adjustments will be made based on the user's real-time feedback and environmental changes, and the design of the cost function L(x,u) will be further optimized by collecting user interaction data.

8. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S5 further comprises: Step 5.1: Based on steps S1 to S4, the emotional state of the virtual character is modeled by a hybrid power system; The emotional state θ(t) is represented as a vector containing the emotional variables of the avatar, which are described by the following state equation: Among them, θ(t) is the emotional state vector of the virtual character, u(t) is the external control input, y(t) is the user behavior feedback of interacting with the virtual object, and f θ It is a function that describes the changes in the emotional state of the virtual character; Step 5.2: In order to adjust the emotional feedback of the virtual character, a hybrid system model is used to describe the change of emotional state. The hybrid system includes discrete events and continuous dynamics to describe the discrete changes and continuous evolution of emotions. The hybrid powertrain model is expressed as follows: Where x(t) is the state variable of the system, u(t) is the external control input, A is the system matrix, B is the input matrix, and X is the state space.

9. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S5 further comprises: Step 5.3: The adaptive control law is used to adjust the emotional feedback of the virtual character. It can be adjusted according to the user's behavior or the interactive state of the virtual object. The core idea of ​​adaptive control is to dynamically adjust the control law according to the system state feedback; The design of the control law is based on the dynamic equation of the virtual character's emotion. Let the emotional feedback control input be v(t), which is the adaptive control input for adjusting the emotional state. The goal is to minimize the cost function. The cost function Y is defined as: Among them, Q is used to weigh the importance of the virtual character's emotional state, R is used to weigh the cost of the control input, θ(t) is the virtual character's emotional state vector, and u(t) is the external control input; The adaptive adjustment rule of the control input v(t) is as follows: Where K(t) is the adaptive gain matrix, is the estimated gain matrix; The adaptive algorithm gradually optimizes the control input by adjusting K(t) to achieve the expected change in the emotional state of the virtual character.

10. A 3D animation virtual interactive display method according to claim 1, characterized in that: The step S5 further comprises: Step 5.4: To ensure the stability and response speed of the emotional feedback, the Lyapunov stability theory is used to design an adaptive controller. Let v(θ(t)) be the Lyapunov function, which is a function of the emotional state θ(t), representing the "energy" of the system and satisfying: The stability of the emotional feedback system is ensured by using a suitable Lyapunov function v(θ(t)). The design goal of the adaptive controller is to ensure that the system can quickly return to the desired emotional state under disturbance or change. Step 5.5: Once the control law is designed, the emotional feedback of the virtual character can be applied to the virtual interactive system; Step 5.6: By collecting user interaction data, evaluating the effect of emotional feedback, further optimize the parameters in the cost function Y and the adaptive control gain K(t) to improve the quality of the virtual character's emotional feedback and interactive experience.

Images