A three-dimensional interactive gesture generation and correction method, system and storage medium

Abstract

The application provides a three-dimensional interactive posture generation and correction method, system and storage medium, and belongs to the technical field of intelligent sensing, comprising the following steps: extracting the functional surface unit normal vector and the functional surface geometric center of a target object from a three-dimensional object point cloud; converting the structured prompt containing the functional surface unit normal vector and the functional surface geometric center into natural language motion description text through a large language model; generating an initial multi-frame human body interactive posture sequence through a diffusion model with the natural language motion description text as a condition; performing analytical normal vector alignment correction on the posture sequence by using the Rodrigues rotation formula; performing gravity consistency checking on the corrected posture; and performing global orientation disambiguation on the posture passing the gravity checking to eliminate the 180-degree orientation ambiguity of the normal vector correction and output the final interactive posture. The application can output a three-dimensional human body interactive posture with geometric precision, physical naturalness and reasonable interactive logic at extremely low computational cost, and can meet the performance requirements of real-time interaction in VR, AR and other scenes.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing technology, specifically relating to a method, system, and storage medium for generating and correcting three-dimensional interactive postures. Background Technology

[0002] With the development of technologies such as collaborative robots, digital twins, virtual reality / augmented reality, smart manufacturing, and digital human interaction, systems not only need to identify objects in 3D scenes, but also need to understand "how humans will interact with objects." 3D interactive posture generation is a key technology addressing this need: its goal is to automatically generate human postures that conform to human movement patterns, object functional attributes, and spatial constraints, given 3D objects, scene geometry, and interaction intentions. For example, for a chair, the system not only needs to know that its category is "chair," but also needs to generate a human posture that allows for natural sitting; for a door handle, the system not only needs to identify its position, but also needs to generate a grasping or opening posture that matches the arm and wrist direction with the handle.

[0003] Generating 3D interactive postures has significant application implications. On one hand, in human-robot collaboration and intelligent manufacturing scenarios, accurately predicting or generating future interactive postures helps robots anticipate human intentions, enabling safe obstacle avoidance, path planning, and collaborative operations, reducing the risk of collisions and misoperations. On the other hand, in VR / AR training, digital human animation, game simulation, and embodied intelligence tasks, systems need to automatically synthesize natural, believable, and controllable human movements based on target objects and interactive semantics, reducing the cost of manual animation production and pose annotation. If the generated posture only satisfies the requirement of "reaching the target position" but does not meet the requirements of the target object's functional surface orientation, contact relationships, and gravity rationality, problems such as body suspension, clipping, misalignment of contact parts, incorrect sitting posture orientation, or abnormal gripping angles can easily occur.

[0004] Among existing technologies, GAP3DS (Gaze-DrivenDual-Mode Human Motion Prediction in 3D Scenes, CVPR 2025), proposed by Stanford University and the Max Planck Institute, is one of the most representative methods for generating 3D interactive poses. GAP3DS is the first to integrate affordance detection and diffusion-based human motion generation into a unified framework: First, a 3D scene understanding network is used to detect affordance labels on object point clouds (e.g., a chair surface is labeled "can be sat on", a door handle is labeled "can be grasped"). Then, the detected affordance labels and gaze direction are injected as conditional signals into a pre-trained motion diffusion model (MDM) to generate a sequence of human interactive poses associated with functional regions of the scene.

[0005] However, GAP3DS suffers from a fundamental flaw that limits the geometric accuracy of interactive poses: its functionality availability detection module outputs only at the semantic level, providing information such as discrete labels or heatmaps indicating which areas of an object support which interactive functions (e.g., the chair surface supports 'sitting'). Furthermore, because GAP3DS relies entirely on coarse visual attention signals—using the gaze direction as orientation guidance—and since the gaze direction is not a precise geometric constraint, the generated interactive poses exhibit severe deviations in orientation accuracy. Character poses often display geometric misalignments such as "buttocks hovering above the chair surface but facing crookedly" or "hands reaching the target object but with an incorrect grip angle," making it difficult to meet the requirements of real-world interactive scenarios. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method for generating and correcting 3D interactive postures. Specifically, it involves extracting functional surface normal vectors from the functional regions of a target object, generating motion descriptions using a large language model, driving a diffusion model to generate interactive postures, and outputting the 3D interactive postures through three levels of geometric, physical, and semantic correction. The aim is to establish a complete closed-loop process from functional surface normal vector extraction, language cue bridging diffusion generation, Rodrigues analytical orientation correction, gravity consistency verification to global orientation disambiguation, ensuring that the generated 3D interactive postures not only meet the semantic requirements of interaction but also achieve precise geometric alignment between the interactive parts and the functional surfaces.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for generating and correcting 3D interactive poses includes the following steps: Obtain the 3D point cloud of the target object; Extract the functional surface unit normal vector and functional surface geometric center of the target object from the 3D object point cloud; generate structured prompt text representing human-computer interaction context information based on the target object category and functional label, human interaction part name, and functional surface direction description converted from the functional surface unit normal vector; perform semantic generation and text encoding on the structured prompt text in sequence to generate an initial human interaction posture sequence. The unit direction vector corresponding to the human interaction part is extracted from the initial multi-frame human interaction posture sequence; the target geometric constraint is determined according to the interaction type; the posture is rotated and corrected by combining the unit direction vector and the functional surface unit normal vector, so that the orientation of the human interaction part meets the corresponding target geometric constraint; the translation vector is solved based on the geometric center of the functional surface; the rotation posture is translated as a whole by using the translation vector; the human interaction reference joint is aligned to the geometric center of the functional surface; and the corrected human interaction posture sequence is output.

[0008] Preferably, extracting the unit normal vector and geometric center of the functional surface of the target object from the three-dimensional object point cloud specifically includes the following steps: The OpenAD network is used to detect functional regions on a 3D object point cloud and output a point-by-point confidence heatmap. With adaptive threshold τ for thermodynamics Figure 2 Value-based analysis is performed to extract the functional surface point set of the functional region through three-dimensional connected component analysis; the mean value of all three-dimensional coordinates within the functional surface point set is calculated to obtain the geometric center of the functional surface; The RANSAC algorithm is used to remove outlier noise points from the functional surface point set. Least square plane fitting is performed on the remaining functional surface points to construct the covariance matrix of the interior point set and perform eigenvalue decomposition. The eigenvector corresponding to the smallest eigenvalue is taken, normalized, and used as the unit normal vector of the functional surface. The interior point is a three-dimensional point in the functional surface point set whose distance to the fitting plane generated by RANSAC iteration is less than a preset error threshold.

[0009] Preferably, the step of generating structured prompt text representing human-computer interaction context information based on the target object category and function label, the name of the human body interaction part, and the surface direction description obtained by converting the functional surface unit normal vector specifically includes the following steps: Obtain the current target object category and function label from the 3D object point cloud; Based on the current target object category and function label, query the preset mapping table to match the name of the human body interactive part corresponding to the current interactive action; Based on the three-dimensional spatial orientation rules, the unit normal vector of the functional surface is converted into a functional surface orientation description in natural language form; The target object category, function label, human interaction part name and functional surface direction description are unified and integrated; Following a fixed and regular text format, all integrated interactive information is arranged and output to generate structured prompt text that represents the context information of human-computer interaction; the complete context information of human-computer interaction includes semantic information of the target object, information of human body interaction limbs, and spatial orientation information of functional surfaces.

[0010] Preferably, the structured prompt text is sequentially subjected to semantic generation and text encoding to generate an initial human interaction posture sequence, specifically including the following steps: The structured prompt text is input into the large language model, and the corresponding human-computer interaction motion description text is generated based on the human-computer interaction context information contained in the structured prompt text. The generated human-computer interaction motion description text is input into the CLIP text encoder, the text is encoded, and the corresponding semantic feature vector is output. The obtained semantic feature vector is used as a conditional input to the MDM diffusion model, which is then used to control the MDM diffusion model to perform backdiffusion sampling operations and output an initial multi-frame human interaction posture sequence.

[0011] Preferably, the rotation correction of the attitude using the Rodrigues rotation formula specifically includes the following steps: Based on the unit direction vector corresponding to the human body interaction parts and the unit normal vector of the functional surface, the rotation matrix is ​​obtained by solving the rotation logic of the Rodrigues rotation formula, and then the rotation matrix is ​​solved. The solved rotation matrix is ​​used to synchronously perform rotation transformation on all three-dimensional joint coordinates of the human body within the initial human interaction pose sequence; Complete all joint rotation operations to orient the human interactive parts toward the preset target geometric constraints, and obtain the human interactive posture sequence after rotation correction.

[0012] Preferably, the method further includes performing a gravity consistency check on the corrected human interaction posture sequence, specifically: The verticality of the posture is determined by the human torso vector; if the verticality does not reach the preset threshold, the gravity compensation rotation matrix is ​​solved to correct the posture orientation, and the compensation rotation is superimposed on the original Rodrigues rotation matrix. The posture alignment operation is then performed again until the verticality of the posture meets the standard. Global orientation disambiguation is performed on postures that meet the verticality verification standard to eliminate the 180° bidirectional equivalent orientation ambiguity generated by Rodrigues normal vector correction. The unique reasonable human orientation is determined by combining scene spatial information, and the final human interaction posture sequence is output. The scene spatial information includes the geometric center of the functional surface, the unit normal vector of the functional surface, and the three-dimensional coordinates of all joints and the gravity direction vector of the torso in the current human posture sequence.

[0013] Preferably, the step of determining the verticality of the posture through the human torso vector specifically includes the following steps: Vertical measurement function Quantifying the spine in the direction of gravity The average projected length on; In the formula, It is an indicator of verticality and is used to determine whether the human body is tilted. is the vertical unit vector in the world coordinate system. For the first The three-dimensional position of the thoracic joint in the frame. For the first The three-dimensional position of the pelvic joint in the frame; K This represents the total number of frames contained within the current pose sequence. When s(P) is lower than the preset minimum verticality threshold, an additional rotation matrix around the pelvic joint is automatically constructed to align the spine with the pelvic joint. Align the corrected attitude and send it back to the Rodrigues correction step to re-align the normal vectors until the verticality of the attitude meets the standard.

[0014] Preferably, the global orientation disambiguation of the posture that has passed the verticality verification specifically includes the following steps: The functional surface unit normal vector, the vector from the object's centroid to the center of the functional surface, the observer's orientation vector, and the multi-dimensional features of the human root joint pose are input into a three-layer MLP network. The three-layer MLP network outputs binary classification probabilities for objects facing and objects facing away from them. When the probability of facing away is greater than the probability of facing forward, apply a 180-degree rotation matrix around the z-axis of the world coordinate system. Perform orientation flipping to achieve global orientation disambiguation.

[0015] The present invention also provides a three-dimensional interactive pose generation and correction system, comprising: The data acquisition module is used to acquire the 3D point cloud of the target object. The target-oriented availability parsing module is used to extract the functional surface unit normal vector and functional surface geometric center of the target object from the 3D object point cloud; generate structured prompt text representing human-computer interaction context information based on the target object category and functional label, human interaction part name, and functional surface direction description converted from the functional surface unit normal vector; and perform semantic generation and text encoding on the structured prompt text in sequence to generate an initial human interaction posture sequence. The orientation-constrained interactive pose synthesis module is used to extract unit orientation vectors corresponding to human interaction parts from the initial multi-frame human interaction pose sequence; determine target geometric constraints based on the interaction type; combine the unit orientation vectors with the functional surface unit normal vectors; and perform rotation correction on the pose using the Rodrigues rotation formula to ensure that the orientation of the human interaction parts meets the corresponding target geometric constraints; solve for the translation vector based on the geometric center of the functional surface; use the translation vector to translate the rotated pose as a whole; align the human interaction reference joints to the geometric center of the functional surface; and output the corrected human interaction pose sequence. The present invention also provides a computer-readable storage medium storing a computer program, which, when loaded by a processor, is capable of executing any of the steps in the three-dimensional interactive pose generation and correction method.

[0016] The three-dimensional interactive pose generation and correction method provided by this invention has the following beneficial effects: This invention extracts the unit normal vector and geometric center of the functional surface, transforming purely semantic-level availability information into quantifiable and precise geometric constraints, replacing fuzzy gaze direction signals. It then generates an initial interactive posture based on the unit normal vector of the functional surface, incorporating precise spatial orientation information during the posture generation stage. Simultaneously, it achieves translational alignment based on the geometric center of the functional surface, precisely calibrating the initial interactive posture to ensure accurate alignment between the interactive part and the object's functional surface. The output interactive posture simultaneously meets the requirements of interactive semantics, spatial geometry, and human body position matching, significantly reducing geometric alignment errors. It can output postures that closely resemble real human-computer interaction, adapting to the accuracy and performance requirements of real-time VR and AR interactive scenarios. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an architecture diagram of the three-dimensional interactive posture generation and correction system of Embodiment 1 of the present invention; Figure 2 This is a data processing flowchart of the three-dimensional interactive posture generation and correction system of Embodiment 1 of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0020] Given objects in a 3D scene, automatically generating realistic, physically reasonable, and geometrically accurate human-object interaction postures is a common underlying support for virtual reality (VR) immersive interaction, augmented reality (AR) operation guidance, robot imitation learning, automatic generation of digital human animation, and intelligent behavior synthesis of game characters.

[0021] Existing methods for 3D functional availability detection and diffusion generation suffer from a systematic deficiency in the crucial information conversion stage from semantics to geometry. Specifically, the system knows "what" (can sit here) and "generally which direction to look" (where to look), but it doesn't know the precise spatial orientation of the functional surface (how the surface is oriented). Experimental reports show an orientation error as high as 1.421 radians (approximately 81.4 degrees), thus failing to provide explicit, quantifiable orientation constraints for human interaction points. This semantic-geometric gap is the core technical problem that this invention aims to solve.

[0022] Unlike the traditional task of "reconstructing human pose from a given human image", the core challenge of 3D interactive pose generation lies in the fact that the system needs to infer a reasonable human spatial configuration based solely on object information and interactive intent, without any human observation input. This includes the human's position and orientation relative to the object, the posture of each joint, and the motion sequence.

[0023] Therefore, the significance of this invention lies in elevating the availability of object functions such as "can be sat on," "can be grasped," and "can be placed" from discrete semantic labels to functional surface normal vector constraints with geometric directions. This constraint is then integrated into the processes of language prompting, diffusion generation, and posture correction, ensuring that the generated postures simultaneously meet the requirements of interactive semantics, spatial geometry, and physical plausibility. Consequently, this invention provides a more reliable, natural, and interpretable 3D interactive posture generation capability for real-time human-computer collaboration, immersive interactive training, digital human behavior generation, and intelligent understanding of 3D scenes.

[0024] As a fundamental geometric quantity describing the orientation of a three-dimensional surface, the normal vector is of decisive significance in guiding the spatial orientation of human interaction parts. For example, when sitting in a chair, the human hip-spine direction needs to be orthogonal to the normal vector of the chair surface, and when opening a door, the hand-forearm direction needs to be parallel to the normal vector of the door handle area.

[0025] Based on this, this invention first proposes a 3D interactive pose generation and correction system with functional surface normals as the core geometric constraint throughout. This system consists of a data acquisition module, a GDAP module (Goal-directed Affordance Parser), and an OIPS module (Orientation-Constrained InteractivePoses Synthesizer). The data acquisition module is used to acquire the 3D point cloud of the target object, while the GDAP and OIPS modules are the two core modules.

[0026] In terms of functional availability detection, 3D AffordanceNet (CVPR 2021) proposed by Deng et al. and OpenAD (ICRA 2024) from the CMU Robotics Institute have achieved 3D functional detection with fixed categories and open vocabulary, respectively, but both remain at the semantic label output level. In terms of diffusion-based motion generation, MDM (ICLR 2023) and AffordMotion (CVPR 2024) generate motion with text and heatmaps as conditions, respectively, but they also lack explicit geometric constraints. Therefore, the GDAP module is responsible for detecting functional regions from the point cloud of the target object and explicitly extracting the geometric normal vectors of the functional surfaces, completing the key leap from semantics to geometry.

[0027] In terms of attitude correction, existing methods mainly rely on numerical iterative optimization or learning-based correction, while analytical geometric correction methods are still lacking. Therefore, the OIPS module is responsible for bridging structured information such as normal vectors to the diffusion model through a large language model to generate interactive poses, and performing a three-level progressive correction on the generated poses, including Rodrigues rotation formula correction, gravity consistency check, and global orientation disambiguation.

[0028] This invention employs an information flow path from semantic understanding, geometric extraction, language bridging, diffusion generation to three-level geometric correction. The system architecture is as follows: Figure 1 As shown.

[0029] This invention takes 3D object point clouds and functional labels as input, and completes the process from semantic detection to geometric information extraction through four modules of the GDAP module. Then, it completes language bridging, pose generation, three-level correction and output integration through seven modules of the OIPS module. Figure 1 The dashed arrows indicate mapping table queries (accessed on demand), while solid arrows indicate data flow. The normal vector α serves as the core geometric constraint connecting GDAP and OIPS. Simultaneously, the geometric direction prompts input into the PromptWrapper provide directional information for language generation, and the Rodrigues correction module serves as the geometric target for orientation alignment. The mapping table is a structured prior encompassing 66 valid mapping combinations across 15 object categories, 6 interactive functions, and 5 human body parts.

[0030] The functions of each core module are further explained below: 1. Functional surface normal vector extraction is performed using the GDAP module, specifically as follows: Figure 1 Modules 1-3.

[0031] Diffuse models such as MDM use the gaze direction as the orientation condition, but the gaze direction is only a coarse attention signal and cannot provide precise geometric constraints for the interactive parts of the human body. Due to the lack of normal vectors as alignment targets, there is an uncontrollable angular deviation between the orientation of the interactive parts generated in the pose and the normal vectors of the functional surfaces, resulting in problems such as orientation skew and contact surface misalignment.

[0032] Therefore, the GDAP module achieves a crucial leap from semantics to geometry in 3D object point clouds, explicitly outputting functional surface normal vectors. This is the most critical step in the entire process, upgrading functional availability information from semantic labels to a geometric representation that includes orientation information for the first time, providing a precise orientation target for downstream pose generation and correction.

[0033] The processing flow of the GDAP module is as follows: (1) OpenAD Functional Region Detection (Module 1): Based on a pre-trained 3D-LLM architecture network (PointBERT-base point cloud backbone + CLIP ViT-L / 14 text encoder), the open vocabulary alignment capability of CLIP is used to map the functional label text embedding and the local geometric features of the point cloud to a shared embedding space. Zero-shot functional region detection is achieved through similarity calculation. For each point P in the input point cloud P, i The output belongs to the functional category. confidence score .

[0034] (2) Adaptive threshold filtering and region extraction (Module 2): Using an adaptive threshold τ to filter thermal regions Figure 2 Values ​​are extracted to retain high-confidence point sets; three-dimensional connected component analysis (CCA) is performed based on the k-nearest neighbor graph (k=16) to remove fragmented and isolated noise regions, retaining only the point sets of main functional regions with spatial continuity. When no valid point sets are found after filtering, a threshold reduction strategy is triggered (step size 0.1, minimum reduction to 0.5).

[0035] (3) RANSAC Enhanced Least Squares Plane Fitting and Normal Vector Extraction (Module 3): Perform covariance matrix eigenvalue decomposition on the functional surface point set to obtain the initial normal vector estimate, take the eigenvector corresponding to the smallest eigenvalue, and improve the robustness to outliers by RANSAC random sampling consistency iteration in the outer layer. Finally, output the unit normal vector of the functional availability surface of the target object. ( ) and geometric center The normal vector undergoes direction consistency processing, meaning that after determining whether the normal vector points to the outside or inside of the object based on the interaction type, it becomes the core geometric constraint that runs through the entire OIPS module.

[0036] in, αTo perform RANSAC-enhanced least-squares plane fitting on the filtered and purified continuous functional surface point set, after covariance matrix eigenvalue decomposition and interaction type-driven orientation consistency processing, the output unit functional surface normal vector is used to characterize the spatial orientation of the target interactive surface; ||α||2=1 indicates that the normal vector has been normalized to a unit length, so that it only represents orientation information. For example, the orientation normal of interactive surfaces such as chair seats, door handles, and tabletops; α As the target alignment standard for posture correction, it is used throughout the entire process of language prompt generation and Rodrigues correction.

[0037] 2. Language prompt bridging and diffusion generation (OIPS module, modules 5-7) The OIPS module automatically transforms discrete structured information (object labels, function labels, normal vector directions, and interaction part names) into rich natural language motion descriptions using a large language model. These descriptions are then used as conditions to generate an initial sequence of human interaction postures via a diffusion model. This bridging mechanism eliminates the bottleneck of manually writing text conditions required in existing technologies.

[0038] The processing flow of the OIPS module is as follows: (1) PromptWrapper five-segment structured template (module 5): The object category, function label, interactive parts (obtained from the 15×6 mapping table in module 4), and normal vector direction description are filled into the template in the following order: system role prompt segment → object and action context segment → interactive parts segment → geometric direction prompt segment → command segment. Among them, the geometric direction prompt segment is the key node where normal vector information is injected into the generation process for the first time in the form of natural language.

[0039] (2) LLM Semantic Bridging (Module 6): Calls lightweight instructions to fine-tune the large language model (default Qwen2.0-7B-Instruct, inference temperature 0.2), takes the filled structured prompts as input, and generates natural language motion description text covering six dimensions of semantics: motion type, interaction method, interaction part, object attribute, spatial relationship, and motion style. (20-80 words).

[0040] (3) CLIP encoding and MDM diffusion generation (Module 7): The CLIP ViT-L / 14 text encoder will generate... The feature is encoded as a 512-dimensional semantic feature vector. Using this feature as a condition, an 8-layer denoising network based on the Transformer architecture of MDM is used to gradually denoise the random Gaussian noise by accelerating sampling with DDIM (50 steps) to generate a 22-joint human interaction pose sequence P with K=4 frames.

[0041] 3. Three-level progressive attitude correction (OIPS module, modules 8-10) The OIPS module performs a three-level progressive correction on the initial pose generated by the diffusion model, from local geometric accuracy to global semantic rationality. It is the core of the entire process to achieve accurate geometric alignment.

[0042] Level 1: Analytical Geometric Correction of Rodrigues' Rotation Formula (Module 8) The first level is the core of the three-level correction. It uses the functional surface normal vector α extracted by module 3 as the orientation target, and performs analytical rigid correction on the generated posture through the Rodrigues rotation formula. The closed solution precisely rotates the direction β of the interaction part to satisfy the predetermined geometric constraints with the functional surface normal vector.

[0043] The specific calibration process is as follows: Step 1: Extract the direction vector of the interaction part β .

[0044] Select the corresponding joint pair to define the direction vector based on the interaction function type (e.g., use the pelvis→chest direction for sitting posture, and the elbow→wrist direction for gripping posture), and extract and normalize the unit direction vector from the generated posture P. .

[0045] The local human vectors extracted from the original human pose joints generated by the diffusion model are variable vectors to be corrected. Each generated pose will produce different... β This characterizes the orientation of the limbs used for interaction in the human body, such as the direction of the hips and spine, or the direction of the arms and wrists.

[0046] Step 2: Determine the target geometric constraints.

[0047] For interactions involving sitting postures, leaning, and other orthogonal surface interactions, the objective constraint is: That is, the interaction direction is orthogonal to the normal vector; for surface-reverse interactions such as grasping and opening doors, the target is... That is, the interaction direction is parallel to the opposite direction of the normal vector.

[0048] in It is a three-dimensional rotation matrix; The unit normal vector is obtained by fitting the functional surface point set, representing the geometric orientation of the target interactive surface; This is a unit direction vector calculated from the adjacent joints of the target interaction part in the generated pose. For example, a sitting posture can be represented by the direction from pelvis to chest, and a gripping posture can be represented by the direction from elbow to wrist.

[0049] Step 3: Calculate the axis of rotation and the angle of rotation.

[0050] Rotation axis ,in For the functional surface normal vector The target direction and rotation angle are determined by the interaction type. .

[0051] Step 4: Constructing the Rodrigues rotation matrix: ; in It is a three-dimensional identity matrix. The unit rotation axis is obtained by normalizing the cross product of the current direction and the target direction. Let be the antisymmetric matrix of the rotation axis u; Let be the rotation angle, representing the angle by which the rotation about u is required; this formula gives The minimum geodesic rotation on the manifold from the current orientation to the target orientation.

[0052] It is the mathematical notation for the Special Orthogonal Group in 3D, used to represent the set of all legal 3D rotation matrices. If a matrix... belong , usually written Its meaning is:

[0053] ; in, This indicates that the matrix is ​​an orthogonal matrix, and that rotation will not change the vector length, angle, or relative geometry of the human skeleton; This indicates that the transformation is a pure rotation, without mirroring, scaling, or shearing. Therefore, in this invention, it is constructed using the Rodrigues formula. A rotation matrix means that the formula yields a strictly legal three-dimensional rigid rotation matrix that can be directly applied to the joints of a human body posture, allowing the posture to rotate as a whole without disrupting the proportions of the human skeleton and the relative relationships of the joints.

[0054] “ The "minimum geodesic rotation on a manifold" can be simply understood as: among all legal 3D rotations that can rotate the current interaction part's orientation to the target orientation, select the rotation with the smallest rotation angle and the shortest path. For this invention, this means that the system uses the minimum necessary rotation amount to align the orientation of the interaction part generating the posture to the target orientation determined by the functional surface normal vector, thereby avoiding unnecessary large rotations and making the corrected posture more natural and stable.

[0055] Step 5: Application of rigidity correction: Apply the same rotation matrix R to all relevant nodes in the attitude sequence, and use the translation vector t to align the reference joint with the geometric center μ of the functional surface point set, usually written as: ; in, It is a translation vector. Used as a reference joint for translation alignment, such as the pelvis joint in a seated pose or the wrist joint in a gripping scene.

[0056] The above parameters work together to accomplish two actions: first, use Adjust the orientation of the interactive parts of the human body, then use... Adjust the spatial position of the human body relative to the functional surface.

[0057] For example α =(0.02, 0.00, 0.9998), represents the normal vector when the chair surface is facing upwards. As long as the chair remains unchanged, this vector will never change. Original generated pose β =(0.15, -0.02, -0.988), indicating the original orientation of the person's pelvis and spine, which is tilted and not perpendicular to the chair surface; Correction logic: Construct rotation matrix R to rotate β ,make and The dot product is approximately 0, and the buttocks are perpendicular to the chair surface.

[0058] Level 2: Gravity Consistency Check (Module 9) This addresses the limitation of Rodrigues correction, which only ensures local normal alignment but fails to consider global physical plausibility. It quantifies the projected length of the human spine along the gravitational direction (z-axis in the world coordinate system) using a vertical metric function and compares it to a preset minimum verticality threshold. If an unreasonable posture, such as a skewed sitting position (verticality below the threshold), is detected, additional rotational correction around the Pelvis joint is automatically applied. If the correction fails, the posture is returned to module 8 for recalibration, forming a closed loop.

[0059] Level 3: OrientAware global orientation disambiguation (Module 10) To address the inherent 180-degree orientation ambiguity of Rodrigues rotation correction, where the normal vector defines only one direction line (with both positive and negative directions), simple normal vector correction may simultaneously produce two results: "facing the object" and "facing away from the object," both satisfying the constraints locally but with opposite global semantics. OrientAware uses a lightweight three-layer MLP (15→128→64→2) that takes 15 dimensions of features—normal vector, object geometric relationships, observer orientation, and human root joint pose—as input and outputs a binary classification probability of "facing / facing away," based on which it decides whether to apply a 180-degree flip around the z-axis.

[0060] like Figure 2 As shown, the above process is divided into four stages. The first stage (S1-S5) is geometric feature availability resolution: detecting functional regions from the object point cloud and explicitly extracting normal vectors, achieving a crucial leap from semantics to geometry. The second stage (S6-S9) is language-guided diffusion generation: bridging discrete labels and continuous motion space using a mapping table and a large language model to generate initial interactive poses. The third stage (S10-S13) is a three-level progressive correction: successively completing Rodrigues geometric correction, gravity physics verification, and OrientAware semantic disambiguation, ensuring pose quality step by step from local to global. The fourth stage (S14) is output integration, packaging and outputting the corrected pose sequence, aligned orientation vectors, and correction quality indicators. S12 is the only conditional branch in the process; when the gravity check fails, the corrected pose is sent back to S10 for normal vector alignment correction again.

[0061] This invention explicitly extracts the interaction surface normal vector from the functional availability region of the target object, and uses this normal vector as a unified geometric constraint throughout the entire generation and correction process. It establishes a closed-loop process from normal vector extraction, language bridging, diffusion generation, Rodrigues analytical correction, gravity verification to orientation disambiguation, thus solving the core defect of existing technologies that cause serious deviations in interaction posture orientation due to the lack of geometric normal vector constraints.

[0062] Functional availability region refers to the local spatial surface point set in a 3D object point cloud that has specific interactive functions and can support human or robot to complete corresponding actions.

[0063] The three-dimensional interactive pose generation and correction system proposed in this invention has the following advantages: (1) Analytical: No numerical iterative optimization is required, avoiding the computational overhead and local minima problem of hundreds to thousands of iterations of traditional gradient descent-based physical constraint optimization methods. The Rodrigues formula only requires one cross product, one inverse cosine, and one matrix construction to obtain the exact solution.

[0064] (2) Computationally efficient: It only involves basic linear algebra operations such as cross product, trigonometric functions and matrix multiplication, with a computational complexity of O(1). The total time for a single complete correction is less than 1 millisecond, which fully meets the real-time interaction requirements of VR / AR scenarios (usually required <16ms / frame).

[0065] (3) Minimum correction amplitude: Rodrigues rotation is The minimum geodesic rotation on the manifold results in the smallest correction amplitude and the shortest motion path, making the corrected human posture natural and smooth, avoiding the strange central orientation caused by "rotating around a long way".

[0066] (4) Numerical stability: When the rotation angle θ approaches zero, The formula automatically degenerates into an identity transformation. There is no singularity problem.

[0067] Taking a chair-sitting scenario as an example: before correction, the angle between the hip-spine direction and the chair surface normal vector in the generated pose deviated by approximately 9.6 degrees (normal vector alignment error of approximately 0.17). After Rodrigues correction, this deviation was reduced to nearly 0 degrees (normal vector alignment error). This achieves precise geometric alignment between interactive parts and functional surfaces. Gravity consistency checks further ensure the verticality of the posture (vertical projection of the spine is approximately 0.38m, within a reasonable range), and OrientAware disambiguation ensures that the character is facing the chair (facing confidence 0.92).

[0068] Each module in the aforementioned 3D interactive pose generation and correction system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0069] Based on the same inventive concept, this invention provides a method for generating three-dimensional interactive postures, specifically a method for generating three-dimensional interactive postures that integrates functional surface normal vector constraints and language-guided diffusion generation, comprising the following steps: Step 1: Detection of functional regions of 3D objects and extraction of high-confidence point sets The system acquires the 3D point cloud of the target object and its corresponding interactive function labels. It then uses an OpenAD network with a PointBERT point cloud backbone and a CLIP text encoder to perform zero-sample functional region detection, outputting a confidence heatmap of each coordinate point in the point cloud corresponding to various interactive functions. An adaptive confidence threshold is set to binarize the heatmap. Based on the k-nearest neighbor graph with k=16, 3D connected component analysis is performed to remove scattered noise points and retain only the spatially continuous main functional region point set. If there is no effective point set after filtering, the threshold is gradually lowered and the points are re-filtered.

[0070] Step 2: Solving for the unit normal vector and geometric center of the functional surface and calibrating the orientation. Perform RANSAC enhanced least squares plane fitting on the functional surface point set obtained in step 1, construct the point set covariance matrix and complete eigenvalue decomposition, select the eigenvector corresponding to the smallest eigenvalue as the initial normal vector, obtain the unit normal vector after normalization, and calculate the mean of the functional region point set as the geometric center; distinguish the normal vector pointing rules according to the current interactive function type, uniformly adjust the normal vector to face the inside or outside of the object, complete the direction consistency processing, and output the standardized functional surface unit normal vector α and the region center coordinate μ, that is, the functional surface geometric center.

[0071] Existing technologies only output functional heatmaps / discrete labels. This invention is the first to extract explicit geometric normal vectors from the functional regions of an object, upgrading availability from purely semantic information to geometric constraints with spatial orientation. Combined with RANSAC plane fitting set interaction type orientation calibration, it solves the semantic-geometric gap problem of only knowing where interaction is possible but not knowing which direction to interact.

[0072] Step 3: Query the mapping relationship of interactive parts and assemble the structured prompt text. The system retrieves a pre-built mapping knowledge base containing the correspondence between object categories, interactive functions, and human body interaction parts. Using the current object tag and function tag as search keys, it matches the target human body joint corresponding to this interaction. A five-segment PromptWrapper template is then used to sequentially fill in system role prompts, object and action context, interactive parts, natural language direction descriptions converted from normal vectors, and execution commands, concatenating them to generate a structured prompt text that fully represents the human-computer interaction context. The knowledge base uses object category and function tag as joint query keys to output the corresponding interactive human body part names, which guide template filling and correct target joint selection.

[0073] Step 4: Generate standardized motion description text using a large language model The assembled structured prompts are input into a lightweight large language model that has been fine-tuned by instructions. The inference temperature is set to 0.2, and the model is driven to output a natural language motion description covering six categories of semantic information: motion type, interaction method, interaction part, object attributes, spatial position relationship, and motion style. This is the human-computer interaction motion description text, with the text length controlled in the range of 20 to 80 words.

[0074] Traditional diffusion pose generation either relies on manually written text prompts or only uses a rough gaze direction as a condition. This invention transforms the geometric information of the normal vector into natural language through a fixed five-segment template, and then automatically generates a six-dimensional structured motion description using LLM. It automatically bridges the cross-modal gap between discrete function labels, geometric normal vectors, and diffusion model text input, eliminating the need for manual annotation / writing of prompts, and providing an original automated condition generation chain.

[0075] Step 5: Generate initial multi-frame human interaction pose sequence using the diffusion model. The CLIPViT-L / 14 text encoder is used to convert the human-computer interaction motion description text output by the large model into a 512-dimensional semantic feature vector. This feature vector is then used as a conditional input to the MDM motion diffusion model based on the Transformer architecture. The DDIM accelerated sampling strategy is used to perform 50 steps of reverse denoising to generate an original interactive pose sequence containing 4 frames and 22 human joints from random Gaussian noise.

[0076] Step 6: The Rodrigues rotation formula is used to perform analytical alignment correction of the local interactive parts. The joint direction vectors of the interactive parts are extracted from the original posture sequence and normalized. Geometric constraints are determined based on the interaction type. For sitting and leaning interactions, the interaction direction must be orthogonal to the normal vector of the functional surface after rotation. For grasping and opening interactions, the two must be parallel and opposite. The rotation axis is solved by cross product operation, and the rotation angle is calculated by inverse cosine function. These are then substituted into the Rodrigues rotation formula to construct the rotation. The spatial minimum geodesic rotation matrix is ​​applied synchronously with the translation vector to all joints of the attitude, completing the geometric alignment of the interactive parts and the functional surfaces.

[0077] Existing pose correction methods all rely on gradient iteration optimization and learning-based correction, lacking closed-form analytical solutions. This invention applies the Rodrigues rotation formula to human interaction pose alignment, directly providing... The minimum geodesic rotation closed-form solution is calculated in O(1) time, is iterative, and is numerically stable. It is designed with two types of geometric constraints, orthogonal and antiparallel, specifically for the functional surface normal vector.

[0078] Step 7: Gravity Consistency Verification and Attitude Closed-Loop Correction Construct a spinal verticality measurement function, calculate the average projection length of the thoracic cavity and pelvic joint on the vertical z-axis of the world coordinate system within all posture frames, and compare the calculation result with a preset verticality threshold. If the average projection length is lower than the threshold, automatically generate a compensation rotation matrix around the pelvic joint to correct the human body tilt state, and return the corrected posture to the Rodrigues correction step to re-execute the local alignment operation until the gravity verticality standard is met.

[0079] Simply rotating the spine in Rodrigues only ensures local joint alignment, which can lead to overall body tilt and abnormal posture that does not conform to gravity. This invention designs a spinal verticality measurement function. If the function fails to meet the standard, it automatically compensates for pelvic rotation and reverts to Rodrigues correction and realignment, forming a closed-loop correction mechanism that combines geometric correction with physical verification.

[0080] Step 8: Eliminate ambiguity in the 180-degree orientation of the normal vector using a multi-layer MLP network. Extract 15 features, including the unit normal vector, the vector from the object's centroid to the functional region center, the observer's direction, and the human root joint pose. Input these features into a three-layer MLP network with a 15→128→64→2 structure. The network outputs the probabilities of two outcomes: the person is facing away from the object and the person is facing away from the object. If the probability of facing away from the object is higher, apply a 180-degree rotation matrix around the world z-axis to flip the global orientation of the human body, thus completing global semantic rationality disambiguation.

[0081] A normal vector represents only a straight line, which naturally has ambiguity regarding orientation at 180°. Existing methods cannot distinguish between "facing an object" and "facing away from an object". This invention constructs a lightweight MLP with 15-dimensional joint feature input for binary classification, automatically judges and performs 180° flipping on the z-axis as needed, eliminating the inherent global semantic ambiguity of normal vector correction.

[0082] The 180-degree orientation ambiguity arises when, using the Rodrigues rotation formula to match the geometric constraints between the orientation of the human interaction part and the normal vector of the functional surface, there are mathematically two sets of equivalent rotation solutions differing by 180°. Both sets of solutions satisfy the vector constraints, but one set of postures does not conform to the physical logic of real human-computer interaction. This binary fuzzy problem is the orientation ambiguity. This ambiguity is eliminated by using global orientation disambiguation combined with spatial and gravity constraints to select the only reasonable posture.

[0083] Step 9: Integrate multiple types of data to output the final standardized interactive gesture results. The four human pose sequences after three levels of correction, along with the target orientation vector for alignment, normal vector alignment error, spinal verticality, orientation classification confidence, and other correction quality indicators, are summarized, and then uniformly packaged and integrated as the final human interactive pose sequence.

[0084] This invention first extracts the unit normal vector and geometric center of the interactive surface from the functional availability region of the target object, upgrading the object interaction information, which only has discrete semantic labels, to geometric constraints with precise spatial orientation and position information. Then, the geometric information, such as the normal vector and geometric center, is integrated into structured prompts, and motion description text is automatically generated using a large language model. This bridges the semantic gap between discrete functional labels and motion diffusion models, achieving end-to-end condition generation without manual prompt writing. Subsequently, attitude alignment correction is completed using the Rodrigues rotation formula with a closed-form analytical solution, and precise matching between the interactive part and the functional surface is achieved using minimum geodesic rotation. By avoiding the computational overhead and numerical instability issues caused by traditional iterative optimization, and by combining gravity consistency closed-loop verification and global orientation disambiguation mechanism, the shortcomings of single geometric rotation, which can only achieve local alignment and is prone to human posture tilting and orientation logic confusion, are overcome. The entire process forms a complete closed loop, eliminating the shortcomings of existing technologies such as large interaction posture orientation deviation, low geometric alignment accuracy, lack of physical rationality, and orientation ambiguity from geometric perception and semantic generation to multi-level progressive correction. Finally, it outputs geometrically accurate, physically natural, and logically reasonable 3D human interaction postures with extremely low computational overhead, which can meet the performance requirements of real-time interaction in VR, AR and other scenarios.

[0085] Example 1 Interactive posture generation: sitting chair scene Scenario Description: The user is in a virtual living room with a standard dining chair in front of them. The system needs to generate the user's sitting posture.

[0086] Detailed steps: 1. Input preparation: = chair, the target object category is "chair". (3D point cloud of the chair) point), This refers to the semantic labels of object categories obtained from OpenAD parsing, rather than the point cloud itself; .

[0087] 2. OpenAD Functional Region Detection: The OpenAD network detects the "sit" functional region on the chair point cloud. It outputs a point-by-point confidence heatmap. The confidence level for the area above the seat is between 0.85 and 0.95, while the confidence level for the area of ​​the chair legs is <0.1.

[0088] 3. Threshold filtering: For threshold filtering, 523 high-confidence points in the area above the chair surface were extracted to form a functional surface point set. .

[0089] 4. Normal vector extraction: For Perform RANSAC-enhanced least squares plane fitting to obtain the chair surface normal vector. =(0.02, 0.00, 0.9998) (approximately vertically upward), geometric center μ = (1.20, 0.50, 0.45) (coordinates of the center of the chair surface in space).

[0090] 5. Mapping table lookup: Query the mapping table to find the body parts corresponding to "chair"-"sit" as pelvis, spine, and hip.

[0091] 6. PromptWrapper: Fills in the prompt template: Object: "chair", Action: "sit", Primarybody part: "pelvis and hips", Normal direction hint: "the seat surface faces upward". Generates the complete prompt.

[0092] 7. LLM Semantic Bridging: Qwen2.0-7B generates motion description T = "A person lowers their body smoothly onto the flat horizontal seat surface of a chair, bending their knees and positioning their pelvis and hips directly above the seat before making contact, maintaining an upright spine posture throughout the motion." (Approximately 45 words, covering six semantic dimensions).

[0093] 8. CLIP encoding: CLIP ViT-L / 14 encodes T as a 512-dimensional feature vector.

[0094] 9. MDM Diffusion Generation: With CLIP(T) as the condition, DDIM generates a K=4 frame attitude sequence from random noise through 50 denoising steps. The current orientation of the Pelvis joint in the generated third frame (positioning frame). = (0.15, -0.02, -0.988) (slightly skewed, not perpendicular to the chair surface normal vector).

[0095] 10.Rodrigues correction: Calculate the axis of rotation: Normalization Calculate the included angle: , Rodrigues matrix: ; ; Application correction: After correction (Close to 0, satisfies orthogonal constraints)

[0096] 11. Gravitational consistency: Calculation (Neck is clearly above pelvis), passing the threshold check, no correction is needed.

[0097] 12. OrientAware Disambiguation: MLP Output The person is facing the chair. .

[0098] 13. Output: Final corrected attitude (4 frames), aligned to (Facing the front of the chair). Calibration quality indicators: , , .

[0099] Effect verification: Compared with the uncorrected generated pose (pelvis direction is skewed by about 9.6 degrees, and there is a visible angle mismatch between the character's buttocks and the chair surface), the corrected pose achieves parallel alignment between the buttocks and the chair surface, and the character presents a natural upright sitting posture.

[0100] Example 2 Interactive gesture generation: door opening scene Scenario description: The user is standing in front of a door, and the system needs to generate the user's interactive gesture of reaching out to open the door.

[0101] Detailed steps: 1. Input: (3D point cloud of a standard gate) .

[0102] 2. Normal Vector Extraction: OpenAD detects the "open" functional area (the area near the door handle), and after filtering, 147 high-confidence points of the handle are obtained. Plane fitting is then used to obtain the normal vector of the handle area. = (-0.996, 0.087, 0.012) (approximately horizontal, pointing towards the inside of the door), center μ = (2.30, 1.10, 1.05) (door handle position).

[0103] 3. Mapping table lookup: "door"-"open" corresponds to wrist, hand, forearm.

[0104] 4. LLM generation: T = "A person reaches forward with their right hand tograsp a door handle, extending their arm and wrist toward the handle at chestheight, preparing to pull the door open." 5. MDM Generation: Generate 4 frames of pose. In frame 2 (the gripping frame), the wrist-to-elbow direction (Right arm direction) = (0.823, -0.051, -0.566).

[0105] 6. Rodrigues correction: Objective constraints: (Hands facing inwards from the door, opposite to the normal vector) Rotation axis: ; Rotation angle: ; Construct R and apply it: Align the corrected wrist direction with the door handle normal vector.

[0106] 7. Gravity check: ,pass.

[0107] 8. OrientAware: Facing the door, it does not require a 180-degree rotation.

[0108] 9. Output: In the corrected posture, the wrist position is aligned with the door handle, and the hand orientation meets the geometric requirements for gripping the handle.

[0109] Example 3 VR leaning-in operation scenarios: The key role of gravity correction Scenario Description: In VR applications, users need to lean forward to manipulate objects on a tabletop. Because the user's real-time viewpoint in the virtual scene may be tilted relative to the functional surface, the generated posture can easily result in an unnatural, crooked posture.

[0110] Detailed steps: 1. Input: , .

[0111] 2. Normal Vector Extraction: Desktop Normal Vector = (0.00, 0.00, 1.00) (vertically upward), center μ=(1.00,0.80, 0.75).

[0112] 3. LLM generation: T = “A person leans forward slightly, extending their right forearm and hand toward a table surface to place an object on it.” (Note that this includes “lean forward”: a description of leaning forward, which may introduce tilting).

[0113] 4. MDM Generation: Due to the presence of "leans forward slightly" in the LLM description, the z-axis difference between the neck and pelvis is small in the generated pose of the second frame. (Caused by leaning forward).

[0114] 5. Rodrigues correction: After applying the correction, the wrist is aligned with the normal vector of the table, but the overall posture is still slightly forward.

[0115] 6. Gravity Consistency Check (Critical Step): This triggers gravity correction. The system applies additional rotation around the pelvis joint to the posture, improving the alignment of the spine vector (neck-pelvis) with the z-axis. After correction... The posture returns to a natural slight forward tilt rather than an excessive tilt.

[0116] 7. Final output: The corrected posture presents a natural, slightly forward-leaning operating table posture, with a reasonable vertical direction and the wrist aligned with the functional area of ​​the table.

[0117] Key finding: This embodiment demonstrates the irreplaceable role of gravity consistency checks in a three-level correction system: even if Rodrigues corrections ensure local geometric alignment, physically unreasonable tilt postures will be output without gravity checks.

[0118] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions, on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the embodiments of the three-dimensional interactive pose generation and correction method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0119] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for generating and correcting three-dimensional interactive poses, characterized in that, Includes the following steps: Obtain the 3D point cloud of the target object; Extract the functional surface unit normal vector and functional surface geometric center of the target object from the three-dimensional object point cloud; Based on the target object category and function label, the name of the human body interaction part, and the functional surface direction description obtained by converting the functional surface unit normal vector, a structured prompt text representing the human-computer interaction context information is generated. The structured prompt text is sequentially semantically generated and text-encoded to generate an initial human interaction posture sequence; The unit direction vector corresponding to the human interaction part is extracted from the initial multi-frame human interaction posture sequence; the target geometric constraint is determined according to the interaction type; the posture is rotated and corrected by combining the unit direction vector and the functional surface unit normal vector, so that the orientation of the human interaction part meets the corresponding target geometric constraint; the translation vector is solved based on the geometric center of the functional surface; the rotation posture is translated as a whole by using the translation vector; the human interaction reference joint is aligned to the geometric center of the functional surface; and the corrected human interaction posture sequence is output.

2. The three-dimensional interactive pose generation and correction method according to claim 1, characterized in that, Extracting the unit normal vector and geometric center of the functional surface of the target object from the 3D object point cloud specifically includes the following steps: The OpenAD network is used to detect functional regions on a 3D object point cloud and output a point-by-point confidence heatmap. The heat map is binarized using an adaptive threshold τ, and the functional surface point set of the functional region is extracted through three-dimensional connected component analysis. The mean value of all three-dimensional coordinates within the functional surface point set is calculated to obtain the geometric center of the functional surface. The RANSAC algorithm is used to remove outlier noise points from the functional surface point set. Least square plane fitting is performed on the remaining functional surface points to construct the covariance matrix of the interior point set and perform eigenvalue decomposition. The eigenvector corresponding to the smallest eigenvalue is taken, normalized, and used as the unit normal vector of the functional surface. The interior point is a three-dimensional point in the functional surface point set whose distance to the fitting plane generated by RANSAC iteration is less than a preset error threshold.

3. The three-dimensional interactive pose generation and correction method according to claim 2, characterized in that, The process of generating structured prompt text representing human-computer interaction context information based on target object category and function label, human interaction part name, and surface direction description obtained by converting functional surface unit normal vectors specifically includes the following steps: Obtain the current target object category and function label from the 3D object point cloud; Based on the current target object category and function label, query the preset mapping table to match the name of the human body interactive part corresponding to the current interactive action; Based on the three-dimensional spatial orientation rules, the unit normal vector of the functional surface is converted into a functional surface orientation description in natural language form; The target object category, function label, human interaction part name and functional surface direction description are unified and integrated; Following a fixed and regular text format, all integrated interactive information is arranged and output to generate structured prompt text that represents the context information of human-computer interaction; the complete context information of human-computer interaction includes semantic information of the target object, information of human body interaction limbs, and spatial orientation information of functional surfaces.

4. The three-dimensional interactive pose generation and correction method according to claim 3, characterized in that, The structured prompt text is sequentially subjected to semantic generation and text encoding to generate an initial human interaction posture sequence, specifically including the following steps: The structured prompt text is input into the large language model, and the corresponding human-computer interaction motion description text is generated based on the human-computer interaction context information contained in the structured prompt text. The generated human-computer interaction motion description text is input into the CLIP text encoder, the text is encoded, and the corresponding semantic feature vector is output. The obtained semantic feature vector is used as a conditional input to the MDM diffusion model, which is then used to control the MDM diffusion model to perform backdiffusion sampling operations and output an initial multi-frame human interaction posture sequence.

5. The three-dimensional interactive pose generation and correction method according to claim 1, characterized in that, The rotation correction of the attitude using the Rodrigues rotation formula specifically includes the following steps: Based on the unit direction vector corresponding to the human body interaction parts and the unit normal vector of the functional surface, the rotation matrix is ​​obtained by solving the rotation logic of the Rodrigues rotation formula, and then the rotation matrix is ​​solved. The solved rotation matrix is ​​used to synchronously perform rotation transformation on all three-dimensional joint coordinates of the human body within the initial human interaction pose sequence; Complete all joint rotation operations to orient the human interactive parts toward the preset target geometric constraints, and obtain the human interactive posture sequence after rotation correction.

6. The three-dimensional interactive pose generation and correction method according to claim 1, characterized in that, This also includes performing gravity consistency verification on the corrected human interaction posture sequence, specifically: The verticality of the posture is determined by the human torso vector; if the verticality does not reach the preset threshold, the gravity compensation rotation matrix is ​​solved to correct the posture orientation, and the compensation rotation is superimposed on the original Rodrigues rotation matrix. The posture alignment operation is then performed again until the verticality of the posture meets the standard. Global orientation disambiguation is performed on postures that meet the verticality verification standard to eliminate the 180° bidirectional equivalent orientation ambiguity generated by Rodrigues normal vector correction. The unique reasonable human orientation is determined by combining scene spatial information, and the final human interaction posture sequence is output. The scene spatial information includes the geometric center of the functional surface, the unit normal vector of the functional surface, and the three-dimensional coordinates of all joints and the gravity direction vector of the torso in the current human posture sequence.

7. The three-dimensional interactive pose generation and correction method according to claim 6, characterized in that, The method of determining the verticality of posture using human torso vectors specifically includes the following steps: Vertical measurement function Quantifying the spine in the direction of gravity The average projected length on; In the formula, It is an indicator of verticality and is used to determine whether the human body is tilted. is the vertical unit vector in the world coordinate system. For the first The three-dimensional position of the thoracic joint in the frame. For the first The three-dimensional position of the pelvic joint in the frame; K This represents the total number of frames contained within the current pose sequence. When s(P) is lower than the preset minimum verticality threshold, an additional rotation matrix around the pelvic joint is automatically constructed to align the spine with the pelvic joint. Align the corrected attitude and send it back to the Rodrigues correction step to re-align the normal vectors until the verticality of the attitude meets the standard.

8. The three-dimensional interactive pose generation and correction method according to claim 7, characterized in that, The global orientation disambiguation process for postures that pass the verticality check includes the following steps: The functional surface unit normal vector, the vector from the object's centroid to the center of the functional surface, the observer's orientation vector, and the multi-dimensional features of the human root joint pose are input into a three-layer MLP network. The three-layer MLP network outputs binary classification probabilities for objects facing and objects facing away from them. When the probability of facing away is greater than the probability of facing forward, apply a 180-degree rotation matrix around the z-axis of the world coordinate system. Perform orientation flipping to achieve global orientation disambiguation.

9. A three-dimensional interactive pose generation and correction system, characterized in that, include: The data acquisition module is used to acquire the 3D point cloud of the target object. The target-oriented availability parsing module is used to extract the functional surface unit normal vector and functional surface geometric center of the target object from the 3D object point cloud. Based on the target object category and function label, the name of the human body interaction part, and the functional surface direction description obtained by converting the functional surface unit normal vector, a structured prompt text representing the human-computer interaction context information is generated. The structured prompt text is sequentially semantically generated and text-encoded to generate an initial human interaction posture sequence; The orientation-constrained interactive posture synthesis module is used to extract the unit direction vector corresponding to the human interaction parts from the initial multi-frame human interaction posture sequence; determine the target geometric constraints according to the interaction type; combine the unit direction vector with the functional surface unit normal vector; use the Rodrigues rotation formula to rotate and correct the posture so that the orientation of the human interaction parts meets the corresponding target geometric constraints; solve the translation vector based on the geometric center of the functional surface; use the translation vector to translate the rotated posture as a whole; align the human interaction reference joints to the geometric center of the functional surface; and output the corrected human interaction posture sequence.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by a processor, it is able to execute the steps of the method according to any one of claims 1 to 8.

Images