A jointed object pose generation method based on physical perception and graph diffusion

Abstract

This invention discloses a method for generating poses of articulated objects based on physical perception and graph diffusion, relating to the field of computer vision. The method first reconstructs the componentized geometric and physical properties of objects from images using a bi-branch neural implicit network and initializes component connectivity relationships. Subsequently, it refines the component relationship graph through kinematic fitting and temporal consistency checks, and uses a physically enhanced graph diffusion process to infer the prior pose distribution of components on an SE(3) manifold. Finally, using this prior and reconstructed information as conditions, a conditional diffusion model is constructed on the SE(3) manifold, and backsampling is performed through a physically guided two-step backsampling framework to generate a diverse and physically plausible set of pose assumptions for articulated objects. This invention achieves efficient generation of diverse and highly physically plausible poses of articulated objects by tightly coupling physical laws with data-driven generation.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically, to a method for generating articulated object poses based on physical perception and graph diffusion. Background Technology

[0002] Articulated objects are objects composed of two or more parts connected by joints, capable of relative motion. Examples include tables, chairs, cabinets, and robotic arms. Generating poses for articulated objects is one of the core challenges in computer vision and robotics. With the rapid development of related fields, increasingly higher demands are being placed on the accuracy, physical plausibility, and pose diversity of articulated object pose generation.

[0003] Currently, methods for generating poses of articulated objects are mainly divided into two categories: traditional geometric modeling methods and deep learning-based generation methods. Traditional geometric modeling methods rely on manually designed features and precise physical models to solve for the object's pose through kinematic equations. Although this can ensure a certain degree of physical rationality, it has poor adaptability to complex scenes and is difficult to handle interference factors in real images such as occlusion and blurring, resulting in low pose generation efficiency.

[0004] Deep learning-based generative methods have become mainstream research areas due to their powerful feature extraction capabilities. Among them, graph neural network-based methods model articulated objects as graph structures composed of component nodes and joint edges, and infer the relationships between components through graph message passing, achieving some progress in pose structure modeling. However, these methods often ignore the physical properties of objects (such as mass, inertia, joint constraints, etc.), leading to physical violations in the generated poses, such as component penetration and joint movement exceeding the range of motion, making them unsuitable for direct application in real-world physical interaction scenarios.

[0005] In recent years, diffusion models have been gradually applied to pose generation tasks due to their excellent generative diversity. However, existing diffusion-based methods mostly perform sampling updates in Euclidean space, while the poses of articulated object parts belong to the SE(3) manifold (SE(3) is the mathematical space composed of all three-dimensional rigid body motions (rotation + translation), which is a smooth manifold). Directly transferring the diffusion process from Euclidean space to the SE(3) manifold will lead to distortion of pose representation. At the same time, existing methods do not effectively integrate physical constraints into the diffusion update process, making it difficult to balance pose diversity and physical rationality.

[0006] Therefore, the core challenge facing current articulated object pose generation technology is how to improve the physical rationality and structural accuracy of poses while ensuring the diversity of pose generation. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention proposes a method for generating poses of articulated objects based on physical perception and graph diffusion. By reconstructing the physical perception of components, the geometric and physical properties of articulated objects are obtained, providing a reliable physical constraint basis for pose generation. Through the graph diffusion process of graph refinement and physical enhancement, the accuracy of component relationships and the rationality of pose prior reasoning are improved. A physical-guided conditional diffusion model is constructed on the SE(3) manifold to achieve pose generation of articulated objects that combines diversity and physical rationality.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows:

[0009] A method for generating poses of articulated objects based on physics perception and graph diffusion includes the following steps:

[0010] Step 1: Construct a dual-branch joint representation network to reconstruct the geometry and physical properties of the jointed object's components from the image data, initialize the joint connection relationships between the components, and output the initialization parameters of the relationship graph between the components;

[0011] Step 2: Physically augmented hierarchical component relationship graph diffusion. The relationship graph initialization parameters are input into the physically augmented graph diffusion model inference to infer the attitude probability distribution of each component under task guidance.

[0012] Step 3: Based on the geometry and physical properties of the component and the probability distribution of the pose, construct a conditional diffusion probability model on the SE(3) manifold; adopt a physically guided two-step inverse sampling framework to gradually recover and optimize the pose from the noise, and finally generate a set of diverse and physically reasonable pose assumptions for articulated objects.

[0013] Furthermore, step 1 is detailed as follows:

[0014] Step 101: Encode the input image data in three ways: position, orientation, and pose. Then, concatenate the three encoding results to obtain shared features. Then, use a learnable gating splitting function to split the shared features into geometric features and physical features.

[0015] Step 102: Construct a dual-branch joint representation network, including a geometric branch and a physical branch;

[0016] Geometric Branch: Based on the extracted geometric features, firstly, an independent geometric feature extraction network is used to extract component-level geometric features for each component. All component-level geometric feature vectors are stacked into a matrix, and max pooling aggregation is performed on the matrix along the component dimension. Then, a fully connected layer is passed to obtain a fixed-dimensional global geometric encoding. Finally, a linear transformation layer is passed to simultaneously predict the signed distance value and its attribution logical value of the query point from the component-level geometric features.

[0017] The physics branch: Based on the extracted physical features, for each component i, a component-level physical attribute vector is predicted using an independent physical feature extractor; simultaneously, a global physical encoding describing the overall physical attributes is obtained through a global physical feature extractor and a linear layer; based on the component-level physical attribute vector, a lightweight joint parameter encoder is used to predict the joint parameters of joint i and the joint k connecting it to its parent component. The output includes complete parameters such as axial direction, joint type, pitch, and range of motion.

[0018] Step 103: Output the initialization parameters of the component relationship graph, including node attributes, edge candidate set, global geometric encoding, and global physical encoding; where node attributes are component attributes, including signed distance values, component-level physical attribute vectors, and local coordinate system transformations; each edge in the edge candidate set includes joint parameters.

[0019] Furthermore, step 2 is detailed as follows:

[0020] Step 201: Optimize the joint constraint energy function according to the joint type definition to accurately estimate the joint parameters; then use multi-view time series observation data to perform motion consistency test and confidence assessment on the kinematic relationship defined by the accurately estimated joint parameters, and then output a deterministic part relationship diagram with accurate joint parameters and confidence.

[0021] Step 202: Define the hidden state and message function of the node based on the component pose and physical information. Then, calculate the attention weight of the node using a physically-aware graph attention mechanism. Perform weighted aggregation based on the messages of neighboring nodes and attention weights to obtain the aggregated message of the current node. Calculate the state change using a differentiable simplified Newton-Euler dynamics module. The dynamics calculation is performed according to the joint type. Finally, input the current hidden state, the aggregated message, and the state change obtained from the dynamics calculation into the gated loop unit to update the node's hidden state.

[0022] Step 203: First, define the attitude energy function to measure the degree to which the attitude satisfies joint constraints and the consistency between the attitude and the task description; then, predict the attitude probability distribution at the minimum value of the attitude energy function, approximating the attitude distribution of each component as... Gaussian distribution on the manifold outputs the prior orientation distribution parameters for each component.

[0023] Furthermore, step 3 is detailed as follows:

[0024] Step 301: The conditional diffusion probability model includes two processes: forward diffusion and inverse denoising. The forward diffusion process is triggered by the component's attitude and is denoised through the corresponding Lie algebra. Gaussian noise is gradually added to the tangent space and projected back to the manifold via exponential mapping, completely destroying the attitude information into random noise; the inverse denoising process starts from the random noise, guided by multimodal conditional information, and learns a parameterized denoising network to gradually predict and remove noise, thus restoring the component attitude that conforms to physical and geometric constraints.

[0025] Step 302: The task obtains the task conditional feature vector through the pre-trained text encoder. Then, the global physical encoding, global geometric encoding, and task conditional feature vector are concatenated. Based on the cross-modal attention mechanism, the current noise pose and the concatenated features are dynamically fused to obtain the fused conditional vector. A scoring network for the inverse denoising process is constructed. Given the noise pose at step t, the number of diffusion steps, and the fused conditional vector, inverse denoising is performed. During training, the standard diffusion model denoising loss and the physical regularization loss based on physical laws are included. The physical regularization loss includes penetration penalty, static stability penalty, motion energy penalty, and joint constraint penalty.

[0026] Step 303: In In the tangent space, the Langevin dynamics update is performed based on the learned score network to obtain the initial denoising proposal from the noise, and then the proposed pose on SE(3) is obtained through exponential mapping; the physical violation degree under the proposed pose is calculated based on the physical regularization loss, and the physical correction gradient is obtained by backpropagation. The prediction result is corrected in the tangent space, and this step is repeated until the iteration stopping condition is reached.

[0027] Step 304: From Starting from different random noise types, run step 303 to generate N independent sampling trajectories, and finally obtain N candidate posture hypotheses; perform offline physical evaluation on each candidate posture, including evaluating the collision score of penetrating between components, the stability score of anti-tipping capability, energy score, and joint compliance score; sort all candidate postures according to the evaluation results, and output the top M postures with the highest scores.

[0028] Furthermore, the joint parameters are expressed as follows: ,in, It is a unit vector along the joint axis; It refers to joint types, including rotational, translational, fixed, and spiral joints. This is a pitch parameter, effective only for helical joints; range of motion Indicates the permissible range of joint movement. Represents the set of real numbers. As the lower limit, This is the upper limit.

[0029] Furthermore, step 201 is as follows:

[0030] Step 2011: For each candidate edge Using multi-view time-series observation data H represents the component pose, and the joint constraint energy function is optimized. To accurately estimate its joint parameters, the formula is as follows:

[0031]

[0032] in, This indicates the pose of component i. This indicates the pose of component j. Let K be the joint parameter. For accurate estimation of joint parameters, Based on joint type definition:

[0033]

[0034] in, These are the rotation matrices for components i and j, respectively. It is a three-dimensional rotation group. for up to its Lie algebra logarithmic mapping, Indicates the positions of components i and j. Let be the desired relative rotation vector. This represents a unit vector perpendicular to the joint axis. The amount;

[0035] Step 2012: Using multi-view time-series observation data, analyze the joint parameters based on accurate estimation. The defined kinematic relationship is subjected to motion consistency testing and confidence assessment; based on the joint parameters, the theoretical relative motion that components i and j should exhibit at each time step is calculated. At the same time, the measured relative motion at the corresponding moment is directly extracted from the actual observation data. The specific form of the relative motion is determined according to the joint type: for rotary joints it is the relative rotation angle, for translational joints it is the axial linear displacement, and for helical joints it is the generalized displacement of rotation-translation coupling. Its specific definition is consistent with the motion constraints of the corresponding joint type in step 2011.

[0036] The confidence score of the candidate connection is calculated by comparing the difference between the theoretical prediction and the actual observation. The formula is shown below:

[0037]

[0038] in, Indicates the number of time steps. For scale parameters; set threshold ,when If so, it is judged as a fake link and filtered out;

[0039] Step 2013: Determine the output of the refined component relationship diagram; output the refined component relationship diagram, where nodes contain the component code, signed distance value, physical attribute vector, and local coordinate system; edges contain precise joint parameters. and confidence level .

[0040] Furthermore, step 202 is as follows:

[0041] Step 2021: Hidden state of each node Through a recurrent neural network encoder Maintenance and updates:

[0042]

[0043] in, The orientation of component i on SE(3) Functions that map to vectors The angular velocity and linear velocity of the component are calculated from multi-view RGB-D time-series images or videos. This is a vector of physical properties of the component. External force;

[0044] via message function Map the hidden states and joint parameters of component nodes i and j to the message vector using external forces. ;

[0045] Message aggregation employs a physically-aware graph attention mechanism, with attention weights... It is determined by both node state similarity and connection stiffness, as shown in the following formula:

[0046]

[0047] in, For attention weights, Let k be the set of neighboring nodes of node i, where k is the index of a neighboring node. Let k be the hidden state of neighbor node k. The function represents the similarity between the hidden states of nodes i and j. The function represents the similarity between the hidden states of node i and its neighbor node k. This represents the estimate of the connection stiffness between the hidden states of nodes i and j. This represents the estimation of the connection stiffness between the hidden states of node i and its neighbor node k. An adjustable coefficient; the aggregated message of node i. The sum of all neighbor messages based on attention weights;

[0048] Step 2022: Perform dynamic calculations based on the joint type, specifically based on the quasi-static assumptions, and incorporate the Coulomb friction model to calculate the forces and torques at the joints;

[0049] Based on Newton's second law and rotational law, the calculated nodal forces and torques are used to calculate the linear acceleration of node i. and angular acceleration ;

[0050] Then, the linear acceleration of node i is obtained through a lightweight multilayer perceptron. and angular acceleration Angular velocity of components calculated from multi-view RGB-D time-series images or videos. and linear velocity These four physical quantities are encoded into a fixed-dimensional feature vector. ;

[0051] The final hidden state of the node is updated through a gated recurrent unit, which uses the current hidden state, aggregated message, and feature vector as input. For input.

[0052] Furthermore, step 203 is specifically as follows:

[0053] The attitude energy function formula is as follows:

[0054]

[0055] in, Let be the attitude energy function. For the relationship diagram, The parameters represent the joint between components i and j, and task represents the task. Measuring posture Consistency with the task description;

[0056] Assume that the probability distribution of the attitude distribution follows a Boltzmann distribution, that is:

[0057]

[0058] in, Representing the conditional probability distribution; performing a second-order Taylor expansion at the minimum of the attitude energy function to approximate the attitude distribution of each component as an optimal attitude. The covariance matrix of the Gaussian distribution in the tangent space centered at the center is... Defined in optimal pose tangent space The formula is as follows:

[0059]

[0060] in, Indicates in Gaussian distribution on manifold, mean Located on a manifold, covariance Tangent space defined at the mean point In the middle, belonging to real matrix,

[0061] Output the prior orientation distribution parameters for each component i. This serves as a strong conditional prior for the conditional diffusion model in step 3.

[0062] Furthermore, the physical regularization loss specifically includes four physical constraints, each of which has its weight adjusted by a weighting coefficient.

[0063] Penetration penalty :

[0064]

[0065] in, For point The component to which it belongs, This represents the total number of points in the set. for The activation function, signed distance (SDF), is an implicit geometric representation. For any point p in space, SDF returns the signed distance from that point to the surface of an object. Point The value in the SDF field of component j;

[0066] Static stability penalty :

[0067]

[0068] in, For a small threshold, based on the predicted component quality Calculate the overall center of gravity based on the component positions, estimate the supporting polygon, and construct a two-dimensional convex hull from the projections of the contact points onto the horizontal plane. For the horizontal projection of the center of gravity to The distance to the boundary of the convex hull;

[0069] Exercise energy penalty :

[0070]

[0071] in, Let i be the height of the center of gravity of component i relative to the reference plane. It is the gravitational constant;

[0072] Joint restraint punishment :

[0073]

[0074] in, This represents the set of edges in the component relationship diagram. Indicates the total number of joints. It is the joint displacement calculated relative to the attitude. , These represent the lower and upper limits of joint displacement.

[0075] The beneficial effects of this invention are as follows: For the first time, physical depth is embedded into the pose generation framework, ensuring the diversity of generated poses while strictly adhering to physical constraints. By constructing and refining component joint graphs, an understanding and modeling of the internal structure of objects is achieved. The strong pose priors derived using physical enhancement graph reasoning significantly improve the generation efficiency and quality on the SE(3) manifold, resulting in physically reasonable generated poses and diverse results. Attached Figure Description

[0076] Figure 1 This is a diagram showing the overall architecture of the system of the present invention.

[0077] Figure 2 This is a diagram of the implicit reconstruction structure of component-based physical perception neural networks.

[0078] Figure 3 This is a schematic diagram illustrating the refinement of the component relationship diagram and the prior inference of attitude.

[0079] Figure 4 This is a schematic diagram of physically guided diffusion attitude optimization.

[0080] Figure 5 A schematic diagram of the framework for optimizing the attitude of a two-step diffusion guided by physics. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0082] A method for generating poses of articulated objects based on physical perception and graph diffusion, such as Figure 1 As shown, it includes the following steps:

[0083] Step 1, Implicit Neural Reconstruction of Component-Based Physical Perception. This step constructs a dual-branch joint representation network to reconstruct the geometry and physical properties of the components of an articulated object from multi-view RGB-D image sequences / video data, initializes the joint connections between components, and outputs the initialization parameters of the relationship graph between components.

[0084] Step 101: Construct a dual-branch joint representation network. For example... Figure 2 As shown, this step first encodes the input information into high-dimensional features and fuses them through a shared encoder. Then, a gating mechanism is used to decouple the fused features into two independent branches: geometry and physics.

[0085] Step 1011, Multi-source information encoding and fusion. This step encodes the input image data in three ways: position, orientation, and pose, to extract high-dimensional features that the network can process.

[0086] Position encoding is based on the 3D point cloud of the target object surface obtained by depth back projection of the input image. The coordinates of any point in the 3D point cloud in the camera coordinate system are denoted as... By using sine and cosine functions of different frequencies to simulate Fourier features, high-dimensional positional features are generated to help multilayer perceptrons (MLPs) learn high-frequency geometric details. The encoding formula is shown below:

[0087]

[0088] in, Point Location coding features It is the frequency of position encoding, which controls the frequency of geometric details. This indicates the encoder.

[0089] Directional encoding is based on the camera's gaze direction *d* corresponding to the input image. *d* is a normalized gaze vector pointing from the camera's optical center to an image pixel, representing the viewpoint observed at that 3D point. A sine-cosine function is used to encode *d*, generating high-dimensional directional features to capture viewpoint-dependent color and appearance variations. The encoding formula is shown below:

[0090]

[0091] in, Encoding features for the observation direction d, It is the frequency of directional encoding, which controls the frequency of color changes.

[0092] Pose encoding generates a set of feature vectors representing the relative spatial relationships between components by obtaining the root node and initial 6D pose data of each component from the input image. The root node refers to the rigid body (such as cabinet or base) identified as the motion reference of the entire object in the image, and its pose serves as the global reference origin; the component refers to the rigid parts segmented from the image that constitute the jointed object. The initial 6D pose is obtained by directly inferring from the input image by a trained lightweight convolutional-transformer hybrid network (CNN-Transformer). This network outputs the 3D rotation (represented as SO(3) matrix, i.e., the standard mathematical representation of the three-dimensional rotation group) and 3D translation (represented as a three-dimensional vector, i.e., the spatial position of the origin of the local coordinate system of the component in the camera coordinate system) of each component in the camera coordinate system.

[0093] Specifically, the translation component of the root node is read from the initial 6D pose estimated by the root node to obtain the translation vector of the root node in the camera coordinate system. Meanwhile, for each component i, the estimated initial 6D pose defines the homogeneous transformation matrix from the component's local coordinate system to the camera coordinate system. This matrix consists of the output 3D rotation matrix and 3D translation vector. The root node translation vector is then used... Projecting onto the local coordinate system of each component, its local representation is calculated to characterize the relative spatial relationships between components. The formula for this process is shown below:

[0094]

[0095] in, attitude parameters The coding features, for homogeneous coordinate form, Let represent the homogeneous transformation matrix from the local coordinate system to the world coordinate system of the i-th component. This indicates that the target object has a total of Each component.

[0096] The feature vectors encoded by position, orientation, and pose are concatenated and input into the multilayer perceptron encoder to generate shared features that integrate information from multiple sources. The formula is shown below:

[0097]

[0098] The Encoder is a feedforward neural network with eight fully connected layers that generates a highly abstract shared feature by incorporating information from multiple sources through deep nonlinear transformations. .

[0099] Step 1012, Feature Separation Based on Gating Mechanism. A learnable gating decomposition function is used to separate shared features. The features are broken down into geometric and physical characteristics, as shown in the following formula:

[0100]

[0101] in, Representing geometric features for shape reconstruction. These are physical characteristics used for estimating physical properties. The function is a learnable gating mechanism that splits shared features into two independent vectors, decoupling geometric and physical information. This function achieves feature selection by generating complementary gating vectors for the two branches. The gating vector generation formula is shown below:

[0102]

[0103]

[0104]

[0105] in, , For learnable geometric weights and biases, , For learnable physical weights and biases, Represents a geometrically gated vector. This represents the physical gating vector. Branch features are obtained by element-wise multiplication of the gating vector and the shared features; the specific formula for splitting is as follows:

[0106]

[0107]

[0108] in, This represents element-wise multiplication (Hadamard product).

[0109] Step 102, detailed output of the geometry branch and the physics branch.

[0110] Step 1021, Branch 1 is the geometry branch, which outputs the global geometry code and component-level geometry vector.

[0111] (1) For each component i, a separate MLP is used to extract component-level geometric features:

[0112] in, Let i represent the geometric feature vector of the i-th component. This represents a parameter-independent geometric feature extraction network unique to the i-th component.

[0113] Through a linear transformation layer, from Simultaneously predict query points The SDF (signed distance) value and its associated logical value, where This represents the coordinates of a 3D point in the local coordinate system of the component, i.e. , This represents the coordinates of a 3D point in the camera coordinate system.

[0114]

[0115] in, This represents a linear transformation layer dedicated to component i, which achieves feature decoding through linear weighting and bias superposition. For point The signed distance value in the local coordinates of component i. For point The unnormalized logarithmic probability of belonging to component i. Normalize the logarithmic values ​​of all components using softmax normalization to obtain the point. The probability of belonging to component i is calculated by using a soft minimum mixing method to smoothly mix the SDF of each component into the overall SDF of the scene.

[0116] (2) By aggregating and refining the geometric feature vectors of all components, a global geometric code representing the overall shape and structure of the object is generated. The formula is shown below:

[0117]

[0118] Specifically, The geometric feature vectors of each component are stacked into a matrix, and max pooling is performed on this matrix along the component dimensions. The global geometric feature vector is obtained. Through a fully connected layer Will Mapped to a fixed-dimensional global geometric code .

[0119] The global geometry encoding By integrating the shape information of all components, the overall structure, component composition, and topological relationships of the object are characterized, providing high-level geometric priors for subsequent steps.

[0120] Step 1022, Branch 2 is the physical branch, which outputs the global physical code, component-level physical attribute vector and initialized joint parameters.

[0121] (1) A global encoding describing the overall physical properties is obtained through an MLP and a linear layer, as shown in the following formula:

[0122]

[0123] in, This represents the physical features obtained by separating them from the shared features. It is a global physical feature extractor. Represents global physical encoding. This means that physical features are linearly mapped to the physical coding space.

[0124] (2) For each component i, a component-level physical property vector is predicted using an independent MLP. The formula is shown below:

[0125]

[0126] in, The physical attribute vector of component i, This is an independent physical feature extractor for component i. The vector encodes the component quality. Inertial tensor Stiffness coefficient of friction All physical parameters are learnable parameters.

[0127] (3) To construct the component relationship diagram, it is necessary to initialize the connection relationships between components. Based on the component physical properties Through a lightweight joint parameter encoder Predict the parameters of joint i and the joint k that may be connected to its parent component j. The output includes complete parameters such as axial direction, joint type, pitch (for helical joints only), and range of motion, as shown in the formula below:

[0128]

[0129]

[0130] in, It is a unit vector along the joint axis; It refers to joint types, including rotational, translational, fixed, and helical joints, i.e. ; This is a pitch parameter, effective only for helical joints; range of motion Indicates the permissible range of joint movement. Represents the set of real numbers. As the lower limit, This is the upper limit. Specifically, for rotary joints, this range constrains the range about the axis. The relative rotation angle; for translational joints, the constraint along the axis The relative linear displacement; for a fixed joint, the range is... This indicates that relative motion is not allowed; for helical joints, the constraint is a combined rotational and translational motion coupled by pitch.

[0131] Step 103, output the initialization parameters of the component relationship diagram, including:

[0132] (1) Node attributes, i.e. component attributes, for each component i, output local SDF network parameters. Physical property vector Local coordinate system transformation .

[0133] (2) Edge candidate set, i.e., joint candidate connection set. Based on the spatial proximity of component bounding boxes. (i.e., the Euclidean distance between the center points of the bounding boxes of components i and j) generates a set of candidate connection edges, when If the value is less than a preset threshold, components i and j are considered adjacent and may be connected via a joint. Each edge... The formula represents the relative motion constraint between parts i and j, i.e., the initial parameters of joint k between parts i and j, as shown below:

[0134]

[0135] (3) Global geometric coding Global physical coding .

[0136] Step 2, diffusion of the hierarchical component relationship graph of physical enhancement. For example... Figure 3 As shown, this step refines the initial component relationship graph output from step 1 and uses a physically enhanced graph diffusion model to infer the prior attitude distribution of each component under task guidance. The input is the graph initialization parameters from step 1, and the output is the attitude probability distribution of each component, which serves as a strong constraint for the subsequent generation process.

[0137] Step 201, Graph Construction and Joint Parameter Refinement. This step, based on the edge candidate set and component attributes output from Step 1, filters out spurious links through kinematic fitting and temporal consistency testing, determines precise joint parameters, and constructs a deterministic component relationship graph with confidence.

[0138] Step 2011, joint parameter estimation based on kinematic fitting. For each candidate edge... Using multi-view time-series observation data, (H represents the component pose), by optimizing the joint constraint energy function To accurately estimate its joint parameters, the formula is as follows:

[0139]

[0140] in, This indicates the pose of component i. This indicates the pose of component j. Let K be the joint parameter. For accurate estimation of joint parameters, Based on joint type definition:

[0141]

[0142] in, These are the rotation matrices for components i and j, respectively. It is a three-dimensional rotation group. for up to its Lie algebra logarithmic mapping ( yes The corresponding Lie algebra describes a three-dimensional vector space undergoing rotational changes. Indicates the positions of components i and j. The desired relative rotation vector (obtained by fitting the observed data). This represents a unit vector perpendicular to the joint axis. The amount, The pitch parameter of the helical joint.

[0143] Step 2012, Motion Consistency Test and Confidence Assessment. Using multi-view time-series observation data, the motion consistency based on joint parameters is tested. The defined kinematic relationship is subjected to motion consistency testing and confidence assessment. Based on the joint parameters, the theoretical relative motion that components i and j should exhibit at each time step is calculated. At the same time, the measured relative motion at the corresponding moment is directly extracted from the actual observation data. The specific form of the relative motion is determined according to the joint type: for rotary joints it is the relative rotation angle, for translational joints it is the axial linear displacement, and for helical joints it is the generalized displacement of rotation-translation coupling. Its specific definition is consistent with the motion constraints of the corresponding joint type in step 2011.

[0144] The confidence score of the candidate connection is calculated by comparing the difference between the theoretical prediction and the actual observation. The formula is shown below:

[0145]

[0146] in, This represents the time step count, i.e., the number of frames in the multi-view time-series data. This represents the relative motion predicted by the candidate joint model. This represents the relative motion obtained from actual measurements. This is a scaling parameter used to control the sensitivity to errors.

[0147] Set threshold ,when If so, it is judged as a fake link and filtered out.

[0148] Step 2013: Determine the output of the refined component relationship diagram. Where V represents the set of all parts of an object, E represents the set of connections between parts, and nodes Includes component codes Physical attribute encoding Local coordinate system ;side It contains precise joint parameters. and confidence level .

[0149] Step 202, Physically Enhanced Graph Diffusion and Node State Update. This step performs physically enhanced message passing on the graph structure, simulating the transmission of forces and motions between components, thereby updating the feature vectors representing the component states.

[0150] Step 2021: Generation and aggregation of physically sensed messages.

[0151] Hidden state of each node (component) It is a fixed-dimensional vector that passes through a recurrent neural network encoder ( Maintenance and updates, encoding the following information for the component:

[0152]

[0153] in, The orientation of component i on SE(3) Functions that map to vectors The angular velocity and linear velocity of the component are calculated from multi-view RGB-D time-series images or videos. The component physical property vector from step 1, This is an optional external force.

[0154] Define message function This function comprehensively considers the hidden state of component nodes, joint parameters, and external forces, mapping the above information to a message vector. The formula is shown below:

[0155]

[0156] in, The joint parameters are accurately estimated between components i and j.

[0157] Interaction forces Based on the physical properties of the nodes, their relative motion state, and external forces The calculation yields the following specific form:

[0158]

[0159] in, It is a learnable force mapping function that maps nodal states and external forces into inter-node forces and nodal angular velocities. and linear velocity It was calculated using multi-view time-series observation data.

[0160] Message aggregation employs a physically-aware graph attention mechanism, with attention weights... It is determined by both node state similarity and connection stiffness, as shown in the following formula:

[0161]

[0162] in, For attention weights, Let k be the set of neighboring nodes of node i, where k is the index of a neighboring node. Let k be the hidden state of neighbor node k. The function represents the similarity between the hidden states of nodes i and j. The function represents the similarity between the hidden states of node i and its neighbor node k. This represents the estimate of the connection stiffness between the hidden states of nodes i and j. This represents the estimation of the connection stiffness between the hidden states of node i and its neighbor node k. This is an adjustable coefficient.

[0163] The message aggregated by node i The weighted sum of all neighbor messages is calculated using the following formula:

[0164]

[0165] Step 2022, Node State Update Based on Simplified Dynamics. This step updates the node states using aggregated messages and a differentiable simplified Newton-Euler dynamics module. This step is based on quasi-static assumptions and incorporates a Coulomb friction model (friction coefficient). (Predicted from step 1). Dynamic calculations are performed based on joint type.

[0166] Step 20221: Calculate the forces and torques acting on the nodes. For each node i, calculate the net force and net torque acting on that node based on the type of joint it is connected to.

[0167] For a rotary joint, the net torque acting on component i is calculated using the following formula:

[0168]

[0169] in, This represents the set of neighboring components of component i. Indexing neighboring components, Indicates frictional torque. Represents the lever arm vector (from the rotation center of node i to the force). (vector of the point of application) ,in, The relative angular velocity is calculated from the nodal angular velocity. The normal force is in the direction of the joint axis. For the effective friction radius, For symbolic functions, This is a unit vector along the joint axis (to ensure the friction torque direction is correct).

[0170] For translational joints, calculate the net force. That is, the sum of all forces acting on component i, including joint-transmitted forces and frictional forces:

[0171]

[0172] Among them, friction The calculation formula is ,in, This is the relative linear velocity, calculated from the nodal linear velocity.

[0173] For a helical joint, the total frictional force The weighted sum of rotational friction and translational friction is given by the following formula:

[0174]

[0175] in, This is a weighted sum of rotational and translational friction. This represents the translation amount per radian. Due to the helical joint, net force and net torque need to be updated simultaneously; net force is updated (added to the axial force). Net torque is .

[0176] To avoid The discontinuity of the function at zero velocity is approximated using a smoothing method. , where A is a large constant that controls the approximate sharpness.

[0177] Step 20222: Calculate acceleration and angular acceleration. Calculate the linear acceleration at node i according to Newton's second law and rotational law. and angular acceleration The formula is shown below:

[0178]

[0179]

[0180] in, This represents the net force acting on node i. For the mass of component i, Let i be the inertia tensor of component i, and both are derived from the physical property vector predicted in step 1. Decoded from the middle, The inverse matrix of the inertia tensor is used to... Convert to angular acceleration This is the net torque acting on component i.

[0181] Step 20223, Encoding state change characteristics The calculated changes in motion are encoded into a fixed-dimensional feature vector. This is used to update the hidden state of a node, and the formula is shown below:

[0182] in, It is a lightweight multilayer perceptron used to map physical quantities to a hidden state space.

[0183] Step 20224: Update the node hidden state. The final node hidden state is updated through a gated recurrent unit (GRU), which takes the current hidden state, the aggregated message, and the state change calculated by dynamics as input:

[0184]

[0185] in, This indicates the number of message passing layers in a graph neural network. Indicates the first The hidden state of node i in layer 1 Indicates the first The hidden state of node i in the layer.

[0186] Step 203, Task-Driven Attitude Distribution Inference. Given a high-level task description, this step infers the physically reasonable attitude probability distribution of each component based on the updated graph node states, ensuring that the attitude distribution meets the task requirements.

[0187] Step 2031, Construct the attitude energy model. Define an attitude energy function. This function incorporates joint constraint energy. Energy matching the task The formula is shown below:

[0188] in, As defined in Step 2011, this measures the degree to which posture satisfies joint constraints. This represents the parameters of the joint between components i and j. Measuring posture Consistent with the task description, "task" refers to the task itself, i.e., a semantic description, such as "open the drawer." The task text is encoded into latent space vectors using a pre-trained vision-language model (VLM), and then compared with the pose... Calculate the distance of the encoded features:

[0189]

[0190] in, This is an attitude encoder.

[0191] Step 2032, Attitude Probability Distribution Prediction. Assume the attitude probability distribution follows a Boltzmann distribution, meaning the attitude probability distribution is inversely proportional to its energy:

[0192]

[0193] in, This represents the conditional probability distribution; for ease of calculation and subsequent use, it is defined at the minimum value of the energy function (optimal posture). A second-order Taylor expansion is performed at ( ), approximating the attitude distribution of each component as a product of ( ). The covariance matrix of the Gaussian distribution in the tangent space centered at the center is... Defined in tangent space The formula is as follows:

[0194]

[0195] in, Indicates in Gaussian distribution on manifold, mean Located on a manifold, covariance Tangent space defined at the mean point In the middle, belonging to The real matrix describes the range of attitude fluctuations around the optimal value.

[0196] Step 2033, Output the attitude prior distribution. Output the attitude prior distribution parameters for each component i. This serves as a strong conditional prior for the conditional diffusion model in step 3.

[0197] Step 3: Construction of a conditional diffusion model on the manifold. This step uses the geometric and physical property information reconstructed in Step 1 and the prior distribution of component poses inferred in Step 2 as conditions to construct a conditional diffusion probability model on the SE(3) manifold. Through a physically guided two-step inverse sampling framework, the poses are gradually recovered and optimized from the noise, ultimately generating a diverse and physically reasonable set of jointed object pose assumptions.

[0198] The tangent space of the SE(3) manifold This corresponds to a "local linear approximation" of a point (a specific pose) on the manifold, and is often used to describe instantaneous motion quantities such as angular velocity and linear velocity. The noise addition and gradient update operations in this step... The motion is performed in the first step and then projected back to the SE(3) manifold through exponential mapping to ensure that the generated posture is always a valid rigid body motion.

[0199] Step 301, define the diffusion process on the SE(3) manifold. For example... Figure 4 As shown, a denoising diffusion probability model is defined on the SE(3) special Euclidean group manifold, which includes two processes: forward diffusion and reverse denoising.

[0200] The forward diffusion process begins from a clean component orientation. Starting from its corresponding Lie algebra Gaussian noise is gradually added to the tangent space and then subjected to exponential mapping. The projected manifold gradually perturbs the data distribution into an isotropic Gaussian distribution. After... After the step, the attitude information is completely destroyed into random noise. .

[0201] The reverse denoising process, from noise Starting with multimodal conditional information, a parameterized denoising network is learned to progressively predict and remove noise, recovering a clean pose that conforms to physical and geometric constraints. .

[0202] Step 302, Design of a scoring network for multimodal conditional fusion and physical regularization. This step designs a conditional denoising network (i.e., a scoring network) in the inverse process. Its core function is to predict the gradient direction (score) that should move toward "cleaner data" given the noise pose, diffusion step number and multimodal conditions at step t during the reverse denoising process.

[0203] Step 3021, Multimodal Conditional Fusion. The conditional information is fused from multiple sources in the preceding steps, specifically including: global physical coding. Global geometric coding and task condition feature vector ,in, The feature vector obtained by the pre-trained text encoder (CLIP) for the task. .

[0204] A cross-modal attention mechanism is used for dynamic condition fusion to integrate the current noise pose. The feature encoding is used as the query Q. The above condition information is concatenated to form the key K and value V, and the fused condition vector is calculated. The formula is shown below:

[0205]

[0206] in, and For learnable query projection networks and key-value projection networks, Attention mechanism; fused conditional vector Provide dynamic guidance for each step of denoising to ensure that the generated poses meet the requirements of dynamic constraints and scene interaction.

[0207] Step 3022, Physically Regularized Network Training. During the training phase, physical priors are injected into the denoising network (scoring network) of the diffusion model, so that when predicting noise, it not only considers the data distribution, but also tends to generate gradient directions under physical rules.

[0208] Score Network The U-Net architecture is used, with the input being the Lie algebra coordinates of the current noisy pose. Diffusion Step Embedding and fusion condition vector The output is an estimate of the gradient of the clean data (i.e., the predicted noise residual).

[0209] To guide the scoring network Learning physical laws and denoising loss in the standard diffusion model Based on this, a differentiable physical regularization loss is introduced. The formula for the total training loss is:

[0210]

[0211] in, It includes the following four physical constraints, These are the weighting coefficients.

[0212] (1) Penetration penalty Encourage no-penetration between components. For a batch of generated poses, randomly sample a set of surface points. The penetration depth of the component in the SDF field is calculated using the following formula:

[0213]

[0214] in, For point The component to which it belongs, This represents the total number of points in the set. for The activation function, signed distance (SDF), is an implicit geometric representation. For any point p in space, SDF returns the signed distance from that point to the surface of an object. Point The value in the SDF field of component j;

[0215] (2) Static stability penalty Encourage stable equilibrium. Based on predicted component mass. With component position Calculate the overall center of gravity. Estimate the supporting polygon, which is a two-dimensional convex hull formed by the projections of the contact points onto the horizontal plane. Penalize the horizontal projection of the center of gravity onto... Distance to the boundary of the convex hull The formula is shown below:

[0216] in, For a small threshold, the operation involves minimal uncertainty.

[0217] (3) Exercise energy penalty This encourages lower energy states (more static, more stable). The approximate potential energy of the system is calculated using the following formula:

[0218]

[0219] in, Let i be the height of the center of gravity of component i relative to the reference plane. is the gravitational constant.

[0220] (4) Joint restraint punishment To ensure that joint movement is within a reasonable range, calculate the displacement of each joint. Exceeding the specified range The degree. The formula is as follows:

[0221] in, This represents the set of edges, i.e., the set of joints, in the component relationship diagram. Indicates the total number of joints. It is the joint displacement calculated relative to the attitude.

[0222] The calculations of the four physical constraints mentioned above together constitute a differentiable physics calculation module, which is used for calculations during the training process. The gradient is used to guide the scoring network to learn physical laws.

[0223] Through joint optimization and Score network Trained to not only learn the data distribution but also internalize physical laws, its predicted gradient direction will tend to point to the physically more plausible region. After training, The internally integrated differentiable physics calculation module will be used for reverse sampling in step 303.

[0224] Step 303, physically guided two-step inverse sampling and attitude optimization. For example... Figure 5 As shown, this step performs inverse sampling, employing a two-step iterative framework of "score-driven - physics correction," which will extract from... arrive The generation trajectory is constrained within a physically reasonable manifold region.

[0225] Step 3031, score-driven prediction step. In the tangent space, a scoring network based on the learned value. Perform Langevin dynamics updates from noise Preliminary noise reduction proposals were obtained. .

[0226]

[0227] in, The noise attitude at step t represents the high-dimensional vector formed by concatenating the Lie algebra coordinates corresponding to the attitudes of all components at step t in the reverse diffusion process. For scoring networks, For fusion condition vectors, The update step size for step t is calculated based on a predefined noise scheduling scheme. It is a random noise vector.

[0228] Step 3032, Physics-guided correction step. The predicted Lie algebra coordinates are... The proposed pose on SE(3) is obtained through exponential mapping. Subsequently, the calculation method used in step 3022 is invoked. The differentiable physics computation module directly calculates the degree of physical violation under the proposed attitude and obtains the physical correction gradient through backpropagation. The prediction results are then corrected in the tangent space, using the following formula:

[0229]

[0230] in, It is the correction step size, which is a hyperparameter.

[0231] Step 3033, Manifold Projection and Iteration. The corrected noise pose is then... Projecting back to the SE(3) manifold via an exponential mapping, To obtain the noise attitude for the next step. This ensures that the iteration process always takes place within a legal rigid body motion space. Repeat steps 3031-3033 until t=0.

[0232] Step 304: Generation and evaluation of multimodal pose hypotheses. A diverse set of candidate poses is generated by running multiple sampling chains in parallel, and these poses are evaluated, filtered, and then output.

[0233] Step 3041, Diverse Trajectory Generation. From Different random noise seeds Starting from step 303, the physical guided inverse sampling process is executed (total... step, The total number of diffusion steps defined in step 301 (a predefined hyperparameter) generates N independent sampling trajectories, ultimately yielding N candidate pose hypotheses. In which each candidate pose Each of these is a complete object pose configuration, represented as a collection of 6D poses of all the object's parts.

[0234] Step 3042, Comprehensive Physical Evaluation Pre-screening. This involves pre-screening from the generated multiple candidate poses. The optimal set is selected from the data, and a refined offline physical evaluation is performed on each posture. This evaluation uses a scoring system, with each score normalized to the range [0,10], and a higher score indicates better physical plausibility.

[0235] (1) Collision score Accurately assess penetration between components. Dense and uniform sampling is performed on the surface of each component to obtain a point set. The approximate penetration volume for all cross-part point pairs is calculated using the following formula:

[0236]

[0237] in, This is the scaling factor. For the set of sampling points, This represents the total number of sampling points for all components. Point The signed distance to the surface of component j; a negative value indicates that the point is inside the component. The score ranges from 0 to 10, with lower scores for more severe penetration.

[0238] (2) Stable score A rigorous assessment of anti-tipping capability was conducted. The stability score of the attitude was calculated based on classical static stability criteria. This was determined through component mass... and component posture The translation component (i.e., the position of the component's centroid in the world coordinate system) is used to calculate the total centroid. All component surface points in contact with the "ground" (or a known support surface) are found and projected onto a horizontal plane. The convex hull is then calculated. The centroid is projected onto the same horizontal plane, and the minimum signed distance from the horizontal projection of the centroid to the edge of the support polygon is calculated. (The interior of the polygon is positive), the formula is as follows:

[0239] in, This is the safety margin that is set.

[0240] (3) Energy score Assess the "naturalness" of the posture. Based on the physical system's tendency to be in a low-energy state, and encouraging low-energy states, the formula for calculating the total potential energy is as follows:

[0241]

[0242] in, Let be the height of the component's center of gravity relative to the reference plane. The energy score is calculated using potential energy, as shown in the following formula:

[0243]

[0244] in, This is the scaling factor.

[0245] (4) Joint compliance score Check the joint limits. For all For each joint, calculate its current displacement q, which exceeds the limit. The degree to which the joint exceeds its limit. , which represents the absolute amount by which the current displacement of joint i exceeds the allowable range.

[0246]

[0247] in, For all joint sets, This represents the total number of joints. This is the scaling factor. The square of the out-of-bounds value is used to amplify the penalty effect of severe out-of-bounds errors, while ensuring the differentiability of the function.

[0248] (5) Based on the combined scores and screening, the weighted sum is used to calculate the total score for overall rationality, as shown in the following formula:

[0249]

[0250] in, , , and These are the scoring weights. The principles for setting these weights include task importance, physical salience, etc., and the scoring weights... It can adaptively adjust according to the task type. For example, collision weight is high in grasping tasks, stability weight is high in stacking tasks, energy weight is high in dynamic tasks, and joint compliance weight is the highest in precision tasks.

[0251] Step 3043, final pose set output. All candidate poses are sorted according to the comprehensive score, and hypotheses with scores higher than a threshold are selected. The top M poses with the highest scores are output, constructing the final diverse and physically reasonable set of pose hypotheses for the jointed object. Each output contains complete 6D pose parameters of the part and its evaluation score.

[0252] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for generating pose of articulated objects based on physical perception and graph diffusion, characterized in that, Includes the following steps: Step 1: Construct a dual-branch joint representation network to reconstruct the geometry and physical properties of the jointed object's components from the image data, initialize the joint connection relationships between the components, and output the initialization parameters of the relationship graph between the components; The dual-branch joint representation network includes a geometric branch and a physical branch; Geometric Branch: Based on the extracted geometric features, firstly, an independent geometric feature extraction network is used to extract component-level geometric features for each component. All component-level geometric feature vectors are stacked into a matrix, and max pooling is performed on the matrix along the component dimension. Then, a fully connected layer is used to obtain a fixed-dimensional global geometric encoding. And through a linear transformation layer, the signed distance value and its attribution logical value of the query point are predicted simultaneously from the component-level geometric features; The physics branch: Based on the extracted physical features, for each component i, a component-level physical attribute vector is predicted using an independent physical feature extractor; simultaneously, a global physical encoding describing the overall physical attributes is obtained through a global physical feature extractor and a linear layer; based on the component-level physical attribute vector, a lightweight joint parameter encoder is used to predict the joint parameters of joint i and the joint k connecting it to its parent component. The output includes complete parameters such as axial direction, joint type, pitch, and range of motion. Step 2: Physically augmented hierarchical component relationship graph diffusion. The relationship graph initialization parameters are input into the physically augmented graph diffusion model inference to infer the attitude probability distribution of each component under task guidance. Step 201: Optimize the joint constraint energy function according to the joint type definition to accurately estimate the joint parameters; then use multi-view time series observation data to perform motion consistency test and confidence assessment on the kinematic relationship defined by the accurately estimated joint parameters, and then output a deterministic part relationship diagram with accurate joint parameters and confidence. Step 202: Define the hidden state and message function of the node based on the component pose and physical information. Then, calculate the attention weight of the node using a physically-aware graph attention mechanism. Perform weighted aggregation based on the messages of neighboring nodes and attention weights to obtain the aggregated message of the current node. Calculate the state change using a differentiable simplified Newton-Euler dynamics module. The dynamics calculation is performed according to the joint type. Finally, input the current hidden state, the aggregated message, and the state change obtained from the dynamics calculation into the gated loop unit to update the node's hidden state. Step 203: First, define the attitude energy function to measure the degree to which the attitude satisfies joint constraints and the consistency between the attitude and the task description; then, predict the attitude probability distribution at the minimum value of the attitude energy function, approximating the attitude distribution of each component as... Gaussian distribution on the manifold outputs the prior orientation distribution parameters for each component; Step 3: Based on the geometry and physical properties of the component and the probability distribution of the pose, construct a conditional diffusion probability model on the SE(3) manifold; adopt a physically guided two-step inverse sampling framework to gradually recover and optimize the pose from the noise, and finally generate a set of diverse and physically reasonable pose assumptions for articulated objects. Step 301: The conditional diffusion probability model includes two processes: forward diffusion and inverse denoising. The forward diffusion process is triggered by the component's attitude and is denoised through the corresponding Lie algebra. Gaussian noise is gradually added to the tangent space and projected back to the manifold via exponential mapping, completely destroying the attitude information into random noise; the inverse denoising process starts from the random noise, guided by multimodal conditional information, and learns a parameterized denoising network to gradually predict and remove noise, thus restoring the component attitude that conforms to physical and geometric constraints. Step 302: The task obtains the task conditional feature vector through the pre-trained text encoder. Then, the global physical encoding, global geometric encoding, and task conditional feature vector are concatenated. Based on the cross-modal attention mechanism, the current noise pose and the concatenated features are dynamically fused to obtain the fused conditional vector. A scoring network for the inverse denoising process is constructed. Given the noise pose at step t, the number of diffusion steps, and the fused conditional vector, inverse denoising is performed. During training, the standard diffusion model denoising loss and the physical regularization loss based on physical laws are included. The physical regularization loss includes penetration penalty, static stability penalty, motion energy penalty, and joint constraint penalty. Step 303: In In the tangent space, the Langevin dynamics update is performed based on the learned score network to obtain the initial denoising proposal from the noise, and then the proposed pose on SE(3) is obtained through exponential mapping; the physical violation degree under the proposed pose is calculated based on the physical regularization loss, and the physical correction gradient is obtained by backpropagation. The prediction result is corrected in the tangent space, and this step is repeated until the iteration stopping condition is reached. Step 304: From Starting from different random noise types, run step 303 to generate N independent sampling trajectories, and finally obtain N candidate posture hypotheses; perform offline physical evaluation on each candidate posture, including evaluating the collision score of penetrating between components, the stability score of anti-tipping capability, energy score, and joint compliance score; sort all candidate postures according to the evaluation results, and output the top M postures with the highest scores.

2. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 1, characterized in that, Step 1 is described in detail as follows: Step 101: Encode the input image data in three ways: position, orientation, and pose. Then, concatenate the three encoding results to obtain shared features. Then, use a learnable gating splitting function to split the shared features into geometric features and physical features. Step 102: Construct a dual-branch joint representation network, including a geometric branch and a physical branch; Step 103: Output the initialization parameters of the component relationship graph, including node attributes, edge candidate set, global geometric encoding, and global physical encoding; The node attributes are component attributes, including signed distance values, component-level physical attribute vectors, and local coordinate system transformations; each edge in the edge candidate set includes joint parameters.

3. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 2, characterized in that, The joint parameters are expressed as follows: ,in, It is a unit vector along the joint axis; It refers to joint types, including rotational, translational, fixed, and spiral joints. This is a pitch parameter, effective only for helical joints; range of motion Indicates the permissible range of joint movement. Represents the set of real numbers. As the lower limit, This is the upper limit.

4. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 3, characterized in that, Step 201 is as follows: Step 2011: For each candidate edge Using multi-view time-series observation data , This indicates the pose of component i. The pose of component j is represented by the joint constraint energy function optimized. To accurately estimate its joint parameters, the formula is as follows: ， in, Let K be the joint parameter. For accurate estimation of joint parameters, Based on joint type definition: ， in, These are the rotation matrices for components i and j, respectively. It is a three-dimensional rotation group. for up to its Lie algebra logarithmic mapping, Indicates the positions of components i and j. Let be the desired relative rotation vector. This represents a unit vector perpendicular to the joint axis. The amount; Step 2012: Using multi-view time-series observation data, analyze the joint parameters based on accurate estimation. The defined kinematic relationship is subjected to motion consistency testing and confidence assessment; based on the joint parameters, the theoretical relative motion that components i and j should exhibit at each time step is calculated. At the same time, the measured relative motion at the corresponding moment is directly extracted from the actual observation data. The specific form of the relative motion is determined according to the joint type: for rotary joints it is the relative rotation angle, for translational joints it is the axial linear displacement, and for helical joints it is the generalized displacement of rotation-translation coupling. Its specific definition is consistent with the motion constraints of the corresponding joint type in step 2011. The confidence score of the candidate connection is calculated by comparing the difference between the theoretical prediction and the actual observation. The formula is shown below: ， in, Indicates the number of time steps. For scale parameters; set threshold ,when If so, it is judged as a fake link and filtered out; Step 2013: Determine the output of the refined component relationship diagram; output the refined component relationship diagram, where nodes contain the component code, signed distance value, physical attribute vector, and local coordinate system; edges contain precise joint parameters. and confidence level .

5. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 4, characterized in that, Step 202 is as follows: Step 2021: Hidden state of each node Through a recurrent neural network encoder Maintenance and updates: ， in, The orientation of component i on SE(3) Functions that map to vectors The angular velocity and linear velocity of the component are calculated from multi-view RGB-D time-series images or videos. This is a vector of physical properties of the component. External force; via message function Map the hidden states and joint parameters of component nodes i and j to the message vector using external forces. ; Message aggregation employs a physically-aware graph attention mechanism, with attention weights... It is determined by both node state similarity and connection stiffness, as shown in the following formula: ， in, For attention weights, Let k be the set of neighboring nodes of node i, where k is the index of a neighboring node. Let k be the hidden state of neighbor node k. The function represents the similarity between the hidden states of nodes i and j. The function represents the similarity between the hidden states of node i and its neighbor node k. This represents the estimate of the connection stiffness between the hidden states of nodes i and j. This represents the estimation of the connection stiffness between the hidden states of node i and its neighbor node k. An adjustable coefficient; the aggregated message of node i. The sum of all neighbor messages based on attention weights; Step 2022: Perform dynamic calculations based on the joint type, specifically based on the quasi-static assumptions, and incorporate the Coulomb friction model to calculate the forces and torques at the joints; Based on Newton's second law and rotational law, the calculated nodal forces and torques are used to calculate the linear acceleration of node i. and angular acceleration ; Then, the linear acceleration of node i is obtained through a lightweight multilayer perceptron. and angular acceleration Angular velocity of components calculated from multi-view RGB-D time-series images or videos. and linear velocity These four physical quantities are encoded into a fixed-dimensional feature vector. ; The final hidden state of the node is updated through a gated recurrent unit, which uses the current hidden state, aggregated message, and feature vector as input. For input.

6. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 5, characterized in that, Step 203 is as follows: The attitude energy function formula is as follows: ， in, Let be the attitude energy function. For the relationship diagram, The parameters represent the joint between components i and j, and task represents the task. Measuring posture Consistency with the task description; Assume that the probability distribution of the attitude distribution follows a Boltzmann distribution, that is: ， in, Representing the conditional probability distribution; performing a second-order Taylor expansion at the minimum of the attitude energy function to approximate the attitude distribution of each component as an optimal attitude. The covariance matrix of the Gaussian distribution in the tangent space centered at the center is... Defined in optimal pose tangent space The formula is as follows: ， in, Indicates in Gaussian distribution on manifold, mean Located on a manifold, covariance Tangent space defined at the mean point In the middle, belonging to real matrix, Output the prior orientation distribution parameters for each component i. This serves as a strong conditional prior for the conditional diffusion model in step 3.

7. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 6, characterized in that, The physical regularization loss specifically includes four physical constraints, each of which has its weight adjusted by a weighting coefficient. Penetration penalty : ， in, For point The component to which it belongs, This represents the total number of points in the set. for Activation function Point The value in the symbolic distance field of component j; Static stability penalty : ， in, For a small threshold, based on the predicted component quality Calculate the overall center of gravity based on the component positions, estimate the supporting polygon, and construct a two-dimensional convex hull from the projections of the contact points onto the horizontal plane. For the horizontal projection of the center of gravity to The distance to the boundary of the convex hull; Exercise energy penalty : ， in, Let i be the height of the center of gravity of component i relative to the reference plane. It is the gravitational constant; Joint restraint punishment : ， in, This represents the set of edges in the component relationship diagram. Indicates the total number of joints. It is the joint displacement calculated relative to the attitude. , These represent the lower and upper limits of joint displacement.

8. The method for generating articulated object pose based on physical perception and graph diffusion according to claim 7, characterized in that, Step 303 is as follows: Step 3031: In In the tangent space, a scoring network based on the learned value. Perform Langevin dynamics updates from noise Preliminary noise reduction proposals were obtained. : ， in, The noise attitude at step t is represented by a high-dimensional vector formed by concatenating the Lie algebra coordinates corresponding to the attitudes of all components at step t in the reverse diffusion process. For scoring networks, The conditional vector for fusion. Let t be the update step size. It is a random noise vector; Step 3032: Calculate the predicted Lie algebra coordinates. The proposed pose on SE(3) is obtained through exponential mapping. Then, the physical regularization loss under the proposed pose is calculated, and the physical correction gradient is obtained through backpropagation. The prediction results are then corrected in the tangent space, using the following formula: ， in, It is the correction step size, which is a hyperparameter; Step 3033: Correct the noise attitude Projecting back to the SE(3) manifold via an exponential mapping, To obtain the noise attitude for the next step. Repeat steps 3031-3033 until t=0.

Images