Robot work space interference detection method based on digital twinning

Abstract

The application relates to the technical field of robot working space interference detection, and proposes a robot working space interference detection method based on digital twinning, which comprises the following steps: a three-dimensional model of a robot is established, and a three-dimensional model of the surrounding environment is established; vertex set and triangular mesh set of the triangular mesh data of the three-dimensional model are extracted; a covariance matrix of each BVH node is calculated and feature decomposition is carried out to determine the local coordinate axis, center and half edge length of the OBB bounding box; node segmentation is carried out based on the projection value of the triangular mesh barycenter on the longest main shaft of the OBB, left and right child nodes are recursively constructed, and all nodes are divided into leaf nodes; an AABB bounding box is established for each model in the digital twinning body, and whether the AABB bounding box collides in the simulation frame is detected; the BVH structure is traversed, and whether the OBB bounding box intersects is recursively detected until the leaf node; for the triangular meshes of each pair of leaf nodes, whether the collision occurs is judged by using the GJK algorithm, and the collision point information is calculated.

Description

Interferometric Detection Method for Robotic Workspace Based on Digital Twin Technical Field

[0001] This invention relates to the field of robot operation space interferometry detection technology, and in particular to a robot operation space interferometry detection method based on digital twins. Background Technology

[0002] In recent years, with the rapid development of science and technology, intelligent manufacturing technology has promoted the intelligentization and digitalization of industrial production. In the process of industrial production, robot operation has become an important part of industrial automation. In order to ensure the safety of robot operation, it is necessary to ensure that the robot does not collide or interfere with surrounding objects during its movement, that is, the robot's safe working space.

[0003] Currently, common methods for detecting interference in robot workspaces have the following drawbacks: The robot workspace is difficult to visualize and is a static space, while robot operation is a dynamic space involving the motion processes between teaching points. Simply using the robot workspace to ensure robot operation safety is no longer sufficient. Physical robot debugging is difficult and time-consuming. During layout, the robot needs to be taught and run to ensure a safe working environment. If collisions occur during physical operation, the risk of robot damage increases. Collision detection performance is poor in virtual environments. During simulation, collision detection between the robot and its surroundings needs to be performed every frame to check for interference. Using narrow collision detection will significantly reduce the algorithm speed, causing frame drops and failing to truly simulate real-world conditions. Conventional collision detection methods such as SAT (Separating Axis) cannot support collision detection between any convex shapes, requiring additional operations, such as adding special code and algorithms to handle curved surfaces. Summary of the Invention

[0004] To address the aforementioned shortcomings, the present invention aims to propose a robot operation space interference detection method based on digital twins. This method constructs a digital twin of the robot and its surrounding environment and establishes its BVH data structure. By combining generalized and narrow collision detection algorithms, it achieves accurate and efficient detection and visual feedback of interference during robot operation, thereby significantly improving layout safety and debugging efficiency.

[0005] To achieve this objective, the present invention adopts the following technical solution:

[0006] A robot operation space interferometric detection method based on digital twins, comprising the following steps:

[0007] S1: Constructing a digital twin of the robot and its surrounding environment:

[0008] Based on the robot's mechanical structure and kinematic parameters, a three-dimensional model of the robot is established, and forward and inverse kinematic algorithms are constructed according to the robot's kinematic relationships to establish a three-dimensional model of the surrounding environment.

[0009] S2: Construct the BVH data structure for the 3D model of the robot and its surrounding environment:

[0010] Extracting vertex and triangular mesh sets from the triangular mesh data of the 3D model, and recursively constructing BVH nodes based on OBB bounding boxes, includes the following sub-steps:

[0011] S21: Calculate the covariance matrix for each BVH node and perform eigenvalue decomposition to determine the local coordinate axes, center, and half-side length of the OBB bounding box;

[0012] S22: Node segmentation is performed based on the projection value of the centroid of the triangular mesh onto the longest principal axis of the OBB, and left and right child nodes are recursively constructed until all nodes are divided into leaf nodes.

[0013] S3: Perform generalized collision detection based on AABB bounding boxes:

[0014] An AABB bounding box is created for each model in the digital twin, and collisions between pairs of AABB bounding boxes are detected in each simulation frame.

[0015] S4: For the model pairs that detected collisions in S3, perform narrow collision detection based on the GJK algorithm:

[0016] Traverse the BVH structure of the two models and recursively check whether their OBB bounding boxes intersect until the leaf nodes. For each pair of leaf nodes, use the GJK algorithm to determine whether a collision has occurred and calculate the collision point information.

[0017] S5: Based on the collision point information, provide visual prompts and position adjustments for the objects that collide in the virtual environment.

[0018] Preferably, step S1 includes:

[0019] Based on the mechanical structure of the robot, each joint is modeled independently, and the joint models are assembled into a complete joint kinematic chain according to the kinematic relationship of the robot.

[0020] The joint kinematic chains are corrected based on the robot's standard dimensional parameters or DH parameters;

[0021] Based on the corrected robot's kinematics chain and DH parameters, a forward kinematics algorithm for calculating the robot's end effector pose and an inverse kinematics algorithm for calculating the pose of each joint of the robot are constructed.

[0022] Preferably, step S2 includes:

[0023] Extract the triangular mesh data of the three-dimensional model to obtain the vertex set and the triangular mesh index set;

[0024] For each BVH node, calculate the covariance matrix of all the triangular meshes it includes, and perform eigenvalue decomposition on the covariance matrix to determine the local coordinate axes of the OBB bounding box;

[0025] Project all vertices within the BVH node onto the local coordinate axes to determine the center point and half-side length of the OBB bounding box;

[0026] Calculate the projection values ​​of the centroids of all triangular meshes within the BVH node onto the longest principal axis of the OBB, and use the median of the projection values ​​as the segmentation threshold to divide the triangular meshes of the current node into two subsets, left and right.

[0027] Recursively construct child nodes for each subset until a child node consists of only a triangular mesh and is marked as a leaf node, thus completing the construction of the BVH data structure.

[0028] Preferably, the vertex set satisfies the following relation: The triangular mesh set satisfies the following relation: The triangular grid index set satisfies the following relation: ;

[0029] The covariance matrix of the triangular mesh satisfies the following relationship:

[0030] ;

[0031] in To calculate the first The centroid of the vertices of a triangular mesh. , , For the first The three vertices of a triangle, It is the covariance matrix;

[0032] The local coordinate axes satisfy the following relationship: The calculation of half-side length satisfies the following relationship: ,in This represents the maximum value of all vertices within a BVH node projected onto the local coordinate axis. This represents the minimum value of all vertices within a BVH node projected onto the local coordinate axis.

[0033] Preferably, the collision between any two AABB bounding boxes is detected to satisfy the following relationship:

[0034] ;

[0035] in, , and These represent the maximum values ​​of the bounding box on the three coordinate axes (X, Y, Z), respectively. , and These represent the minimum values ​​of the bounding box on the three coordinate axes (X, Y, Z).

[0036] Preferably, step S4 includes:

[0037] The BVH structure of the two models is recursively traversed. The intersection relationship between the nodes of the two tree structures is detected by depth-first traversal. The separation axis theorem is used to check whether the two OBB bounding boxes intersect, until the two nodes are both leaf nodes.

[0038] For each pair of leaf nodes, calculate the Minkowski difference of the triangular mesh to obtain the convex body M, and determine the positional relationship of the origin O relative to the convex body M: if the origin O is inside M, it is determined to be intersecting; if it is on the boundary of M, it is determined to be in contact; if it is outside M, it is determined to be separated.

[0039] Initialize the maximum number of iterations and tolerance error of the GJK algorithm. Use the difference vector between the centroids of the two triangular meshes as the initial search direction. Call the support function to calculate the farthest point in that direction and add it to the simplex set.

[0040] Determine if the origin is inside the current simplex: if not, calculate the nearest point from the origin to the simplex, use the direction of that point as the next search direction, and continue iterating until the tolerance is met or the maximum number of iterations is reached;

[0041] The nearest point from the origin to the Minkowski difference set is calculated based on the final simplex set. The nearest points on the two triangular meshes are then calculated in reverse. Finally, the normal vector and position information of the collision point are calculated.

[0042] If two triangular meshes collide, calculate their penetration depth: traverse all candidate separating axes of the two triangles, including the normal vectors of the two triangles and the cross product vector between each side, project the two triangles onto each axis to obtain the projection interval; if the intervals overlap, calculate the overlap amount, and take the minimum overlap amount among all axes as the penetration depth.

[0043] Preferably, the Minkowski difference is calculated to satisfy the following relation:

[0044] ;

[0045] Where A and B are different triangular meshes;

[0046] Support functions satisfy the following relation: ;

[0047] in, For the vertices of triangular mesh A, These are the vertices of triangular mesh B;

[0048] The nearest point from the origin to the Minkowski difference set satisfies the following relation:

[0049] ;

[0050] in, Denotes a proportion of real numbers that are greater than or equal to 0, where, The vertex on object A corresponding to the i-th simplex point The vertex on object B corresponding to the i-th simplex point can be used to calculate the normal and the intersecting vertex respectively: .

[0051] Preferably, step S5 includes:

[0052] The collision objects are displayed with different material colors to generate the collision, and then the position of the colliding objects is adjusted according to the collision point information until no interference occurs.

[0053] One of the above technical solutions has the following advantages or beneficial effects:

[0054] This invention constructs a digital twin of the robot and its surrounding environment, establishing a kinematic model and 3D geometric representation consistent with the physical object, providing a high-fidelity virtual environment foundation for interference detection. By recursively constructing a BVH hierarchy based on OBB bounding boxes, it achieves spatial segmentation and efficient management of complex geometric scenes, significantly improving collision lookup efficiency. Employing a two-stage collision detection strategy—first generalized AABB detection followed by narrow GJK detection—it effectively reduces computational resource consumption while ensuring detection accuracy. Finally, through visual prompts and position adjustment mechanisms, it provides intuitive feedback and rapid correction of collision interference. Overall, this invention enables the early detection, localization, and resolution of interference problems in the virtual environment, significantly reducing the risks and time costs of physical debugging and improving the safety and reliability of robot operation layout. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 is a flowchart of the robot operation space interferometric detection method based on digital twin provided in an embodiment of the present invention;

[0057] Figure 2 is a flowchart of the construction of the BVH data structure of the robot operation space interferometry detection method based on digital twin provided in the embodiment of the present invention;

[0058] Figure 3 is a schematic diagram demonstrating the BVH construction of the robot operation space interferometry detection method based on digital twin provided in an embodiment of the present invention;

[0059] Figure 4 is a flowchart illustrating the recursive traversal of the BVH structure in the robot operation space interferometry detection method based on digital twin provided in an embodiment of the present invention.

[0060] Figure 5 is a schematic diagram of the Minkowski difference of the robot operation space interferometric detection method based on digital twin provided in the embodiment of the present invention. Detailed Implementation

[0061] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0062] In this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0063] A robot operation space interferometry detection method based on digital twins, as shown in Figure 1, includes the following steps:

[0064] S1: Constructing a digital twin of the robot and its surrounding environment:

[0065] Based on the robot's mechanical structure and kinematic parameters, a three-dimensional model of the robot is established, and forward and inverse kinematic algorithms are constructed according to the robot's kinematic relationships to establish a three-dimensional model of the surrounding environment.

[0066] It's important to note that a digital twin is a virtual model created digitally that corresponds to a physical entity, reflecting its state and behavior in real time. For example, in this step, the robot's digital twin is a virtual model created using 3D modeling software (such as SolidWorks, Unigraphics NX, 3D Studio Max, etc.) based on the robot's mechanical structure and kinematic parameters. This model not only includes the robot's external shape but also accurately reflects its internal mechanical structure and kinematic relationships, enabling the simulation of the robot's real movements in a virtual environment. The forward kinematics algorithm uses the robot's joint poses as input to solve for the robot's end-effector pose matrix, predicting the robot's end-effector position and orientation at a given joint angle. The inverse kinematics algorithm uses the robot's end-effector pose matrix as input to solve for the robot's joint poses, primarily used to plan the robot's trajectory and ensure it accurately reaches the target position. The 3D model of the surrounding environment is a virtual representation of other objects and equipment in the robot's workspace, forming a complete virtual work scene together with the robot's digital twin.

[0067] Understandably, the purpose of building a digital twin is to accurately simulate the robot's operation in a virtual environment, thereby enabling interference detection and path planning in advance. By creating a high-precision model of the robot and constructing forward and inverse kinematics algorithms, various simulation experiments can be conducted quickly and safely without relying on a physical robot, evaluating the feasibility and safety of different layout schemes. Simultaneously, incorporating the surrounding environment into the virtual scene as a 3D model ensures the accuracy of the simulation results, realistically reflecting the interaction between the robot and its environment. By using digital technology to map the physical world onto virtual space and leveraging the powerful computing capabilities of computers, efficient analysis and optimization of complex systems can be achieved.

[0068] Specifically, this process begins by collecting detailed mechanical structure and kinematic parameters of the robot, including the type, range of motion, connection method, and DH parameters of each joint. Then, using specialized 3D modeling software, a 3D model of the robot is constructed based on these parameters, ensuring that the model's kinematic relationships are consistent with the actual robot. Next, based on the robot's DH parameters, forward and inverse kinematics algorithms are developed, typically prioritizing analytical methods for faster solution speeds. Modeling the surrounding environment can also be done using 3D modeling software, creating corresponding 3D models based on actual dimensions and shapes, and integrating them with the robot model into the same virtual scene. For example, for a welding robot in an automotive manufacturing workshop, a digital twin of the robot itself, as well as surrounding equipment such as workpieces, fixtures, and conveyor belts at the welding station, needs to be constructed to completely reproduce the actual production layout in the virtual environment.

[0069] S2: Construct the BVH data structure for the 3D model of the robot and its surrounding environment:

[0070] Extracting vertex and triangular mesh sets from the triangular mesh data of the 3D model, and recursively constructing BVH nodes based on OBB bounding boxes, includes the following sub-steps:

[0071] S21: Calculate the covariance matrix for each BVH node and perform eigenvalue decomposition to determine the local coordinate axes, center, and half-side length of the OBB bounding box;

[0072] S22: Node segmentation is performed based on the projection value of the centroid of the triangular mesh onto the longest principal axis of the OBB, and left and right child nodes are recursively constructed until all nodes are divided into leaf nodes.

[0073] It's important to note that the BVH (BoundingVolumeHierarchy) data structure is a hierarchical bounding box structure used for efficient collision detection and ray tracing. In this step, the OBB (OrientedBoundingBox) bounding box is a directed bounding box. Compared to the AABB (Axis-AlignedBoundingBox) bounding box, it can more closely fit the shape of the object, reducing unnecessary collision detection and improving detection efficiency. Triangular mesh data is the basic building block of the 3D model, consisting of a vertex set and a triangular mesh set. Each triangular mesh is defined by three vertices and their indices. The vertex set contains the coordinates of all vertices of the model, while the triangular mesh set defines how these vertices connect to form triangular faces. The covariance matrix is ​​a statistical measure used to analyze the distribution characteristics of data points. In this step, by calculating the covariance matrix for the vertices of the triangular mesh and performing eigenvalue decomposition, the local coordinate axes of the OBB bounding box can be determined, allowing the bounding box to closely fit the shape of the object. The local coordinate axes are the coordinate system on which the OBB bounding box is based, determining the orientation and shape of the bounding box. The center and half-side length are the position and size parameters of the bounding box in the local coordinate system, used to define the specific range of the bounding box.

[0074] Understandably, the purpose of constructing a BVH data structure is to accelerate the collision detection process. In complex 3D models, directly performing collision detection on each triangular mesh is very time-consuming. By constructing a BVH hierarchy, the model can be decomposed into multiple levels of bounding boxes. Collision detection is first performed at a coarser level, and only when two bounding boxes are likely to collide are their internal child nodes or triangular meshes further detected, greatly reducing the number of triangular meshes that need to be detected and improving detection efficiency. At the same time, using OBB bounding boxes, compared to AABB bounding boxes, can fit the object shape more tightly, reducing the overlapping area between bounding boxes and further reducing unnecessary collision detection times. In addition, node segmentation based on the projection value of the triangular mesh centroid on the longest principal axis of OBB can ensure that the segmented child node bounding boxes still maintain good compactness, which is conducive to constructing an efficient BVH tree structure. In practical applications, the process of constructing the BVH data structure is shown in Figure 2.

[0075] For example, consider an industrial robot performing a parts-grabbing task; its 3D model consists of thousands of triangular meshes. To construct the BVH data structure, the vertex and index information of all triangular meshes is first extracted. Then, starting from the root node, the covariance matrix containing all triangular meshes is calculated, and eigenvalue decomposition is performed to determine the local coordinate axes of the OBB bounding box. Next, the projection range of all triangular mesh vertices on the local coordinate axes is calculated to obtain the center and half-side length of the bounding box. Based on the projection value of the triangular mesh centroid on the longest principal axis of the OBB, the triangular mesh is divided into left and right subsets, and child nodes are created for each subset. This process is repeated until all nodes are divided into leaf nodes. The final constructed BVH tree structure enables the rapid elimination of most triangular meshes that are unlikely to collide during collision detection, allowing detailed detection only of areas where collisions are possible, thus significantly improving detection efficiency. In another embodiment, as shown in Figure 3, the first step in constructing the BVH is to use all triangular meshes of the entire model as a root node. This root node contains the indices of all triangular meshes, represented as 0, 1, 2, 3, 4, 5, 6, 7.

[0076] The root node is split based on the projection of the triangular mesh centroid onto the longest principal axis of the OBB (Oriented Bounding Box). The local coordinate axes of the OBB are determined by calculating the covariance matrix of all triangular mesh centroids and performing eigenvalue decomposition. Then, all triangular mesh centroids are projected onto the longest principal axis, and the median of the projection values ​​is found, dividing the triangular mesh into two subsets. For example, the root node 0,1,2,3,4,5,6,7 is split into a left subset 0,5,2,7 and a right subset 4,1,6,3.

[0077] Repeat the above splitting process for the left child nodes 0, 5, 2, 7. Calculate the covariance matrix of the centroid of the triangular mesh within this subset, determine the local coordinate axes of the OBB, and project the triangular mesh onto the longest principal axis. Find the median projection value and divide the triangular mesh into two subsets, such as 0, 2 and 5, 7.

[0078] Similarly, split the right child nodes 4,1,6,3. Calculate the covariance matrix, determine the local coordinate axes of the OBB, project the centroid of the triangular mesh, and find the median projection value. Divide the triangular mesh into two subsets, such as 6,1 and 4,3.

[0079] Continue splitting each child node until each subset contains only a single triangular mesh; these child nodes are then called leaf nodes. For example, subset 0,2 splits into leaf nodes 0 and 2, subset 5,7 splits into leaf nodes 5 and 7, and so on, until a complete BVH tree structure is formed.

[0080] During the construction of the BVH, the indices of the triangular mesh are rearranged according to the splitting results. The initial indices are 0,1,2,3,4,5,6,7, which become 0,5,2,7,4,1,6,3 after the root node split. With further splitting, the indices continue to be adjusted, eventually forming an index order that adapts to the BVH structure.

[0081] S3: Perform generalized collision detection based on AABB bounding boxes:

[0082] An AABB bounding box is created for each model in the digital twin, and collisions between pairs of AABB bounding boxes are detected in each simulation frame.

[0083] It's important to note that AABB (Axis-Aligned Bounding Box) is an axis-aligned bounding box with minimum and maximum values ​​along each of the three coordinate axes, used to quickly determine the approximate extent of an object. In this step, the role of AABB bounding boxes is to perform rapid collision detection screening for each model in the digital twin. By creating an AABB bounding box for each model, the possibility of collisions between models can be efficiently detected in each simulation frame, thereby quickly narrowing down the detection range and reducing the computational load of subsequent narrow collision detection.

[0084] Understandably, the purpose of performing generalized collision detection based on AABB bounding boxes is to quickly and efficiently eliminate model pairs that are unlikely to collide, reducing the burden on subsequent narrow collision detection. In virtual simulation environments, a large number of collision checks are required for each frame. If narrow collision detection is performed directly on the triangular meshes of all models, the computational load will be extremely large, leading to a significant drop in simulation performance. By first using AABB bounding boxes for generalized collision detection, their simple axis alignment properties can be utilized to quickly determine whether two models are likely to collide. If two AABB bounding boxes do not intersect, then the corresponding models are unlikely to collide, and this pair of models can be immediately excluded without further narrow collision detection. Only when two AABB bounding boxes intersect is more precise narrow collision detection required for this pair of models. The core principle of this method is to leverage the simplicity and speed of AABB bounding boxes to perform preliminary screening for complex collision detection tasks, improving overall detection efficiency.

[0085] For example, in a complex assembly line containing multiple robots and workpieces, each robot's digital twin, along with surrounding workpieces and fixtures, establishes its own AABB bounding box. During simulation, each frame performs pairwise collision detection on the AABB bounding boxes of these models. For instance, when a robot's AABB bounding box and a workpiece's AABB bounding box are detected to meet collision conditions in all three axial directions, the system marks this pair as potentially colliding and performs narrow-sense collision detection based on the GJK algorithm in subsequent steps. Other model pairs that do not meet the conditions are excluded from narrow-sense detection, thus saving significant computational resources and ensuring the smoothness of the simulation process.

[0086] S4: For the model pairs that detected collisions in S3, perform narrow collision detection based on the GJK algorithm:

[0087] Traverse the BVH structure of the two models and recursively check whether their OBB bounding boxes intersect until the leaf nodes. For each pair of leaf nodes, use the GJK algorithm to determine whether a collision has occurred and calculate the collision point information.

[0088] It should be noted that the GJK (Gilbert–Johnson–Keerthi) algorithm is an iterative algorithm for determining whether two convex polyhedra intersect. It determines their relative positional relationship by calculating the Minkowski difference between the two polyhedra. In this step, the role of the GJK algorithm is to perform precise collision detection on model pairs that may still collide after generalized collision detection, and to calculate the collision point information for subsequent interference processing. The BVH (Bounding Volume Hierarchy) structure is a hierarchical bounding box structure used to efficiently organize and query the geometric information of the model. The OBB (Oriented Bounding Box) bounding box is a directed bounding box that can more tightly fit the shape of the object, improving the efficiency and accuracy of collision detection. The leaf node is the bottom-level node of the BVH tree, corresponding to a single triangular mesh of the model. In narrow collision detection, it is necessary to traverse the BVH structures of the two models, recursively checking whether the OBB bounding boxes intersect until the leaf node is reached, and then apply the GJK algorithm to the corresponding triangular mesh for precise collision detection.

[0089] Understandably, the purpose of performing narrow collision detection based on the GJK algorithm is to accurately determine whether a collision has actually occurred between models and to obtain detailed information about the collision points for subsequent intervention processing and path adjustment. In the generalized collision detection stage, although potentially colliding model pairs can be quickly identified, the detection results of AABB bounding boxes may contain many false alarms, indicating that a collision has not actually occurred. Therefore, a more accurate collision detection method is needed to confirm whether a collision has actually occurred and to calculate information such as the position of the collision point and the direction of the normal. The GJK algorithm, due to its efficiency and applicability to any convex shape, is an ideal choice for narrow collision detection. By combining it with a hierarchical detection strategy based on BVH structures, detection efficiency can be further improved and unnecessary computation can be reduced.

[0090] For example, in a collaborative assembly task involving two robotic arms, generalized collision detection identifies a potential collision between their AABB bounding boxes. At this point, the system initiates narrow-sense collision detection. First, it traverses the BVH structure of the two robotic arm models, recursively checking if their OBB bounding boxes intersect. Assuming an intersection is detected, it continues to examine their child nodes until it reaches the leaf nodes, i.e., the two specific triangular meshes. Then, the GJK algorithm is applied to perform collision detection on these two triangular meshes. By calculating their Minkowski difference and iteratively using support functions and simplex sets, it ultimately determines whether the two triangular meshes have actually collided, and calculates information such as the location of the collision point and the direction of the normal. If a collision occurs, the system records this information for subsequent intervention processing and path adjustment.

[0091] S5: Based on the collision point information, provide visual prompts and position adjustments for the objects that collide in the virtual environment.

[0092] It's important to note that visual cues refer to intuitively displaying the location and situation of a collision within a virtual environment, typically achieved by changing the material color of the colliding objects. For example, when two objects collide, their collision area can be displayed in red to warn the user. Position adjustment, based on collision point information, involves moving or rotating the colliding objects to remove them from the collision state. Collision point information includes the location of the collision point, the direction of the normal, and the penetration depth; this information guides the object's movement to avoid interference. In this step, the purpose of visual cues and position adjustment is to allow the user to intuitively understand the collision situation and adjust the object's position automatically or manually to optimize the robot's operational layout and ensure the safety of the operational path.

[0093] Understandably, the purpose of visual prompts and position adjustments is to provide users with intuitive feedback on collision detection results and to eliminate collision interference by adjusting object positions, thereby optimizing the robot's operational path. In a virtual simulation environment, users need to be able to clearly identify which objects have collided and their specific locations. By displaying colliding objects with different material colors, the user's attention can be immediately drawn to the collision area, allowing them to quickly pinpoint the problem. Simultaneously, using collision point information to adjust object positions can automatically or manually modify the objects to remove them from the collision state. The core principle of this process lies in using the detailed information provided by the collision detection algorithm to precisely adjust objects, thereby optimizing the overall operational layout and ensuring the robot's safety and efficiency in actual operation.

[0094] Preferably, step S1 includes:

[0095] Based on the mechanical structure of the robot, each joint is modeled independently, and the joint models are assembled into a complete joint kinematic chain according to the kinematic relationship of the robot.

[0096] The joint kinematic chains are corrected based on the robot's standard dimensional parameters or DH parameters;

[0097] Based on the corrected robot's kinematics chain and DH parameters, a forward kinematics algorithm for calculating the robot's end effector pose and an inverse kinematics algorithm for calculating the pose of each joint of the robot are constructed.

[0098] It should be noted that mechanical structure refers to the physical construction of the robot, including its joints, arms, and end effectors, as well as their connections; kinematic relationships describe the kinematic constraints between the robot's joints, such as rotational and translational relationships; DH parameters are Denavit-Hartenberg parameters, used to accurately describe the geometric relationships between robot links. In this step, independently modeling each joint can realistically reflect the robot's kinematic characteristics, while using DH parameters to correct the joint kinematic chains can eliminate dimensional and assembly errors between the virtual model and the actual robot. The forward kinematics algorithm is used to calculate the end effector's pose based on joint angles, while the inverse kinematics algorithm is used to solve for the joint angles based on the end effector's target pose.

[0099] Understandably, the purpose of this step is to ensure that the virtual robot can realistically reproduce the dynamic behavior of the actual robot by accurately modeling and correcting its kinematic characteristics. Specifically, independent joint modeling and DH parameter-based correction can improve the accuracy of the virtual model, making its kinematic relationships consistent with the actual robot. The construction of forward and inverse kinematics algorithms further supports the robot's path planning and task simulation in the virtual environment. The effect of this process is not only to improve the accuracy of virtual simulation, but also to provide a reliable foundation for subsequent interferometry detection and path optimization.

[0100] For example, suppose we need to build a digital twin of a six-axis welding robot. Engineers first extract the robot's DH parameters, then use SolidWorks to independently model each joint of the robot and assemble them according to the actual kinematic relationships. Next, they develop forward and inverse kinematics algorithms. The forward kinematics algorithm predicts the position and orientation of the welding torch at different joint angles, while the inverse kinematics algorithm plans the joint angles required for the robot to move to the target welding point. By running these algorithms in a virtual simulation environment and comparing them with an actual robot, they verified that the virtual model can realistically reproduce the welding motion of the actual robot, and the error is controlled within an acceptable range.

[0101] Preferably, step S2 includes:

[0102] Extract the triangular mesh data of the three-dimensional model to obtain the vertex set and the triangular mesh index set;

[0103] For each BVH node, calculate the covariance matrix of all the triangular meshes it includes, and perform eigenvalue decomposition on the covariance matrix to determine the local coordinate axes of the OBB bounding box;

[0104] Project all vertices within the BVH node onto the local coordinate axes to determine the center point and half-side length of the OBB bounding box;

[0105] Calculate the projection values ​​of the centroids of all triangular meshes within the BVH node onto the longest principal axis of the OBB, and use the median of the projection values ​​as the segmentation threshold to divide the triangular meshes of the current node into two subsets, left and right.

[0106] Recursively construct child nodes for each subset until a child node consists of only a triangular mesh and is marked as a leaf node, thus completing the construction of the BVH data structure.

[0107] It's important to note that triangular mesh data is a fundamental component of the 3D model, consisting of a set of vertices and a set of triangular mesh indices; the covariance matrix is ​​a statistic used to analyze the distribution characteristics of data points; eigenvalue decomposition is used to determine the orientation of the OBB bounding box; projection values ​​are used for node segmentation, with the median serving as the segmentation threshold. In this step, by extracting triangular mesh data and calculating the covariance matrix, the local coordinate axes of the OBB bounding box can be determined, ensuring that the bounding box closely fits the model shape. The calculation of projection values ​​and the median segmentation strategy ensure that the BVH tree construction process is both efficient and accurate.

[0108] Understandably, the purpose of this step is to provide an optimized data organization method for collision detection by constructing an efficient BVH data structure. Specifically, the orientation and extent of the OBB bounding box are determined using the covariance matrix and eigenvalue decomposition, ensuring that the bounding box closely fits the model shape and reduces overlapping areas between nodes. A node segmentation strategy based on projection values ​​further optimizes the hierarchical structure of the BVH tree, ensuring the efficiency of collision detection. The effect of this process is not only to accelerate collision detection but also to guarantee the accuracy of the detection results, providing a foundation for optimization in subsequent narrow collision detection.

[0109] For example, consider a model of an industrial part with complex geometry. To construct the BVH data structure for this model, the vertices and indices of all triangular meshes are first extracted. Then, starting from the root node, the covariance matrix containing all triangular meshes is calculated, and eigenvalue decomposition is performed to determine the orientation and extent of the OBB bounding box. Next, the vertices of the triangular meshes are projected onto the local coordinate axes to find the center and half-side length of the bounding box. Based on the projection of the triangular mesh centroid onto the longest principal axis of the OBB, the triangular mesh is divided into left and right subsets, and child nodes are created for each subset. This process is repeated until all nodes are classified as leaf nodes. The final constructed BVH tree structure significantly reduces redundant computations in collision detection, improving the smoothness of the simulation.

[0110] Preferably, the vertex set satisfies the following relation: The triangular mesh set satisfies the following relation: The triangular grid index set satisfies the following relation: ;

[0111] The covariance matrix of the triangular mesh satisfies the following relationship:

[0112] ;

[0113] in To calculate the first The centroid of the vertices of a triangular mesh. , , For the first The three vertices of a triangle, It is the covariance matrix;

[0114] The local coordinate axes satisfy the following relationship: The calculation of half-side length satisfies the following relationship: ,in This represents the maximum value of all vertices within a BVH node projected onto the local coordinate axis. This represents the minimum value of all vertices within a BVH node projected onto the local coordinate axis.

[0115] The construction process of the BVH data structure is precisely defined by mathematical formulas to ensure that the bounding box can optimally fit the geometry of the 3D model. Specifically, the vertex set and triangular mesh set provide the basic geometric information of the model, while the covariance matrix and eigenvalue decomposition are used to determine the orientation and extent of the bounding box, enabling it to closely fit the model shape. The calculation of half-side length and center further defines the specific size and position of the bounding box. The effect of this process is that it provides a standardized and repeatable computational method for BVH construction, ensuring optimal fitting of the bounding box and thus improving the efficiency and accuracy of collision detection.

[0116] Preferably, the collision between any two AABB bounding boxes is detected to satisfy the following relationship:

[0117] ;

[0118] in, , and These represent the maximum values ​​of the bounding box on the three coordinate axes (X, Y, Z), respectively. , and These represent the minimum values ​​of the bounding box on the three coordinate axes (X, Y, Z).

[0119] By employing simple AABB bounding box collision detection, potentially colliding model pairs are quickly identified, thereby reducing the computational burden of narrow collision detection and improving overall simulation performance. Specifically, AABB bounding boxes define the maximum and minimum bounding areas of the models along the three coordinate axes to quickly determine whether two models may collide. The detection criteria are based on interval overlap along the three axes; only when two bounding boxes satisfy the overlap condition in all axes are they considered to be colliding. The advantage of this process is that it leverages the simplicity and speed of AABB bounding boxes to perform preliminary screening for complex collision detection tasks, ensuring that subsequent narrow collision detection focuses on potential collision scenarios, thus effectively improving detection efficiency.

[0120] Preferably, step S6 includes:

[0121] The BVH structure of the two models is recursively traversed. The intersection relationship between the nodes of the two tree structures is detected by depth-first traversal. The separation axis theorem is used to check whether the two OBB bounding boxes intersect, until the two nodes are both leaf nodes.

[0122] For each pair of leaf nodes, calculate the Minkowski difference of the triangular mesh to obtain the convex body M, and determine the positional relationship of the origin O relative to the convex body M: if the origin O is inside M, it is determined to be intersecting; if it is on the boundary of M, it is determined to be in contact; if it is outside M, it is determined to be separated.

[0123] Initialize the maximum number of iterations and tolerance error of the GJK algorithm. Use the difference vector between the centroids of the two triangular meshes as the initial search direction. Call the support function to calculate the farthest point in that direction and add it to the simplex set.

[0124] Determine if the origin is inside the current simplex: if not, calculate the nearest point from the origin to the simplex, use the direction of that point as the next search direction, and continue iterating until the tolerance is met or the maximum number of iterations is reached;

[0125] The nearest point from the origin to the Minkowski difference set is calculated based on the final simplex set. The nearest points on the two triangular meshes are then calculated in reverse. Finally, the normal vector and position information of the collision point are calculated.

[0126] If two triangular meshes collide, calculate their penetration depth: traverse all candidate separating axes of the two triangles, including the normal vectors of the two triangles and the cross product vector between each side, project the two triangles onto each axis to obtain the projection interval; if the intervals overlap, calculate the overlap amount, and take the minimum overlap amount among all axes as the penetration depth.

[0127] It should be noted that depth-first traversal is a tree-structure traversal strategy that prioritizes visiting one child node of the root node and all its descendants, then backtracks to visit other child nodes; the Separating Axis Theorem is a method for determining whether two convex polyhedra intersect, proving that the two objects do not intersect by finding a separating axis; the Oriented Bounding Box (OBB) is a directed bounding box that can more closely fit the shape of an object; the Minkowski Difference is a geometric operation used to transform the collision detection of two objects into a relationship detection between an object and the origin; the Gilbert–Johnson–Keerthi (GJK) algorithm is an iterative algorithm for determining whether two convex polyhedra intersect; the Simplex Set is the set used in the GJK algorithm to store the farthest point; and the Penetration Depth is the minimum distance that two colliding objects need to be separated. In this step, by recursively traversing the BVH structure and combining it with the separating axis theorem, we can efficiently detect whether a collision may occur between two models, and use the GJK algorithm to accurately determine whether a collision has actually occurred, while calculating the collision point information and penetration depth.

[0128] Understandably, the purpose of this step is to accurately determine whether a collision has occurred between two models and to obtain detailed collision information for subsequent intervention processing and path adjustment. By recursively traversing the BVH structure, leaf nodes (triangular meshes) that may collide can be quickly located, reducing unnecessary detailed detection. Combining the separating axis theorem to check whether the OBB bounding boxes intersect further narrows the detection range. Using the GJK algorithm to calculate the Minkowski difference and determine the origin position, it is possible to accurately determine whether two triangular meshes intersect and calculate the position and normal vector of the collision point. In addition, by calculating the penetration depth, the severity of the collision can be understood, providing a basis for subsequent adjustments. The effect of this process is that it not only improves the accuracy of collision detection but also provides reliable data support for optimizing robot operation paths and avoiding collision accidents. In practical applications, the recursive traversal of the BVH structure is shown in Figure 4.

[0129] Preferably, the Minkowski difference is calculated to satisfy the following relation:

[0130] ;

[0131] Where A and B are different triangular meshes;

[0132] Support functions satisfy the following relation: ;

[0133] in, For the vertices of triangular mesh A, These are the vertices of triangular mesh B;

[0134] The nearest point from the origin to the Minkowski difference set satisfies the following relation:

[0135] ;

[0136] in, Denotes a proportion of real numbers that are greater than or equal to 0, where, The vertex on object A corresponding to the i-th simplex point The vertex on object B corresponding to the i-th simplex point can be used to calculate the normal and the intersecting vertex respectively: .

[0137] It's important to note that the Minkowski difference is a geometric operation used to transform collision detection between two objects into a relationship detection between an object and the origin. The support function is a function in the GJK algorithm used to find the farthest point of an object in a specified direction. The simplex set is the set in the GJK algorithm used to store the farthest points. The normal is a vector perpendicular to the collision surface, used to indicate the collision direction. The intersection vertex is the specific location of the collision point. In this step, by calculating the Minkowski difference between two triangular meshes, the collision detection problem can be transformed into determining whether the origin lies within the difference set. The support function is used to find the farthest point in a specified direction, thus progressively constructing the simplex set. By iteratively calculating the nearest point from the origin to the simplex set, the location of the collision point and the direction of the normal can be determined, thereby calculating collision information.

[0138] Understandably, the purpose of this step is to accurately determine whether two triangular meshes collide through mathematical operations and to calculate the specific information of the collision point. By calculating the Minkowski difference (as shown in Figure 5), the complex collision detection problem can be simplified to determining the relationship between the origin and the difference set. The introduction of support functions and simplex sets allows this process to be carried out iteratively and efficiently, gradually approximating the collision point. Finally, by calculating the closest point from the origin to the difference set, the position and normal direction of the collision point on the two triangular meshes can be deduced. The effect of this process is that it not only accurately determines whether a collision has occurred, but also provides detailed collision information for subsequent interference processing, ensuring the safety and reliability of robot operations. In addition, according to Carathéodory's fundamental theorem, every point in a convex set in n-dimensional space can be represented by a convex combination of no more than n+1 points of that set, then... It is a real number representing a proportion that is greater than or equal to 0.

[0139] Preferably, step S5 includes:

[0140] The collision objects are displayed with different material colors to generate the collision, and then the position of the colliding objects is adjusted according to the collision point information until no interference occurs.

[0141] It's important to note that MaterialColor refers to the color attribute used in the virtual environment to distinguish different objects or object states; CollisionPointInformation includes the location of the collision point, the direction of the normal, and the depth of penetration, used to guide object repositioning. In this step, displaying colliding objects with different material colors visually indicates to the user the location of the collision and the objects involved. CollisionPointInformation then guides object repositioning, ensuring that the adjusted object layout no longer results in collisions. For example, an object can be moved along the normal direction to escape the collision state.

[0142] The collision detection results are presented intuitively to the user, and the positions of objects are adjusted automatically or manually to optimize the robot's operational layout and ensure the safety of the work path. Visual prompts allow users to quickly identify which objects are colliding and their specific locations. Using collision point information for position adjustment, objects can be moved automatically or manually to avoid collisions. This process not only improves the user's efficiency in identifying collision problems but also optimizes the robot's work path through precise position adjustments, reducing the risk of collisions during actual operation.

[0143] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0144] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for interferometric detection in robot workspace based on digital twins, characterized in that, The interferometric detection method for robot workspace Includes the following steps: S1: Constructing a digital twin of the robot and its surrounding environment: Based on the robot's mechanical structure and kinematic parameters, establish a 3D model of the robot, and construct forward and inverse kinematics algorithms according to the robot's kinematic relationships to establish a 3D model of the surrounding environment; S2: Constructing the BVH data structure of the 3D model of the robot and its surrounding environment: Extract vertex sets and triangular mesh sets from the triangular mesh data of the 3D model, and recursively construct BVH nodes based on OBB bounding boxes, including the following sub-steps: S21: Calculate the covariance matrix of each BVH node and perform eigenvalue decomposition to determine the local coordinate axes, center, and half-side length of the OBB bounding box; S22: Based on the triangular mesh reconstruction... S3: Perform generalized collision detection based on AABB bounding boxes: Build AABB bounding boxes for each model in the digital twin, and detect whether any two AABB bounding boxes collide in each simulation frame; S4: For model pairs that have been detected to collide in S3, perform narrow collision detection based on the GJK algorithm: Traverse the BVH structure of the two models, recursively detect whether their OBB bounding boxes intersect, until leaf nodes, and use the GJK algorithm to determine whether a collision has occurred for the triangular mesh corresponding to each pair of leaf nodes, and calculate the collision point information; S5: Based on the collision point information, provide visual prompts and position adjustments for the objects that have collided in the virtual environment; Step S4 includes: recursively traversing the BVH structures of the two models, detecting the intersection relationship between the nodes of the two tree structures through depth-first traversal, and checking whether the two OBB bounding boxes intersect using the separating axis theorem, until both nodes are leaf nodes; for each pair of leaf nodes, calculating their Minkowski difference to obtain the convex body M, and determining the positional relationship of the origin O relative to the convex body M: if the origin O is inside M, it is determined to be intersecting; if it is on the boundary of M, it is determined to be in contact; if it is outside M, it is determined to be separated; initializing the maximum number of iterations and tolerance error of the GJK algorithm, using the difference vector of the centroids of the two triangular meshes as the initial search direction, calling the support function to calculate the farthest point in that direction, and setting it as the maximum number of iterations and tolerance error of the GJK algorithm; Add to the simplex set; determine if the origin is within the current simplex: if not, calculate the nearest point from the origin to the simplex, use the direction of that point as the next search direction, and continue iterating until the tolerance is met or the maximum number of iterations is reached; calculate the nearest point from the origin to the Minkowski difference set based on the final simplex set, and reverse-engineer the nearest points on the two triangular meshes, then calculate the normal vector and position information of the collision point; if the two triangular meshes collide, calculate their penetration depth: traverse all candidate separation axes of the two triangles, including the normal vectors of the two triangles and the cross product vector between each side, project the two triangles onto each axis to obtain the projection interval; if the intervals overlap, calculate the overlap amount, and take the minimum overlap amount among all axes as the penetration depth.

2. The method for interferometric detection of robot operation space based on digital twin according to claim 1, characterized in that, Step S1 includes: independently modeling each joint based on the mechanical structure of the robot, and assembling each joint model into a complete joint kinematic chain according to the kinematic relationship of the robot; correcting the joint kinematic chain based on the standard size parameters or DH parameters of the robot; and constructing a forward kinematic algorithm for calculating the pose of the robot's end effector and an inverse kinematic algorithm for calculating the pose of each joint of the robot based on the corrected kinematic chain and DH parameters of the robot.

3. The method for detecting interferometric detection in robot workspace based on digital twins according to claim 1, characterized in that, Step S2 includes: extracting the triangular mesh data of the 3D model to obtain a vertex set and a triangular mesh index set; calculating the covariance matrix of all triangular meshes included in each BVH node, and performing eigenvalue decomposition on the covariance matrix to determine the local coordinate axes of the OBB bounding box; projecting all vertices within the BVH node onto the local coordinate axes to determine the center point and half-side length of the OBB bounding box; calculating the projection value of the centroid of all triangular meshes within the BVH node onto the longest principal axis of the OBB, and using the median of the projection value as a segmentation threshold to divide the triangular mesh of the current node into two subsets, left and right; recursively constructing child nodes for each subset until the child node contains only one triangular mesh and is marked as a leaf node, thus completing the construction of the BVH data structure.

4. The method for detecting interferometric interference in robot workspace based on digital twins according to claim 3, characterized in that, The vertex set satisfies the following relation: The triangular mesh set satisfies the following relation: The triangular grid index set satisfies the following relation: The covariance matrix of the triangular mesh satisfies the following relationship: ;in To calculate the first The centroid of the vertices of a triangular mesh. ， ， For the first The three vertices of a triangle, Let be the covariance matrix; the local coordinate axes satisfy the following relationship: The calculation of half-side length satisfies the following relationship: ,in This represents the maximum value of all vertices within a BVH node projected onto the local coordinate axis. This represents the minimum value of all vertices within a BVH node projected onto the local coordinate axis.

5. The method for interferometric detection of robot operation space based on digital twin according to claim 1, characterized in that, Check whether any two AABB bounding boxes collide and satisfy the following relationship: ;in, 、 and These represent the maximum values ​​of the bounding box on the three coordinate axes (X, Y, Z), respectively. 、 and These represent the minimum values ​​of the bounding box on the three coordinate axes (X, Y, Z).

6. The robot operation space interferometric detection method based on digital twin according to claim 1, characterized in that, Calculate the Minkowski difference to satisfy the following relation: Where A and B are different triangular meshes; the support function satisfies the following relation: ;in, For the vertices of triangular mesh A, Let B be the vertex of the triangular mesh; calculate the nearest point from the origin to the Minkowski difference set that satisfies the following relation: ;in, Denotes a proportion of real numbers that are greater than or equal to 0, where, The vertex on object A corresponding to the i-th simplex point The vertex on object B corresponding to the i-th simplex point is then used to calculate the normal and the intersecting vertex respectively: 。 7. The method for interferometric detection of robot operation space based on digital twin according to claim 1, characterized in that, Step S5 includes: displaying the colliding objects with different material colors to generate a collision, and then adjusting the position of the colliding objects based on the collision point information until no interference occurs.

Images