Collision processing method and device, electronic equipment and computer storage medium

Abstract

The invention provides a collision processing method and device, electronic equipment and a computer storage medium. Relates to the technical field of computer science. The method comprises the following steps: receiving a control instruction of the double-arm robot, wherein the control instruction is used for indicating to move the double-arm robot to a target space position; according to the control instruction, determining a collision shell grid and a plurality of target collision pairs corresponding to the double-arm robot; according to the collision shell grid and the multiple target collision pairs, collision judgment is carried out on the target space position, and a collision detection result is generated; according to the collision detection result, a control output signal is generated, and the control output signal is used for driving the double-arm robot to execute collision-free operation. According to the method, the control reliability of the double-arm robot can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a collision processing method, apparatus, electronic device, and computer storage medium. Background Technology

[0002] In scenarios such as industrial automation, medical surgery, hazardous environment operations (such as nuclear radiation areas and deep-sea exploration), and remote collaboration, dual-arm robots can achieve high-precision and high-flexibility remote task execution through teleoperation technology.

[0003] In existing technologies, when a dual-arm robot is detected approaching other parts of itself, the control reduces its movement speed to give the operator more reaction time. It mainly relies on the operator to avoid obstacles, which is limited by human perception and attention, resulting in low reliability of dual-arm robot control. Summary of the Invention

[0004] This application provides a collision handling method, apparatus, electronic device, and computer storage medium to solve the problem of low reliability in the control of dual-arm robots in the prior art.

[0005] In a first aspect, this application provides a collision handling method, comprising:

[0006] Receive control commands from the dual-arm robot, the control commands being used to instruct the dual-arm robot to move to a target spatial location;

[0007] According to the control command, the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot are determined;

[0008] Based on the collision shell mesh and the plurality of target collision pairs, the spatial position of the target is determined to generate a collision detection result.

[0009] Based on the collision detection results, a control output signal is generated, which is used to drive the dual-arm robot to perform collision-free operations.

[0010] Optionally, based on the control commands, the collision shell mesh corresponding to the dual-arm robot and multiple target collision pairs are determined, including:

[0011] According to the control command, the three-dimensional mesh model and structural description file of the dual-arm robot are obtained. The structural description file includes the robot structure, link names and joint types.

[0012] The three-dimensional mesh model is subjected to voxel dilation to generate the collision shell mesh corresponding to the dual-arm robot;

[0013] Based on the structural description file, multiple target collision pairs corresponding to the dual-arm robot are determined.

[0014] Optionally, voxel dilation is performed on the 3D mesh model to generate a collision shell mesh corresponding to the dual-arm robot, including:

[0015] The three-dimensional mesh model of the dual-arm robot is subjected to voxel conversion processing to obtain the three-dimensional voxel matrix of the dual-arm robot;

[0016] The three-dimensional voxel matrix of the dual-arm robot is dilated to obtain an dilated Boolean matrix;

[0017] The collision shell mesh is obtained by performing cube reconstruction on the expanded Boolean matrix.

[0018] Optionally, based on the structural description file, multiple target collision pairs corresponding to the dual-arm robot are determined, including:

[0019] Based on the geometric information of each link and the joint relationships in the structural description file, multiple initial collision pairs are generated;

[0020] By using a three-layer filtering rule, invalid collision pairs are filtered out from the multiple initial collision pairs to obtain multiple target collision pairs. The invalid collision pairs include collision pairs excluded by fixed installation contact or geometric constraints.

[0021] Optionally, the dual-arm robot includes multiple links; based on the collision shell mesh and the multiple target collision pairs, collision judgment is performed on the spatial position of the target to generate a collision detection result, including:

[0022] Based on the target spatial position and the collision shell mesh, determine the pose matrix of the geometry to be detected corresponding to each link. The pose matrix is ​​used to indicate the spatial position and orientation of the geometry to be detected.

[0023] Based on the pose matrix of each link corresponding to the geometry to be detected, collision detection processing is performed on the multiple target collision pairs to obtain the collision detection results.

[0024] Optionally, the plurality of links include a base link, a left arm link, and a right arm link; based on the target spatial position and the collision shell mesh, the pose matrix of each link corresponding to the geometry to be detected is determined, including:

[0025] Determine the target radian vector of the left arm link angle and the target radian vector of the right arm link angle corresponding to the target spatial position;

[0026] Based on the target radian vector of the left arm link angle and the target radian vector of the right arm link angle, the initial configuration vector corresponding to the dual-arm robot is updated to obtain the target configuration vector;

[0027] Based on the collision shell mesh, the geometry to be detected corresponding to each link is determined;

[0028] Based on the target configuration vector, the pose matrix of the geometry to be detected corresponding to each link is determined.

[0029] Optionally, based on the collision detection results, a control output signal is generated, including:

[0030] When the collision detection result is a collision, the control state of the dual-arm robot is adjusted from the normal operation state to the collision lock state;

[0031] The control output signal is generated in the collision lock state.

[0032] Optionally, in the collision lock state, generating the control output signal includes:

[0033] In the collision-locked state, the path from the current position to the target position is determined to be collision-free by using joint space interpolation path;

[0034] If there is no collision on the path, the control state is adjusted from the collision lock state to the tracking recovery state, and the control output signal is a progressive tracking signal.

[0035] Optional, also includes:

[0036] Visualization tools are used to render and display the robot's global pose, the motion trajectory of each link, and the collision status, highlighting the detected collision geometry in a preset color.

[0037] Secondly, embodiments of this application provide a collision processing apparatus, including a receiving module, a determining module, a first generating module, and a second generating module:

[0038] The receiving module is used to receive control commands from the dual-arm robot, the control commands being used to instruct the dual-arm robot to move to a target spatial position.

[0039] The determining module is used to determine the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot according to the control command.

[0040] The first generation module is used to determine the spatial position of the target based on the collision shell mesh and the plurality of target collision pairs, and generate a collision detection result;

[0041] The second generation module is used to generate a control output signal based on the collision detection result, and the control output signal is used to drive the dual-arm robot to perform a collision-free operation.

[0042] Optionally, the determining module is specifically used for:

[0043] According to the control command, the three-dimensional mesh model and structural description file of the dual-arm robot are obtained. The structural description file includes the robot structure, link names and joint types.

[0044] The three-dimensional mesh model is subjected to voxel dilation to generate the collision shell mesh corresponding to the dual-arm robot;

[0045] Based on the structural description file, multiple target collision pairs corresponding to the dual-arm robot are determined.

[0046] Optionally, the determining module is specifically used for:

[0047] The three-dimensional mesh model of the dual-arm robot is subjected to voxel conversion processing to obtain the three-dimensional voxel matrix of the dual-arm robot;

[0048] The three-dimensional voxel matrix of the dual-arm robot is dilated to obtain an dilated Boolean matrix;

[0049] The collision shell mesh is obtained by performing cube reconstruction on the expanded Boolean matrix.

[0050] Optionally, the determining module is specifically used for:

[0051] Based on the geometric information of each link and the joint relationships in the structural description file, multiple initial collision pairs are generated;

[0052] By using a three-layer filtering rule, invalid collision pairs are filtered out from the multiple initial collision pairs to obtain multiple target collision pairs. The invalid collision pairs include collision pairs excluded by fixed installation contact or geometric constraints.

[0053] Optionally, the dual-arm robot includes multiple links; the first generation module is specifically used for:

[0054] Based on the target spatial position and the collision shell mesh, determine the pose matrix of the geometry to be detected corresponding to each link. The pose matrix is ​​used to indicate the spatial position and orientation of the geometry to be detected.

[0055] Based on the pose matrix of each link corresponding to the geometry to be detected, collision detection processing is performed on the multiple target collision pairs to obtain the collision detection results.

[0056] Optionally, the plurality of links include a base link, a left arm link, and a right arm link; the first generation module is specifically used for:

[0057] Determine the target radian vector of the left arm link angle and the target radian vector of the right arm link angle corresponding to the target spatial position;

[0058] Based on the target radian vector of the left arm link angle and the target radian vector of the right arm link angle, the initial configuration vector corresponding to the dual-arm robot is updated to obtain the target configuration vector;

[0059] Based on the collision shell mesh, the geometry to be detected corresponding to each link is determined;

[0060] Based on the target configuration vector, the pose matrix of the geometry to be detected corresponding to each link is determined.

[0061] Optionally, the second generation module is specifically used for:

[0062] When the collision detection result is a collision, the control state of the dual-arm robot is adjusted from the normal operation state to the collision lock state;

[0063] The control output signal is generated in the collision lock state.

[0064] Optionally, the second generation module is specifically used for:

[0065] In the collision-locked state, the path from the current position to the target position is determined to be collision-free by using joint space interpolation path;

[0066] If there is no collision on the path, the control state is adjusted from the collision lock state to the tracking recovery state, and the control output signal is a progressive tracking signal.

[0067] Optionally, the apparatus further includes a rendering module, the rendering module being used for:

[0068] Visualization tools are used to render and display the robot's global pose, the motion trajectory of each link, and the collision status, highlighting the detected collision geometry in a preset color.

[0069] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the method described in the first aspect above and various possible designs of the first aspect.

[0070] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in the first aspect and various possible designs of the first aspect.

[0071] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect and various possible designs of the first aspect.

[0072] The object control method, apparatus, and storage medium provided in this application can generate tokens; sign the tokens using an encryption algorithm to obtain signed tokens, and send the signed tokens to a remote control device; receive control requests from the remote control device after verifying the validity of the time-limited token; and execute control instructions corresponding to the control requests. This allows the local control device to authorize the remote control device, enabling only authorized remote control devices to control local objects, thus improving the reliability of object control. Attached Figure Description

[0073] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0074] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0075] Figure 2 A schematic flowchart illustrating a collision handling method provided in an embodiment of this application;

[0076] Figure 3 A schematic flowchart illustrating another collision handling method provided in an embodiment of this application;

[0077] Figure 4 A schematic diagram of a control state provided for an embodiment of this application;

[0078] Figure 5 A schematic diagram illustrating a collision display provided in an embodiment of this application;

[0079] Figure 6 A schematic diagram of a collision shell mesh provided in an embodiment of this application;

[0080] Figure 7 This is a schematic diagram of the architecture of a collision handling method provided in an embodiment of this application.

[0081] Figure 8 A schematic diagram of a collision handling device provided in an embodiment of this application.

[0082] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0083] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0084] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0085] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. Please refer to [link / reference]. Figure 1 This application can be applied to scenarios such as remote surgery, industrial production lines, hazardous environment operations, and remote collaboration. These scenarios may include a remote control device 101 and a dual-arm robot 102, wherein the dual-arm robot 102 may include a left arm, a right arm, and a base. The remote control device 101 can be a handle, VR device, force feedback device, etc. An operator can control the dual-arm robot 102 to perform tasks in real time through the remote control device 101.

[0086] The remote control device 101 can receive control commands input by the operator, which are used to instruct the dual-arm robot 102 to move to the target spatial position. The remote control device 101 determines the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot 102 according to the control commands, and performs collision judgment on the target spatial position based on the collision shell mesh and multiple target collision pairs, generating collision detection results.

[0087] Based on the collision detection results, the remote control device 101 adjusts the control state of the control command through the state machine response strategy, generates a control output signal based on the control state, and sends the control output signal to the dual-arm robot 102 to drive the dual-arm robot 102 to perform collision-free operations.

[0088] In existing technologies, when a dual-arm robot is detected approaching other parts of itself, the control reduces its movement speed to give the operator more reaction time. It mainly relies on the operator for obstacle avoidance, which is limited by human perception and attention, thus affecting the reliability of dual-arm robot control.

[0089] The collision handling method provided in this application embodiment can receive control commands from a dual-arm robot. These commands instruct the robot to move to a target spatial position. Based on the control commands, the method determines the collision shell mesh corresponding to the robot and multiple target collision pairs. It then performs a collision judgment on the target spatial position based on the collision shell mesh and the multiple target collision pairs, generating a collision detection result. Based on the collision detection result, it adjusts the control state of the control commands using a state machine response strategy. Finally, it generates a control output signal based on the control state, which drives the dual-arm robot to perform collision-free operations. During this process, a collision judgment is performed on the target position before the control commands are executed. If a collision risk is detected, the command is rejected, execution is paused, or the command path is adjusted. This proactively avoids physical collisions and improves the reliability of the dual-arm robot control.

[0090] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0091] Figure 2 This is a schematic flowchart illustrating a collision handling method provided in an embodiment of this application. Please refer to... Figure 2 The method may include:

[0092] S201, Receive control commands from the dual-arm robot.

[0093] The execution subject of this application embodiment can be a remote control device or a collision processing device installed in the remote control device. The collision processing device can be implemented by software or by a combination of software and hardware.

[0094] A dual-arm robot may include multiple links, which can be divided into base links, left arm links, and right arm links.

[0095] Control commands can be used to instruct the dual-arm robot to move to a target spatial position. These control commands can include radian vectors of the left arm link angle and the right arm link angle.

[0096] The spatial attitude and position of each link in the left arm can be determined by the radian vector of the angle of the left arm link; the spatial attitude and position of each link in the right arm can be determined by the radian vector of the angle of the right arm link.

[0097] For example, a dual-arm robot has 7 links in each arm. The radian vector of the left arm link angle can be [0.1, -0.5, 0.3, -1.2, 0.0, 0.8, 0.0], and the radian vector of the right arm link angle can be [-0.2, 0.4, -0.3, 1.0, 0.1, -0.6, 0.0]. Each set of 7-dimensional radian vectors corresponds to the rotation angle of the 7 links, which can be converted into spatial coordinates through forward kinematics calculations.

[0098] S202. Based on the control instructions, determine the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot.

[0099] The collision shell mesh is a simplified 3D mesh optimized for collision detection. It reduces geometric complexity while preserving the collision boundary features of objects, resulting in a lightweight collision boundary shell.

[0100] A collision pair is a geometric combination of different components of a dual-arm robot that may collide, including combinations such as the left arm link and the right arm link, the left arm link and the base link, and the right arm link and the base link.

[0101] Dual-arm robots have a large number of collision pairs, but most of them are invalid collision pairs. Invalid collision pairs can be fixed installation contact parts between the link and the base, or long-distance link pairs that cannot make contact in physical space under geometric constraints.

[0102] Multiple target collision pairs are valid collision pairs retained after filtering out invalid collision pairs.

[0103] In some embodiments, the three-dimensional mesh model and structural description file of the dual-arm robot can be obtained according to control commands; voxel dilation processing can be performed on the three-dimensional mesh model to generate the collision shell mesh corresponding to the dual-arm robot; and multiple target collision pairs corresponding to the dual-arm robot can be determined according to the structural description file.

[0104] Three-dimensional mesh models are 3D mesh data used to indicate the pure geometry of a dual-arm robot. Examples include stereolithography (STL) meshes, object (OBJ) meshes, and 3D manufacturing format (3MF).

[0105] Structural description files are semantic descriptions of the entire dual-arm robot. Examples include Unified Robot Description Format (URDF) files.

[0106] In this application, a high-precision collision shell mesh can be generated by voxel dilation, and effective target collision pairs can be screened by combining the semantic rules of the structural description file. This reduces the detection information before collision judgment and can improve the efficiency of collision detection.

[0107] S203. Based on the collision shell mesh and multiple target collision pairs, perform collision judgment on the spatial position of the target and generate collision detection results.

[0108] It can detect whether there is a collision between the links of a dual-arm robot during its movement and when it reaches the target spatial position.

[0109] The collision detection results are used to indicate whether a collision will occur and the potential collision location. For example, if a collision will occur between target collision pairs, a Boolean value True (collision) and a potential collision link identifier will be output; if a collision will not occur between target collision pairs, False (no collision) and the shortest safe distance will be output.

[0110] Specifically, based on the target's spatial location and the collision shell mesh, the pose matrix of the geometry to be detected corresponding to each link is determined; based on the pose matrix of the geometry to be detected corresponding to each link, collision detection processing is performed on multiple target collision pairs to obtain the collision detection results.

[0111] The geometry to be detected is a simplified 3D mesh corresponding to the link in the collision shell mesh, which is obtained by splitting the collision shell mesh according to the link semantics, preserving the collision boundary features and being lightweight.

[0112] The pose matrix can be used to indicate the spatial position and orientation of the geometry to be detected, and can include translation parameters (x, y, z coordinates) and rotation parameters (such as Euler angles and quaternions).

[0113] In this application, collision judgment of target collision pairs can be achieved through simplified collision detection between the geometric objects to be detected, thereby improving detection efficiency.

[0114] S204. Generate a control output signal based on the collision detection results.

[0115] The control output signal can be used to drive the dual-arm robot to perform collision-free operations, ensuring the continuity and safety of remote operation.

[0116] Specifically, when the collision detection result is a collision, the control state of the dual-arm robot is adjusted from the normal operation state to the collision lock state; in the collision lock state, a control output signal is generated.

[0117] Control states can include idle state, normal state, collision locked state, and tracking recovery state.

[0118] The idle state indicates the initial state of the dual-arm robot, waiting to receive control commands from the dual-arm robot.

[0119] The entry condition for the idle state is that the dual-arm robot has completed its startup initialization, or that no valid control command has been received for a long time (e.g., 5 seconds); the exit condition is that a valid control command for the dual-arm robot has been received; the output behavior is to keep the current position of each link stationary and initialize the collision detection parameters.

[0120] The normal operating status indicates that the dual-arm robot's movements are safe and that it is executing operator commands normally.

[0121] The entry condition for normal operation is to enter from the idle state (i.e., receive a valid control command) or from the tracking recovery state (i.e., the error between the actual position and the target position is less than a preset threshold, such as 0.1mm); the exit condition is that the collision detection result is a collision, or a stop command is received; the output behavior is to directly forward the operator's control command and update the "last safe position".

[0122] The collision lock status is used to indicate that the dual-arm robot has detected a collision risk, suspends the execution of commands that may lead to a collision, and triggers safety protection.

[0123] The entry condition for the collision lock state is either from the normal operation state (i.e., a collision risk is detected at the target position or trajectory) or from the tracking recovery state (i.e., a new collision risk is detected during tracking). The exit condition is that a collision-free path is regenerated through the path planning algorithm, and the path safety verification is passed (the collision shell mesh and the target collision pair confirm no collision). The output behavior is to send the control command corresponding to the "last safe position", calculate the smooth recovery speed through the PD control algorithm, drive the robot to stay in the safe position and maintain a stable posture, and at the same time, send collision warning information back to the remote control device.

[0124] The tracking recovery status is used to indicate that after the collision is cleared, the dual-arm robot gradually tracks the operator's target position.

[0125] The entry condition for tracking recovery state is to enter from collision lock state (path safety verification passed, and the new path has no collision risk); the exit condition is to switch to normal operation state when the error between the actual position and the target position is less than a preset threshold (e.g., 0.1mm), or to switch to collision lock state when a new collision risk is detected; the output behavior is based on the replanned safe path, sending segmented target position commands, limiting the movement speed to 30% of the safe speed, and dynamically adjusting the path in real time based on the collision detection results to avoid collisions caused by rapid movement.

[0126] A four-state finite state machine (FSM) can be used to dynamically adjust the control strategy based on the collision detection results, control state, and actual pose of the dual-arm robot, thereby achieving functions such as collision handling, collision locking, path safety verification, and tracking recovery.

[0127] In this application, a closed-loop control logic of a four-state finite state machine is used to dynamically switch control states based on collision detection results. Combined with path planning and PD control algorithms, a full-process safety control is achieved, including proactive warning before collision, rapid locking during collision, and smooth recovery after collision. This improves the safety of remote operation of the dual-arm robot, the continuity of command execution, and the operator's control experience.

[0128] The collision handling method provided in this application can receive control commands containing the angle vectors of the dual-arm linkages, generate a lightweight collision shell mesh based on the 3D mesh model of the dual-arm robot, and obtain effective target collision pairs through semantic rule filtering combined with the robot's structural description file. It performs real-time, high-precision collision judgment on the robot's motion trajectory and the target's spatial position, generating detection results. Then, it adjusts the control state to generate a control output signal adapted to the current state, driving the dual-arm robot to perform collision-free operations. In the above execution process, not only is the target position predicted before the control command is executed, but collision risks are also monitored in real time during movement. If a collision risk is detected, the current operation can be quickly locked and a safe path can be replanned. After the collision is resolved, the target trajectory is smoothly restored, rather than simply relying on operator obstacle avoidance or passive deceleration. Simultaneously, through the lightweight design of the collision shell mesh and the semantic filtering of target collision pairs, the collision judgment efficiency is improved while ensuring detection accuracy, achieving a balance between high precision and high efficiency. This actively avoids physical collisions while ensuring the continuity and stability of remote operation, improving the reliability of dual-arm robot control, operational safety, and the operator's control experience.

[0129] Figure 3 This is a schematic flowchart illustrating another collision handling method provided in an embodiment of this application. Please refer to... Figure 3 The method includes:

[0130] S301: Receive control commands from the dual-arm robot.

[0131] The execution process of S301 can be found in the execution process of S201, and will not be repeated here.

[0132] S302. According to the control instructions, obtain the three-dimensional mesh model and structural description file of the dual-arm robot.

[0133] The structural description file may include the robot's link structure, link names, and joint types. Joint types include fixed joints, rotary joints, and kinetic joints.

[0134] The 3D mesh model can be pre-stored in the local storage module of the remote control device or in the associated industrial cloud server.

[0135] The structural description file is a standardized file predefined when the robot leaves the factory. It can be pre-written into the robot control algorithm package of the remote control device and directly loaded into memory after the system starts, serving as the basis for kinematic modeling. Alternatively, the remote control device can send a file retrieval command to the main control unit of the dual-arm robot through the industrial Ethernet / ROS communication protocol to obtain the latest structural description file in real time (adapting to scenarios after joint parameter calibration and link structure fine-tuning).

[0136] S303. Perform voxel dilation on the 3D mesh model to generate the collision shell mesh corresponding to the dual-arm robot.

[0137] Specifically, the three-dimensional mesh model of the dual-arm robot is subjected to voxel conversion to obtain the three-dimensional voxel matrix of the dual-arm robot; the three-dimensional voxel matrix of the dual-arm robot is dilated to obtain the dilated Boolean matrix; and the dilated Boolean matrix is ​​reconstructed into a cube to obtain the collision shell mesh.

[0138] When performing voxelization on the 3D mesh model of the dual-arm robot, the voxel resolution (pitch) can be dynamically adjusted according to the task type or environmental complexity, with an initial default value of 5mm.

[0139] Task type triggering: The task type is identified by the task identifier field (such as "precision_assembly" or "rough_grabbing") carried by the control command issued by the operator, or by obtaining the task type through the system's preset task scenario selection interface. Different task types correspond to different precision requirements.

[0140] For example, precision tasks (such as remote surgery and precision assembly) require high-precision collision detection, with the evaluation metric being "allowable collision gap ≤ 3mm"; while rough tasks (such as large-item handling and rough gripping) prioritize detection efficiency, with the evaluation metric being "single-frame collision detection time ≤ 0.5ms".

[0141] Environmental complexity trigger: The environmental complexity is automatically assessed by statistically analyzing the environmental obstacle density index during real-time collision detection. The specific calculation method is as follows: within the bounding box of the robot's working space, the number of obstacle collision shell mesh voxels per unit volume (e.g., 1m³) is counted.

[0142] For example, in a high-density environment: the number of prime objects per unit volume is ≥5000, so the resolution needs to be increased to avoid missing small obstacles; in a medium-density environment: the number of prime objects per unit volume is 2000-5000, so the default resolution is maintained; in a low-density environment: the number of prime objects per unit volume is <2000, so the resolution can be reduced to improve efficiency.

[0143] When the task type changes or the environmental complexity index changes beyond the threshold (e.g., the number of prime objects per unit volume of obstacles fluctuates by ≥1000), the voxel resolution is updated in real time and the collision shell mesh is regenerated (this process is completed within 10ms and does not affect the continuous operation of the robot).

[0144] After determining the voxel resolution, the bounding box of the 3D mesh model is further determined. A regular 3D mesh is then divided within the bounding box, with each mesh having a side length of one pitch. The geometric center of this mesh is used as the voxel center. It is then determined whether each voxel center is inside the 3D mesh model (using ray casting or signed distance field algorithms). The ray casting or signed distance field (SDF) algorithm is used to determine whether each voxel center is inside the 3D mesh model.

[0145] Ray method: Emit rays from the center of the voxel in any direction and count the number of intersections between the rays and the mesh facets. If the number is odd, it is determined to be inside; if the number is even, it is determined to be outside.

[0146] Symbolic distance field algorithm: The symbolic distance field of the 3D mesh is pre-calculated. If the symbolic distance value corresponding to the voxel center is less than or equal to 0, it is determined to be inside; if it is greater than 0, it is determined to be outside.

[0147] Based on the above judgment results, a corresponding three-dimensional voxel matrix is ​​generated, with internal voxels marked as True (occupied) and external voxels marked as False (empty).

[0148] The formula for calculating the dilation radius when dilating a three-dimensional voxel matrix is ​​as follows:

[0149]

[0150] Where radius_vox is the dilation radius, ceil() is the round-up function, offset is the dilation offset, and pitch is the voxel resolution. For example, if offset=5mm and pitch=5mm, then radius_vox=ceil(5 / 5)=1 voxel; if offset=7mm and pitch=5mm, then radius_vox=ceil(7 / 5)=2 voxels.

[0151] Typically, a 3D cross (6-connected) or a 3D cube (26-connected) is used to generate the dilated structuring element corresponding to the 3D voxel matrix. Using the dilated structuring element as a template, the 3D voxel matrix is ​​iteratively dilated for a number of iterations, radius_vox. Binarization dilation is then applied to the 3D voxel matrix, setting all neighboring voxels of each True voxel to True, resulting in a dilated Boolean matrix.

[0152] The Marching Cubes algorithm iterates through the eight vertices of each voxel in the dilated Boolean matrix. Based on the vertex's internal / external state (True / False), it generates one of 256 cube configurations. Linear interpolation is used to determine the precise position of the triangle facet vertices. Duplicate faces are removed and the topology is optimized. All triangle facets are assembled to generate a continuous mesh surface, ultimately resulting in a lightweight collision shell mesh that retains safe boundaries.

[0153] In this application, a continuous 3D mesh is transformed into a computable Boolean matrix through voxel discretization. Combined with configurable dilation offsets, precise control of the safety margin is achieved. Then, a traveling cubes algorithm is used to reconstruct the 3D mesh of the dilated Boolean matrix, enabling visualization and standardization of the collision detection safety boundary. This balances collision detection accuracy and computational efficiency, improving the accuracy of collision prediction. Simultaneously, the lightweight collision shell mesh significantly reduces the computational power consumption for detection, enhancing the real-time performance of the overall collision detection system.

[0154] S304. Based on the structural description file, determine the multiple target collision pairs corresponding to the dual-arm robot.

[0155] Specifically, based on the geometric information of each link and the joint relationships in the structural description file, multiple initial collision pairs are generated; through a three-layer filtering rule, invalid collision pairs are filtered out from the multiple initial collision pairs to obtain multiple target collision pairs.

[0156] In the structural description file of a dual-arm robot, all link geometries can be combined in pairs to form multiple initial collision pairs, with the number of collision pairs reaching more than one hundred, but most of them are invalid collision pairs.

[0157] Invalid collision pairs can include collision pairs that are in fixed contact between links and collision pairs that are physically impossible to contact under geometric constraints.

[0158] By performing semantic analysis on the structure description file, and combining it with a machine learning-based semantic filtering enhancement model, invalid collision pairs can be automatically identified and excluded, retaining only a preset number of valid collision pairs as target collision pairs.

[0159] The training dataset for the machine learning model is constructed as follows: URDF files of multiple robot models and corresponding measured / simulated collision data are collected. Invalid collision pairs are labeled using rule-based pre-annotation, simulation verification, and manual sampling verification. Training / verification / test sets are divided to train a binary classification model to improve the cross-model adaptability and recognition accuracy of semantic filtering.

[0160] When filtering out invalid collision pairs from multiple initial collision pairs, the following three-layer filtering rules can be executed:

[0161] The first layer of filtering rules is named prefix classification filtering. Based on the named prefixes of links and geometries in the structural description file, they are classified into base geometry, left arm geometry, and right arm geometry, thus clarifying the main scope of collision detection.

[0162] For example, the base geometry is named starting with "base_link" (e.g., base_link, base_link_collision, etc.); the left arm geometry is named starting with "Link_L" (e.g., Link_L1, Link_L2, ..., Link_L7, Link_L8, etc.); and the right arm geometry is named starting with "Link_R" (e.g., Link_R1, Link_R2, ..., Link_R7, Link_R8, etc.).

[0163] It can be compatible with different naming styles (such as Link_Left_3, LeftArm_Joint3, etc.) through configuration files without modifying the core filtering logic.

[0164] The second layer of filtering rules is fixed-contact collision pair filtering. It excludes geometric pairs that are physically in permanent contact and will not undergo relative motion.

[0165] Specifically, initial collision pairs connected by a fixed joint (type="fixed") (such as the fixed connection between a link and a base) and without relative motion are identified as invalid collision pairs; link pairs connected by a revolute joint (type="revolute") are not included in the L2 exclusion scope if there is a possibility of dynamic motion.

[0166] The third filtering rule is geometric constraint collision pair filtering. Based on robot kinematic constraints and geometric features, it excludes geometric pairs that cannot collide in physical space, including: invalid collision pairs between the base and the distal link, scene-based exclusion of the end effector, and invalid collision pairs between distant links on the same side.

[0167] Invalid collision pairs between the base and the distal link are used to indicate collision pairs where the minimum movement distance between a link of the dual-arm robot and the base is much greater than the thickness of the collision shell (5-10mm).

[0168] For example, base_link and Link_L2 / Link_R2: the second link (Link_L2 / Link_R2) is limited by the first link within its range of motion and cannot reach the base (typical minimum distance > 200mm), so it is determined to be an invalid collision pair.

[0169] Collision pair exclusion rules for end effectors (such as grippers) can be configured as needed: enable exclusion for precision assembly tasks (to avoid gripper collisions), and disable exclusion for collaborative gripping tasks (to allow grippers to approach); Note: Contact between the end effector's own links (such as Link_L7 / L8) on the same side arm is not within the scope of cross-arm / base collision detection and is included as an invalid collision pair by default.

[0170] The invalid collision pair of the same-side far-distance link is the far-distance link of the same side arm. Due to the physical constraints of the link length and joint angle limit, self-collision cannot occur within the range of motion.

[0171] Regarding the constraint on the length of the link, adjacent links on the same side arm are connected in series. If the number of intervals between two links is greater than or equal to a preset threshold (such as 4 links), then the total length of the link between the two links is much greater than the length of a single link, making it difficult for spatial overlap to occur in the mechanical structure.

[0172] Regarding the joint angle limit constraint, each rotary joint has a physical angle limit (such as ±180°, ±90°). When the interval is ≥4 joints, the cumulative deflection of the joint angle cannot make the distal link touch the proximal link, and there is no possibility of collision geometrically.

[0173] The preset threshold (e.g., 4 links) needs to be verified by forward kinematics traversal in conjunction with the robot's kinematics model: traverse the feasible domain of all joint angles of the side arm (sampling within the joint limit range), calculate the minimum distance of link pairs with different interval numbers in all postures; if the minimum distance of all postures is greater than the collision shell thickness (e.g., 5-10mm) at a certain interval number, then the interval number is determined to be a valid threshold, and the initial collision pairs with an interval ≥ the threshold are marked as invalid collision pairs; for different models of dual-arm robots, the above verification can be completed in batches through automated scripts to adapt to mechanical structures with different link lengths and joint limits.

[0174] Therefore, initial collision pairs with a gap between the two links being greater than or equal to a preset threshold (e.g., 4 links) are considered invalid collision pairs. This threshold is determined through kinematic ergonomic verification and can be adapted to the physical structural constraints of different robot models.

[0175] After filtering out invalid collision pairs, only the core detection combinations are retained among multiple target collision pairs: the collision pair consisting of the left arm link and the right arm link, the collision pair consisting of the left arm link and the base link, and the collision pair consisting of the right arm link and the base link.

[0176] Taking a robot with 7 links each in the left and right arms and 1 link in the base as an example: there are 49 initial collision pairs between the left and right arm links. After geometric constraints, there are approximately 15-20 target collision pairs. There are 7 initial collision pairs between the left arm and the base links. After excluding Link_L1 / L2 (which are fixed to the base / have no collision risk at close range), there are approximately 3-5 target collision pairs. There are 7 initial collision pairs between the right arm and the base links. After excluding Link_R1 / R2, there are approximately 3-5 target collision pairs. Finally, there are approximately 18 effective target collision pairs (the specific number depends on the robot model, ranging from 15-25 pairs).

[0177] In this application, the number of collision pairs can be reduced from more than 100 to 15-25 pairs. There is no need for manual calibration of collision rules. Through three layers of filtering rules, the semantic information of the structure description file is parsed to automatically generate target collision pairs. The filtering logic is compatible with structure description files with different naming conventions, which can be adapted to various dual-arm robots, greatly simplifying the collision detection objects and improving the efficiency of subsequent collision detection.

[0178] S305. Based on the target spatial position and the collision shell mesh, determine the pose matrix of the geometry to be detected corresponding to each link.

[0179] Specifically, the target radian vectors of the left and right arm link angles corresponding to the target spatial position are determined; based on the target radian vectors of the left and right arm link angles, the initial configuration vectors corresponding to the dual-arm robot are updated to obtain the target configuration vectors; based on the collision shell mesh, the geometry to be detected corresponding to each link is determined; based on the target configuration vectors, the pose matrix of the geometry to be detected corresponding to each link is determined.

[0180] The initial configuration vector may include the base free-floating parameters, the initial radian vector corresponding to the angle of the left arm link, and the initial radian vector corresponding to the angle of the right arm link.

[0181] The target configuration vector may include the base free-floating parameters, the target radian vector of the left arm link angle, and the target radian vector of the right arm link angle.

[0182] The free-floating parameters of the base are used to indicate the pose of the base in the world coordinate system. They include three translation parameters (x / y / z, unit: m) and four rotation parameters (quaternions w / x / y / z, representing the attitude). They are used to describe the global spatial position and attitude of the base and serve as the reference reference for calculating the pose of all links of the robot.

[0183] For example, taking a dual-arm robot with 7 links in the left and right arms as an example, the initial configuration vector q (length: 7+14=21) is used, where q[0:7] is the base free-floating parameter (3 positions and 4 quaternions, initialized to unit transformation), q[7:14] is the angle (in radians) of the 7 links of the left arm, and q[14:21] is the angle (in radians) of the 7 links of the right arm. After replacing the initial radian vector of q[7:14] with the target radian vector of the left arm link angle, and replacing the initial radian vector of q[14:21] with the target radian vector of the right arm link angle, the target configuration vector is obtained.

[0184] When determining the pose matrix of each link corresponding to the geometry to be detected based on the target configuration vector, the transformation matrix of each link can be recursively calculated from the base along the rigid body tree model, and the global pose matrix of each link can be obtained by accumulating the results.

[0185] Wherein, the global pose matrix of the geometry to be detected = the global pose matrix of the link × the offset matrix of the geometry in the local coordinate system of the link.

[0186] In this application, the recursive forward kinematics of the rigid body tree model can be used to unify the global coordinate system reference by combining the free floating parameters of the base, and to associate the link angle vector with the spatial pose of the geometry to be detected, so as to realize the real-time and high-precision calculation of the pose of each geometry in the collision shell mesh, thereby improving the accuracy of collision detection.

[0187] S306. Based on the pose matrix of each link corresponding to the geometry to be detected, perform collision detection processing on multiple target collision pairs to obtain collision detection results.

[0188] Any of the following inspection modes can be used:

[0189] Fast detection mode: Based on the pose matrix of the geometry to be detected corresponding to each link, multiple target collision pairs are sequentially processed for collision detection. When a sub-detection result of a target collision pair is detected as a collision, the detection is terminated and the collision detection result is determined as the collision result.

[0190] Full detection mode: Based on the pose matrix of the geometry to be detected corresponding to each link, collision detection is performed on each target collision pair to obtain the sub-detection result of each target collision pair; if any sub-detection result is a collision, then the collision detection result is determined as the collision result.

[0191] It can record the identification of all collision pairs, collision risk level, and shortest distance, making it suitable for debugging, visualization, and path planning.

[0192] In this application, the fast detection mode, which immediately exits the loop upon detecting the first collision, can meet the low latency requirements of real-time safety checks; the full detection mode can collect the names, locations, and distances of all colliding geometries for system debugging, collision risk visualization, and subsequent path optimization.

[0193] For any target collision pair, the target collision pair consists of two geometries to be detected, denoted as the first geometries to be detected and the second geometries to be detected.

[0194] The pose matrix of the first and second geometries to be detected can be obtained. Based on the pose matrix of the first and second geometries to be detected, it is determined whether the first and second geometries to be detected intersect. If they do, the sub-detection result of the target collision pair is the collision result; if not, the sub-detection result of the target collision pair is the non-collision result.

[0195] The Gilbert-Johnson-Keerthi (GJK) algorithm can be used to determine whether the first and second geometries intersect.

[0196] Before determining whether they intersect, based on the pose matrices of the first and second geometries to be detected, the vertex coordinates of the two geometries are uniformly transformed to the world coordinate system to obtain normalized geometric vertex sets P1 (first geometries) and P2 (second geometries). Using P1 and P2 as inputs to the GJK algorithm, the Minkowski difference between the two geometries is iteratively calculated to determine whether the origin is included.

[0197] If the Minkowski difference includes the origin, it means that the two geometric objects intersect. The sub-detection result of the target collision pair is the collision result, and the shortest penetration distance between the two geometric objects is output (characterizing the severity of the collision).

[0198] If the Minkowski difference does not include the origin, it means that the two geometries do not intersect. The sub-detection result of the target collision pair is a non-collision result. At the same time, the shortest safe distance between the two geometries is output (representing the collision risk level, such as <5mm for high risk, 5-10mm for medium risk, and >10mm for low risk).

[0199] In this application, for any target collision pair, collision judgment can be completed by iteratively solving the convex hull of the Minkowski difference. Combined with the accurate coordinate normalization of the pose matrix, high-precision and low-computational-power collision detection of target collision pairs can be achieved. At the same time, it supports flexible switching between fast and full detection modes, which not only meets the real-time requirements of remote operation scenarios, but also takes into account the visualization needs of the debugging stage, improving the efficiency and practicality of collision detection, reducing the system's computing power consumption, and improving the accuracy of collision risk prediction.

[0200] S307. When the collision detection result is a collision, adjust the control state of the dual-arm robot from the normal operation state to the collision lock state.

[0201] Under normal operating conditions, when a collision is detected at the target position or on the trajectory, the name of the colliding geometry (such as {"Link_L3","Link_R4"}) can be recorded, and the control state can be changed from NORMAL to LOCKED.

[0202] Figure 4 This is a schematic diagram illustrating a control state provided in an embodiment of this application. Please refer to [link / reference]. Figure 4 The control states include Idle, Normal, Locked, and Chasing.

[0203] Upon completion of initialization or if no valid control command is received for an extended period (e.g., 5 seconds), the system enters an idle state (IDLE). Upon receiving a valid control command, the system transitions from the idle state (IDLE) to the normal operating state (NORMAL).

[0204] When a collision is detected, the system transitions from the normal state to the collision locked state; when a stop command is received, the system transitions from the normal state to the idle state.

[0205] After the path safety verification is passed (collision-free path replanned), the system transitions from the collision locked state to the tracking recovery state. When the error between the actual position and the target position is less than a preset threshold (i.e., close to the target position), the system transitions from the tracking recovery state to the normal operation state; when a new collision risk is detected, the system transitions from the tracking recovery state to the collision locked state.

[0206] S308. In the collision lock state, generate a control output signal.

[0207] In collision-locked mode, the operator's dangerous commands are rejected, the robot will not perform collision actions, and the robot will remain or retreat to the last safe position, notifying the operator of the collision risk through haptic feedback (VR controller vibration, sound and light alarms).

[0208] The smooth recovery speed can be calculated based on the PD control algorithm, and the controller can make the robot smoothly return to a safe position, avoiding posture jitter caused by sudden stops.

[0209] In some embodiments, the absence of collisions at the target location does not necessarily mean that it can be reached safely. If there are collision points on the path from the current location to the target location, the operation still cannot be performed.

[0210] Specifically, in the collision-locked state, the path from the current position to the target position is determined by interpolating the path in the joint space to determine whether there is a collision; if there is no collision, the control state is adjusted from the collision-locked state to the tracking recovery state, and the control output signal is a progressive tracking signal.

[0211] The maximum link angle change of the left and right arms and the number of interpolation steps can be determined (initially 0.1 radians / step, which can be dynamically adjusted). Collision detection is performed on the interpolated posture at each step. If a collision is detected at any step, the path is unsafe and the LOCKED state is maintained. If there is no collision at all steps, the path is safe and the CHASING state is entered.

[0212] The interpolation step size can be initially set to 0.1 radians / step. If the step size is too large (e.g., >0.2 radians / step), it is easy to miss the risk of local collisions during the motion process. If the step size is too small (e.g., <0.05 radians / step), it will significantly increase the number of interpolation steps and detection computing power, affecting real-time performance.

[0213] 0.1 radians / step can cover the minimum collision gap (approximately 5-10 mm) of typical dual-arm robot joint movements. While ensuring the integrity of collision detection, the verification time for a single path is controlled within 1 ms, meeting the high-frequency control requirements of >1 kHz.

[0214] For example, the step size dynamic adjustment rule can be as follows: In high-precision scenarios (such as remote surgery and precision assembly): the step size is reduced to 0.05 radians / step to improve collision detection accuracy; In high-speed transportation scenarios: the step size is increased to 0.15 radians / step to optimize verification efficiency; When a path collision is detected for the first time, the step size is halved (e.g., from 0.1 radians / step to 0.05 radians / step) to perform refined re-verification of the collision area and avoid misjudgment.

[0215] Even if the target location is safe, if there is a collision on the path, the attempt is still rejected. The entire trajectory is verified in the joint space to ensure safe arrival.

[0216] The raw speed can be extracted from operator instructions and limited to 30% of the maximum safe speed, improving operational continuity while ensuring safety.

[0217] In each control cycle of the CHASING state, collision detection is still performed on the current motion trajectory and target position. If a collision is detected, the system immediately returns to the LOCKED state to achieve real-time safety protection.

[0218] It is worth noting that the MeshCatVisualizer visualization tool can be integrated to render and display the robot's global pose, the motion trajectory of each link, and the collision status in real time through a web browser. The detected collision geometry is highlighted in a preset color (e.g., red), which helps engineers debug system parameters and operators monitor the robot's operating status in real time.

[0219] In the visual model, the display rules for the robot's appearance mesh are as follows: gray in a normal, collision-free state (RGB values: R=0.7, G=0.7, B=0.7, transparency Alpha=1.0); red when a collision risk is detected (including already collided / potential collision) (RGB values: R=1.0, G=0.0, B=0.0, transparency Alpha=1.0). Please see [link to documentation]. Figure 5 , Figure 5 This is a schematic diagram of a collision display provided in an embodiment of this application.

[0220] The collision shell mesh is a safety boundary mesh generated after voxel dilation. Under normal conditions, it appears as a semi-transparent orange (RGB values: R=1.0, G=0.7, B=0.3, transparency Alpha=0.25). This semi-transparent rendering method preserves the visibility of the safety boundary without obscuring the robot's internal structure and operating environment, facilitating engineers' verification of the collision shell's fit with the actual geometry. Please see [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a collision shell mesh provided in an embodiment of this application.

[0221] The highlighting logic maintains three core geometry sets: the collision geometry set from the previous frame, the collision geometry set detected in the current frame, and the geometry set that has been restored to collision-free state.

[0222] The color switching rules are as follows: For geometry that has been restored to non-collision state, the color changes from red to the default gray; for geometry that is colliding in the current frame, the color changes from the default gray to red highlight; the collision shell mesh will only increase its transparency to Alpha=0.5 (while keeping the orange color unchanged) when the visual model associated with the corresponding link triggers red highlighting, thus enhancing the visual cue of the collision boundary.

[0223] Figure 7This is a schematic diagram illustrating the architecture of a collision handling method provided in an embodiment of this application. Please refer to [link / reference]. Figure 7 During collision shell generation (voxel dilation technique), the original STL mesh is discretized using voxels, and the safety boundary is expanded through a binarization dilation operation. Then, the lightweight collision shell mesh is reconstructed using the traveling cube algorithm. During collision pair filtering (intelligent semantic filtering), the link and joint semantic information of the URDF file is parsed, and the initial collision pairs are reduced from hundreds to 15-25 effective collision pairs through a three-layer filtering rule.

[0224] Real-time collision detection module (efficient kinematics calculation): Based on offline generated collision shells and valid collision pairs, combined with the current joint angle vector, the module calculates the pose of each link through forward kinematics, uses the GJK algorithm to perform real-time collision judgment on the target collision pairs, and outputs detection results including collision status, risk level and shortest distance, and generates safe joint angle and velocity commands.

[0225] In state machine management (collision response and recovery), based on a four-state finite state machine, the control state is dynamically switched according to the collision detection result: during normal operation, the command is forwarded; when a collision is detected, the device is locked to a safe position; after the path verification is passed, the target position is smoothly tracked, and finally a safe motor command is output.

[0226] During visualization debugging (real-time web visualization): The MeshCatVisualizer tool is integrated to render the robot's posture, collision shell, and collision status in real time through a web browser. The collision geometry is highlighted in red, and the collision shell is displayed in semi-transparent orange. At the same time, collision alarms are pushed to assist engineers in debugging and operators in monitoring.

[0227] The computationally intensive collision shell generation and collision pair filtering are performed only once in the offline stage, while the online stage retains only lightweight kinematic calculations and state machine logic to meet the high-frequency real-time control requirements of >1kHz. From geometric boundary expansion and invalid collision pair simplification to real-time collision detection, state machine response, and visual monitoring, a safety protection system is formed to ensure the reliability and continuity of the teleoperation of the dual-arm robot.

[0228] Figure 8 This is a schematic diagram of a collision handling device provided in an embodiment of this application. Please refer to... Figure 8 The collision processing device 800 includes a receiving module 801, a determining module 802, a first generating module 803, and a second generating module 804.

[0229] The receiving module 801 is used to receive control commands from the dual-arm robot, which are used to instruct the dual-arm robot to move to a target spatial position.

[0230] The determination module 802 is used to determine the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot according to the control instructions;

[0231] The first generation module 803 is used to perform collision judgment on the spatial position of the target based on the collision shell mesh and multiple target collision pairs, and generate collision detection results;

[0232] The second generation module 804 is used to generate a control output signal based on the collision detection result. The control output signal is used to drive the dual-arm robot to perform collision-free operations.

[0233] Optionally, module 802 is specifically used for:

[0234] Based on the control commands, obtain the 3D mesh model and structural description file of the dual-arm robot. The structural description file includes the robot structure, link names, and joint types.

[0235] Voxel dilation is performed on the 3D mesh model to generate the collision shell mesh corresponding to the dual-arm robot.

[0236] Based on the structural description file, identify multiple target collision pairs corresponding to the dual-arm robot.

[0237] Optionally, module 802 is specifically used for:

[0238] The three-dimensional mesh model of the dual-arm robot is converted into a voxel matrix to obtain the three-dimensional voxel matrix of the dual-arm robot.

[0239] The three-dimensional voxel matrix of the dual-arm robot is dilated to obtain the dilated Boolean matrix;

[0240] The collision shell mesh is obtained by performing cube reconstruction on the dilated Boolean matrix.

[0241] Optionally, module 802 is specifically used for:

[0242] Based on the geometric information of each link and the joint relationships in the structural description file, multiple initial collision pairs are generated;

[0243] By using a three-layer filtering rule, multiple initial collision pairs are filtered for invalid collision pairs to obtain multiple target collision pairs. Invalid collision pairs include those excluded by fixed installation contact or geometric constraints.

[0244] Optionally, the dual-arm robot includes multiple links; the first generation module 803 is specifically used for:

[0245] Based on the target spatial position and the collision shell mesh, determine the pose matrix of the geometry to be detected corresponding to each link. The pose matrix is ​​used to indicate the spatial position and orientation of the geometry to be detected.

[0246] Based on the pose matrix of each link corresponding to the geometry to be detected, collision detection is performed on multiple target collision pairs to obtain the collision detection results.

[0247] Optionally, the multiple links include a base link, a left arm link, and a right arm link; the first generation module 803 is specifically used for:

[0248] Determine the target radian vector of the left arm link angle and the target radian vector of the right arm link angle corresponding to the target spatial position;

[0249] Based on the target radian vectors of the left arm link angle and the right arm link angle, the initial configuration vector corresponding to the dual-arm robot is updated to obtain the target configuration vector.

[0250] Based on the collision shell mesh, determine the geometry to be detected for each link;

[0251] Based on the target configuration vector, determine the pose matrix of the geometry to be detected corresponding to each link.

[0252] Optionally, the second generation module 804 is specifically used for:

[0253] When the collision detection result indicates a collision, the control state of the dual-arm robot is adjusted from the normal operation state to the collision lock state;

[0254] In the collision lock state, a control output signal is generated.

[0255] Optionally, the second generation module 804 is specifically used for:

[0256] In collision-locked mode, the path from the current position to the target position is determined by interpolating the joint space path to determine whether there is a collision.

[0257] If there is no collision on the path, the control state is changed from collision lock state to tracking recovery state, and the control output signal is a progressive tracking signal.

[0258] Optionally, the device also includes a rendering module, which is used for:

[0259] Visualization tools are used to render and display the robot's global pose, the motion trajectory of each link, and the collision status, highlighting the detected collision geometry in a preset color.

[0260] The collision processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0261] Figure 9 A schematic diagram of the structure of an electronic device provided in this application. Please refer to [link / reference]. Figure 9The electronic device 900 may include at least one processor 901 and a memory 902. Exemplarily, the processor 901 and the memory 902 are interconnected via a bus 903.

[0262] Memory 902 stores instructions executed by the computer;

[0263] The processor 901 executes computer execution instructions stored in the memory 902, causing the processor 901 to perform the collision handling method as shown in the above method embodiment.

[0264] Accordingly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the collision handling method of the above-described method embodiments.

[0265] Accordingly, embodiments of this application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the collision processing method shown in the above method embodiments.

[0266] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0267] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable collision processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable collision processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0268] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable collision processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0269] These computer program instructions can also be loaded onto a computer or other programmable collision processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0270] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0271] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0272] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0273] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0274] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A collision handling method, characterized in that, include: Receive control commands from the dual-arm robot, the control commands being used to instruct the dual-arm robot to move to a target spatial location; According to the control command, the collision shell mesh and multiple target collision pairs corresponding to the dual-arm robot are determined; Based on the collision shell mesh and the plurality of target collision pairs, the spatial position of the target is determined to generate a collision detection result. Based on the collision detection results, a control output signal is generated, which is used to drive the dual-arm robot to perform collision-free operations.

2. The method according to claim 1, characterized in that, According to the control commands, the collision shell mesh corresponding to the dual-arm robot and multiple target collision pairs are determined, including: According to the control command, the three-dimensional mesh model and structural description file of the dual-arm robot are obtained. The structural description file includes the robot structure, link names and joint types. The three-dimensional mesh model is subjected to voxel dilation to generate the collision shell mesh corresponding to the dual-arm robot; Based on the structural description file, multiple target collision pairs corresponding to the dual-arm robot are determined.

3. The method according to claim 2, characterized in that, Voxel dilation is performed on the 3D mesh model to generate the collision shell mesh for the dual-arm robot, including: The three-dimensional mesh model of the dual-arm robot is subjected to voxel conversion processing to obtain the three-dimensional voxel matrix of the dual-arm robot; The three-dimensional voxel matrix of the dual-arm robot is dilated to obtain an dilated Boolean matrix; The collision shell mesh is obtained by performing cube reconstruction on the expanded Boolean matrix.

4. The method according to claim 2, characterized in that, Based on the structural description file, multiple target collision pairs corresponding to the dual-arm robot are determined, including: Based on the geometric information of each link and the joint relationships in the structural description file, multiple initial collision pairs are generated; By using a three-layer filtering rule, invalid collision pairs are filtered out from the multiple initial collision pairs to obtain multiple target collision pairs. The invalid collision pairs include collision pairs excluded by fixed installation contact or geometric constraints.

5. The method according to claim 1, characterized in that, The dual-arm robot includes multiple links; based on the collision shell mesh and the multiple target collision pairs, it performs collision judgment on the spatial position of the target and generates collision detection results, including: Based on the target spatial position and the collision shell mesh, determine the pose matrix of the geometry to be detected corresponding to each link. The pose matrix is ​​used to indicate the spatial position and orientation of the geometry to be detected. Based on the pose matrix of each link corresponding to the geometry to be detected, collision detection processing is performed on the multiple target collision pairs to obtain the collision detection results.

6. The method according to claim 5, characterized in that, The plurality of links includes a base link, a left arm link, and a right arm link; based on the target spatial position and the collision shell mesh, the pose matrix of each link corresponding to the geometry to be detected is determined, including: Determine the target radian vector of the left arm link angle and the target radian vector of the right arm link angle corresponding to the target spatial position; Based on the target radian vector of the left arm link angle and the target radian vector of the right arm link angle, the initial configuration vector corresponding to the dual-arm robot is updated to obtain the target configuration vector; Based on the collision shell mesh, the geometry to be detected corresponding to each link is determined; Based on the target configuration vector, the pose matrix of the geometry to be detected corresponding to each link is determined.

7. The method according to claim 1, characterized in that, Based on the collision detection results, control output signals are generated, including: When the collision detection result is a collision, the control state of the dual-arm robot is adjusted from the normal operation state to the collision lock state; The control output signal is generated in the collision lock state.

8. The method according to claim 7, characterized in that, In the collision lock state, generating the control output signal includes: In the collision-locked state, the path from the current position to the target position is determined to be collision-free by using joint space interpolation path; If there is no collision on the path, the control state is adjusted from the collision lock state to the tracking recovery state, and the control output signal is a progressive tracking signal.

9. The method according to claim 1, characterized in that, Also includes: Visualization tools are used to render and display the robot's global pose, the motion trajectory of each link, and the collision status, highlighting the detected collision geometry in a preset color.

10. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of claims 1-9.

Images