Dynamic target grasping method for industrial robot based on binocular vision cooperation

Abstract

The application provides a dynamic target grabbing method of an industrial robot based on binocular vision cooperation, comprising the following steps: synchronously acquiring left and right view images of a target to be grabbed and adjacent dynamic targets by a binocular vision acquisition unit, generating a neighborhood dynamic target pose sequence through visual detection, image recognition and binocular matching; constructing a neighborhood collision propagation graph according to the predicted contact relationship between targets, determining a collision pre-occupation expiration time and generating a collision post-candidate motion branch; combining the current state of the robot, a grabbing safety area and a collision impact pre-occupation state to generate a pre-collision grabbing branch, a continued grabbing branch and a post-collision takeover branch, optimizing the candidate grabbing branch by using a fruit fly algorithm, and extracting a common path segment before branching; continuously updating visual data in the motion process of the robot, confirming whether a collision impact occurs, identifying an actual motion branch and switching a grabbing path.

Description

Technical Field

[0001] This invention relates to the field of industrial robot vision control technology, and in particular to a method for dynamic target grasping of industrial robots based on binocular vision collaboration. Background Technology

[0002] Industrial robots have been widely used in production scenarios such as parts sorting, packaging material transfer, and continuous material feeding. For targets in motion, existing methods typically use binocular vision acquisition units to acquire left and right view images, and through image recognition, binocular matching, and trajectory prediction, calculate the spatial position of the target to be grasped at subsequent time points, and generate a grasping path accordingly to control the end effector to complete the approach, alignment, and grasping.

[0003] However, in scenarios involving the dense transport of parts, packaging boxes, and irregularly shaped workpieces, the target to be grasped does not always move independently. Nearby dynamic targets may collide with the target due to differences in transport speed, localized accumulation, attitude deflection, or rolling slippage. The impact of collisions may also be transmitted to the target through continuous contact between multiple targets. After being affected by a collision, the target's direction of motion, speed, and attitude may change abruptly. Extrapolating the trajectory solely based on the target's historical pose data is insufficient to accurately reflect its subsequent motion state.

[0004] Therefore, this invention proposes a dynamic target grasping method for industrial robots based on binocular vision collaboration. The information disclosed in the background section is only for enhancing understanding of the background of this disclosure and may therefore contain prior art information that is not common knowledge to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a dynamic target grasping method for industrial robots based on binocular vision collaboration, thereby solving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for dynamic target grasping in industrial robots based on binocular vision collaboration includes the following steps: S1. Simultaneously acquire left and right view images of the target to be captured and nearby dynamic targets through the binocular vision acquisition unit. After visual detection, image recognition and binocular matching, calculate the three-dimensional position, motion direction, motion speed and attitude change of each target to generate a neighborhood dynamic target pose sequence. S2. Calculate the expected contact relationship between targets based on the dynamic target pose sequence in the neighborhood, construct a neighborhood collision transfer graph, and generate a collision impact pre-occupancy state that includes the collision transfer path, the collision pre-occupancy expiration time, and the candidate motion branches after the collision. S3. Based on the collision impact pre-occupancy state, the robot's current joint state, the end effector posture, and the grasping safety area, generate the pre-collision grasping branch, the continued grasping branch, and the post-collision takeover branch. Optimize using the fruit fly algorithm and extract the common path segment before the fork. S4. Control the robot to move along the common path segment before the collision grabbing branch or fork, continuously acquire new left and right view images, update the dynamic target pose sequence in the neighborhood, confirm whether the collision effect has occurred and identify the actual movement branch. S5. Generate the grasping branch switching result based on the actual motion branch, switch the robot to the continuing grasping branch or the corresponding collision takeover branch, generate the joint control command sequence, and control the end effector to complete tracking, alignment, grasping and grasping result verification.

[0007] S1 specifically includes: synchronously acquiring left and right view images through a binocular vision acquisition unit; extracting target image regions through visual detection and image recognition; determining target category, target specification identifiers, and surface features; determining the target to be captured and assigning target tracking identifiers according to the capture task table; correcting and binocular matching the left and right view images according to the calibration parameter file; calculating the three-dimensional position, motion direction, motion speed, velocity vector, three-dimensional attitude angle vector, and spatial outer contour dimensions of each target to form a three-dimensional motion state record; calculating the neighborhood filtering distance with the target to be captured as the center; filtering nearby dynamic targets; performing time alignment and invalid record removal on the three-dimensional motion state record; and generating a neighborhood dynamic target pose sequence.

[0008] S2 specifically includes: reading the dynamic target pose sequence in the neighborhood, extrapolating the expected three-dimensional position of each target according to the predicted step size, generating the expected motion envelope, and forming a set of expected contact relationships in the neighborhood based on the distance between targets, the overlapping state of bounding boxes, and the approach direction; taking each target as the target node and the expected contact relationship as the directed connection relationship, reading the velocity transfer coefficient range, attitude deflection range, and response time in the collision response parameter table, and constructing a neighborhood collision transfer graph; filtering the direct collision transfer path and indirect collision transfer path with the endpoint as the target to be captured, recursively calculating the arrival time of the collision impact, determining the collision pre-occupancy expiration time, generating candidate motion branches after the collision, and forming the collision impact pre-occupancy state.

[0009] S3 specifically includes: reading the collision impact pre-occupancy state and the robot's current state data; combining the robot's work area map and grasping parameter table; extracting the expected grasping time along the non-collision expected movement trajectory and the candidate motion branches after the collision; generating a pre-collision grasping branch, a continued grasping branch, and a post-collision takeover branch; combining the continued grasping branch and the post-collision takeover branch into a branch path combination; using the fruit fly algorithm to perform single-path optimization or collaborative optimization on the candidate grasping branches; outputting the optimized candidate grasping branch set; aligning the path nodes in different grasping branches; extracting the common path segment before the fork and the fork path node; and determining the initial execution path based on the grasping safety time margin.

[0010] S4 specifically includes: generating an initial path joint control command sequence based on the initial execution path, controlling the robot's movement, continuously acquiring new left and right view images, updating the neighborhood dynamic target pose sequence and real-time robot execution status; calculating a collision deviation evaluation value based on the updated neighborhood dynamic target pose sequence, confirming whether a collision has occurred based on actual contact evidence, and identifying the actual motion branch or the uncovered actual motion branch based on the branch matching value; performing a branch switching executability check based on the actual motion branch and the real-time robot execution status, and generating a gripping branch switching result that switches to a continuing gripping branch, a corresponding post-collision takeover branch, or executes the corresponding safe exit path.

[0011] S5 specifically includes: reading the grasping branch switching results and real-time robot execution status; resampling the control cycle of path nodes in the target grasping branch to generate a joint control command sequence; controlling the robot to approach the target to be grasped according to the joint control command sequence; calculating the predicted closing position, relative position deviation, and relative posture deviation; performing amplitude limiting correction; and controlling the end effector to close when the closing trigger condition is met to form gripping state data; obtaining the verification left and right view images based on the verification lifting path segment; verifying the grasping result by combining the gripping state data and synchronous movement error; generating grasping result data; and re-executing when the grasping fails and the retry condition is met.

[0012] The beneficial effects of this invention are as follows: This invention simultaneously acquires left and right view images of the target to be grasped and nearby dynamic targets using a binocular vision acquisition unit, forming a neighborhood dynamic target pose sequence. This allows for simultaneous understanding of the three-dimensional motion state of the target to be grasped and its surrounding targets, improving the accuracy of dynamic target recognition and tracking in dense transport scenarios. By constructing a neighborhood collision transfer map and generating a collision impact pre-occupancy state, it can identify the direct and indirect collision impacts that nearby dynamic targets may have on the target to be grasped before a collision actually occurs, reducing grasping deviations caused by relying solely on predictions based on the target's historical trajectory.

[0013] This invention generates pre-collision grabbing branches, continued grabbing branches, and post-collision takeover branches based on the pre-occupancy state caused by collisions, and extracts common path segments before forks. This allows the industrial robot to retain path switching capabilities even before the collision outcome is clear, avoiding premature locking onto a single grabbing path. The fruit fly algorithm is used to optimize candidate grabbing branches, and executable paths are selected by combining grabbing completion time limits, joint constraints, and grabbing safety area constraints. This improves the safety and efficiency of grabbing paths, and reduces invalid tracking and redundant planning.

[0014] This invention confirms whether a collision has occurred based on newly added left and right view images and identifies the actual motion branch of the target to be grasped. It can promptly switch to the corresponding post-collision takeover branch after a change in the target's motion state, improving the adaptability of the industrial robot's dynamic grasping. By calculating the predicted end-of-closure position and combining it with gripping state data, verifying the left and right view images, and confirming the grasping result with synchronous movement errors, it can compensate for the target displacement during the end effector's closure, reducing the probability of missed grasping, misgrabbing, and unstable gripping. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the dynamic target grasping method for industrial robots based on binocular vision collaboration according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example: Figure 1 As shown, this embodiment provides a dynamic target grasping method for industrial robots based on binocular vision collaboration, including the following steps: S1. Simultaneously acquire left and right view images of the target to be captured and nearby dynamic targets through the binocular vision acquisition unit. After visual detection, image recognition and binocular matching, calculate the three-dimensional position, motion direction, motion speed and attitude change of each target to generate a neighborhood dynamic target pose sequence. S2. Calculate the expected contact relationship between targets based on the dynamic target pose sequence in the neighborhood, construct a neighborhood collision transfer graph, and generate a collision impact pre-occupancy state that includes the collision transfer path, the collision pre-occupancy expiration time, and the candidate motion branches after the collision. S3. Based on the collision impact pre-occupancy state, the robot's current joint state, the end effector posture, and the grasping safety area, generate the pre-collision grasping branch, the continued grasping branch, and the post-collision takeover branch. Optimize using the fruit fly algorithm and extract the common path segment before the fork. S4. Control the robot to move along the common path segment before the collision grabbing branch or fork, continuously acquire new left and right view images, update the dynamic target pose sequence in the neighborhood, confirm whether the collision effect has occurred and identify the actual movement branch. S5. Generate the grasping branch switching result based on the actual motion branch, switch the robot to the continuing grasping branch or the corresponding collision takeover branch, generate the joint control command sequence, and control the end effector to complete tracking, alignment, grasping and grasping result verification.

[0018] S1 specifically includes the following sub-steps: The S110 binocular vision acquisition unit includes a left industrial camera, a right industrial camera, a hardware trigger module, and a vision processing controller. The left and right industrial cameras are mounted above and to the side of the industrial robot's working area, and their shared field of view covers the dynamic target delivery area and the gripping safety area. The gripping safety area refers to the spatial area on the robot's working area map that allows the end effector to perform target approach, attitude adjustment, and closed gripping.

[0019] The robot's operating area map is established before equipment commissioning based on the boundaries of the conveyor equipment, the positions of fixed equipment, and preset safety distances. The hardware trigger module simultaneously sends exposure trigger signals to both the left and right industrial cameras according to the acquisition cycle recorded in the process parameter table, ensuring that each set of left and right view images has the same acquisition sequence number and acquisition time. The process parameter table is established during the equipment commissioning phase, recording the acquisition cycle, prediction step size, prediction duration, path node spacing, and various processing thresholds.

[0020] Visual inspection is performed on the left and right view images. When the conveyor is running unloaded, background images are continuously acquired and averaged pixel by pixel to form a left and right background reference image. Gray-level difference is performed between the current left and right view images and their corresponding background reference images. Pixels with gray-level differences reaching the motion determination threshold are identified as moving pixels. Connected regions of adjacent moving pixels are merged, and connected regions with areas smaller than the minimum effective area threshold are removed to obtain the target image region. The motion determination threshold and the minimum effective area threshold are statistically determined from the unloaded and loaded images acquired during the equipment commissioning phase.

[0021] For example, under typical industrial workshop lighting conditions, for a standard hard target with dimensions of 10cm×10cm, the motion determination threshold (grayscale difference) can be set to 30~50, and the minimum effective area threshold can be set to 100~200 pixels.

[0022] Image recognition is performed on the target image region. A target template library, established from standard product images and product process data, is read. The contour boundary, pixel width, pixel height, corner position, and corner description vector of the target image region are compared with the standard records in the target template library. The category corresponding to the standard record with the highest comprehensive similarity reaching the recognition threshold is determined as the target category.

[0023] The target template library must at least record the target category, target specification identifier, standard outline, allowable size range, corner descriptor vector, and rotational symmetry identifier. The target specification identifier is a number used to distinguish different size ranges, quality ranges, or material combinations within the same target category. Target image areas that do not meet the recognition threshold are marked as unknown category targets.

[0024] The vision processing controller reads the grasping task list issued by the industrial robot controller. The grasping task list at least records the grasping task number, target category, target entry position range, allowed movement direction range, and grasping priority. Dynamic targets whose category, entry position, and movement direction meet the task requirements and are expected to enter the grasping safety area first are identified as the targets to be grasped; target tracking tags are assigned to the targets to be grasped and the identified dynamic targets around them.

[0025] The target tracking identifier is a unique number used to associate the corresponding data records of the same dynamic target at continuous acquisition times. The target tracking identifier, target category, target specification identifier, image outer contour size, surface features, and rotational symmetry identifier are combined into the dynamic target recognition result and input into S120.

[0026] S120. Before the equipment is put into production, offline calibration is completed using a dual-target calibration board. The intrinsic parameters of the left and right industrial cameras, the relative position parameters between the left and right industrial cameras, and the transformation matrix from the coordinate system of the left industrial camera to the coordinate system of the industrial robot base are written into the calibration parameter file. The vision processing controller reads the calibration parameter file, performs distortion correction and epipolar alignment on the left and right view images corresponding to the same acquisition sequence number, and performs binocular matching on surface features belonging to the same dynamic target using the target tracking identifier as the association index.

[0027] For the matching point of the i-th dynamic target under the k-th acquisition sequence number, its depth coordinates are calculated according to the following formula:

[0028] In the formula, i represents the target tracking identifier number, and k represents the acquisition sequence number. Represents depth coordinates, B represents the equivalent focal length of the left industrial camera in the horizontal direction, and B represents the baseline distance between the left and right industrial cameras. This represents the difference in horizontal pixel coordinates between the same matching point in the left and right view images.

[0029] The horizontal and vertical coordinates are calculated based on the pixel coordinates of the matching point, the principal point coordinates, the equivalent focal length, and the depth coordinates. Matching points with parallax exceeding the effective range are eliminated, and the geometric center of the remaining effective matching points is transformed to the industrial robot base coordinate system as the three-dimensional position of the dynamic target.

[0030] A 3D directed bounding box is constructed based on the effective 3D contour edge points, and its actual width, actual height, and actual length are defined as the spatial outer contour dimensions. The roll angle, pitch angle, and yaw angle corresponding to the three mutually perpendicular axes of the 3D directed bounding box are used to form a 3D attitude angle vector, and the spatial direction pointed to by the longest side is determined as the target attitude reference direction. For rotationally symmetric targets, attitude changes around the axis of symmetry are not calculated.

[0031] The motion velocity is calculated based on the difference in three-dimensional position and the difference in acquisition time between adjacent acquisition moments. After normalizing the three-dimensional position difference, a unit vector of motion direction is obtained, and the velocity vector is calculated.

[0032] In the formula, This represents the velocity vector corresponding to the i-th dynamic target at the k-th acquisition sequence number. Indicates the speed of motion. This represents the unit vector indicating the direction of motion.

[0033] The difference in the three-dimensional attitude angle vector between adjacent acquisition times is defined as the attitude change. The target tracking identifier, acquisition sequence number, acquisition time, target category, target specification identifier, three-dimensional position, motion direction unit vector, motion velocity, velocity vector, three-dimensional attitude angle vector, attitude change, target attitude reference direction, and spatial outer contour dimension are combined into a three-dimensional motion state record and input into S130.

[0034] S130. Using the target to be captured as the center, calculate the neighborhood filtering distance for each of the other dynamic targets:

[0035] In the formula, j represents the target tracking identifier number corresponding to the candidate dynamic target. Indicates the neighborhood filtering distance. and Let g and g represent the spatial outer contour feature radii of the target to be grasped and the j-th candidate dynamic target, respectively. The spatial outer contour feature radius is half of the largest dimension of the corresponding spatial outer contour. The subscript g indicates the target to be grasped. Indicates the preset prediction duration. Indicates the safety margin for neighborhood filtering; and represents the movement speed of the target to be captured and the j-th candidate dynamic target, respectively.

[0036] The preset prediction time is obtained by adding the estimated time for the industrial robot to move to the grasping safety area, the end effector attitude adjustment time, and the end effector closing time; the neighborhood screening safety margin is obtained by adding the maximum error of binocular measurement, the maximum displacement error of the conveying equipment vibration, and the maximum error of target tracking.

[0037] When the current spatial distance between a candidate dynamic target and the target to be captured is not greater than the neighborhood filtering distance, it is identified as a nearby dynamic target. The acquisition time of the target to be captured is used as the time alignment reference. If a nearby dynamic target is missing one acquisition time for its 3D position, and the effective recording interval before and after is not greater than two acquisition cycles, linear interpolation is used to supplement the 3D position, and the velocity vector and 3D attitude angle vector are recalculated. If two consecutive acquisition times are missing, no interpolation is performed.

[0038] Records with insufficient number of valid matching points, 3D positions exceeding the industrial robot's operating area, adjacent displacements exceeding the allowable range, or more than one consecutive missing record are marked as invalid records. Only dynamic targets with at least three consecutive valid records are retained.

[0039] For example, , , , , , hour, Therefore, all candidate dynamic targets with a current spatial distance of no more than 0.60m are retained as neighboring dynamic targets.

[0040] According to the acquisition time sequence, the three-dimensional motion state records of the target to be captured and each neighboring dynamic target are arranged into a neighborhood dynamic target pose sequence, and the neighborhood dynamic target pose sequence is input into S210.

[0041] S2 specifically includes the following sub-steps: S210: Read the neighborhood dynamic target pose sequence output by S130, and perform short-time extrapolation on the target to be grasped and each neighboring dynamic target according to the prediction step size recorded in the process parameter table. The prediction step size is no greater than the binocular vision acquisition cycle. For the i-th dynamic target, after the prediction time after the current acquisition sequence number k... The predicted three-dimensional position at that time is:

[0042] In the formula, Indicates the expected three-dimensional position. Indicates the current three-dimensional position. Represents the velocity vector. This represents the predicted time relative to the current acquisition time, and The predicted trajectory of the target to be captured when it is not affected by collisions with nearby dynamic targets is defined as the non-collision predicted trajectory, and the non-collision predicted trajectory is input into S310 and S420.

[0043] At each prediction time point, a 3D directed bounding box is constructed centered on the predicted 3D position, based on the spatial outer contour dimensions, the target attitude reference direction, and the attitude change. The 3D directed bounding boxes corresponding to the same dynamic target at all prediction time points are combined to form the predicted motion envelope. For any two dynamic targets, distance filtering, bounding box overlap filtering, and proximity direction filtering are performed sequentially.

[0044] When the distance between the two expected three-dimensional positions is no greater than the sum of the radii of their spatial outer contour features and the collision screening safety margin, and when the corresponding three-dimensional oriented bounding boxes overlap and move along the direction of mutual approach, it is determined that there is an expected contact relationship between the two. The collision screening safety margin is obtained by adding the maximum error of binocular measurement, the maximum displacement error of the conveying equipment vibration, and the maximum calculation error of the spatial outer contour size measured during the equipment commissioning phase.

[0045] The target tracking identifiers, estimated contact time, estimated contact moment, overlapping positions of the 3D directed bounding boxes, and relative velocities before contact of two dynamic targets are written into the neighborhood estimated contact relationship set. The estimated contact time refers to the time interval relative to the current acquisition moment; the estimated contact moment refers to the sum of the current acquisition moment and the estimated contact time.

[0046] For example, if two dynamic targets are currently 0.28m apart, have spatial outer contour feature radii of 0.05m and 0.06m respectively, a collision screening safety margin of 0.02m, and a relative velocity along the approach direction of 0.30m / s, then the time required for them to reach the collision screening distance is 0.50s. The expected contact relationship is only written into the neighborhood expected contact relationship set when the two corresponding 3D directed bounding boxes overlap at 0.50s.

[0047] S220. Using the target to be captured and each neighboring dynamic target as target nodes, and the expected contact relationships in the set of expected contact relationships in the neighborhood as directed connections, construct a neighborhood collision transmission graph. A neighborhood collision transmission graph is a data structure used to represent the possible order in which collision effects are transmitted between neighboring dynamic targets.

[0048] For the a-th dynamic target and the b-th dynamic target, their contact direction unit vector is:

[0049] In the formula, a and b represent the target tracking identifier numbers corresponding to the two dynamic targets, respectively. This indicates the expected contact time between the two. This represents the unit vector of the contact direction from the a-th dynamic target to the b-th dynamic target. A directed connection is established from the a-th dynamic target to the b-th dynamic target when the projection of the relative velocity vectors of the two targets onto the unit vector of the contact direction is greater than 0.

[0050] The velocity transfer coefficient range, attitude deflection range, and response time are read from the collision response parameter table. The collision response parameter table is established through controlled collision tests before the equipment is put into production: collision tests are performed for target category combinations, target specification marking combinations, contact direction ranges, and relative velocity ranges before contact. The velocity vectors and three-dimensional attitude angle vectors before and after the collision are recorded by a binocular vision acquisition unit, and the corresponding parameter ranges are obtained statistically.

[0051] When the actual target cannot be matched with the collision response parameter table, the maximum velocity transfer coefficient range, maximum attitude deflection range, and maximum response time recorded in the process parameter table are read to form a conservative prediction. For example, for a common collision scenario of a rigid plastic packaging box on a conveyor belt, its velocity transfer coefficient... The empirical value range is 0.5~0.8, the attitude deflection range is -15°~15°, and the response time is 0.02s~0.05s.

[0052] The candidate velocity vector of the b-th dynamic target after being affected by the collision is:

[0053] In the formula, Represents the candidate velocity vector. Indicates the speed transmission coefficient. Represents the dot product of vectors; and Let represent the velocity vectors of the a-th and b-th dynamic targets before their collision, respectively.

[0054] The lower, middle, and upper limits of the velocity transfer coefficient range and the attitude deflection range are taken respectively, and combined in pairs to form 9 sets of candidate motion states after collision. Directed connections are arranged from earliest to latest according to the expected contact time; when cyclic connections occur, only directed connections with earlier expected contact times and higher relative velocities before contact are retained. The processed target nodes and directed connections are combined into a neighborhood collision transfer graph and input into S230.

[0055] S230. Filter direct and indirect collision transfer paths whose endpoints are the target to be captured from the neighborhood collision transfer graph. A direct collision transfer path includes only one directed connection; an indirect collision transfer path includes at least two sequentially arranged directed connections. Calculate the arrival time of collision effects level by level along the indirect collision transfer path:

[0056]

[0057] In the formula, Indicates the sequence number of the directed connection. Indicates the impact of the collision along the first The arrival time after the directed connection is passed. This indicates the corresponding expected contact time. This indicates the response time corresponding to the previous directed connection. After each level of recursion is completed, the updated candidate velocity vector is used to re-verify whether the expected motion envelope corresponding to the next directed connection still overlaps; if they no longer overlap or the final arrival time exceeds the preset prediction time, the collision propagation path is deleted.

[0058] For example, if a nearby dynamic target A collides with a nearby dynamic target B at 0.18s, the corresponding response time is 0.03s. If a nearby dynamic target B collides with the target to be captured at 0.26s, then the arrival time of the collision impact is:

[0059] The three time values ​​mentioned above must not be added together directly.

[0060] The estimated collision time is obtained by adding the current acquisition time to the final arrival time of each valid collision propagation path, and the earliest estimated collision time is determined as the collision pre-occupancy expiration time. Based on the candidate velocity vector and 3D attitude angle vector ultimately acting on the target to be grasped, candidate motion branches after the collision are generated. If the maximum spatial distance and attitude difference between two candidate motion branches at the same prediction time point do not exceed the branch merging threshold recorded in the process parameter table, the two are merged. If no valid collision propagation path exists, the current state is marked as a no-collision pre-occupancy state.

[0061] Combine the effective collision propagation path, the expected collision time, the collision pre-occupancy expiration time, the candidate motion branch after the collision, and the non-collision pre-occupancy status flag into the collision impact pre-occupancy status, and input it into S310.

[0062] S3 specifically includes the following sub-steps: S310 reads the collision impact pre-emption state output by S230 and reads the robot's current state data from the industrial robot controller. The robot's current state data is fed back by the joint encoders and servo drives, including the current joint angle, current joint speed, allowable rotation range of each joint, maximum allowable speed, maximum allowable acceleration, and the current pose and opening / closing state of the end effector.

[0063] In this embodiment, the end effector employs two electrically driven grippers that can open and close relative to each other to hold fingers. The electrically driven grippers are equipped with a motor encoder and a driver; the motor encoder outputs the finger spacing, and the driver outputs the drive current and a closing status indicator.

[0064] Read the robot's work area map and grasping parameter table. The robot's work area map should at least record the grasping safety area, prohibited area, safe waiting position, and preset verification position; the preset verification position is located within the common field of view of the binocular vision acquisition unit.

[0065] The grasping parameter table is established according to the target category and target specification identifier, and at least records the allowed gripping area, the range of the end effector approach direction, the allowable range of the target gripping width, the grasping pose offset, the allowable grasping position error, the allowable grasping attitude error, the upper limit of the end effector closing time, the maximum position correction in a single cycle, the maximum attitude correction in a single cycle, the stable gripping current range, the synchronous movement error threshold, the minimum retention ratio of the target image area, the verification lifting distance, the verification lifting speed, the verification duration, and the maximum number of retries.

[0066] The predicted non-collision trajectory generated in S210 and the candidate motion branches after collision generated in S230 are used to extract the predicted grasping time at preset time intervals. The grasping pose is calculated based on the target's 3D position, 3D attitude angle vector, and grasping pose offset. The grasping pose refers to the target's position and orientation in the industrial robot's base coordinate system when the end effector completes the gripping process.

[0067] The inverse kinematics solver built into the industrial robot controller is used to convert the grasping pose into target joint angles. When multiple solutions exist, the solution with the smallest total deviation from the current joint angle is retained. Linear interpolation in joint space is performed between the current joint angle and the target joint angle. This is then converted into a continuous motion path for the end effector using a forward kinematics solver, and path nodes are generated according to the path node spacing recorded in the process parameter table. Path nodes are the position and orientation records within the continuous motion path of the end effector.

[0068] Paths that exceed joint constraints, enter prohibited areas, or cannot complete the grab on time are eliminated. The remaining paths are designated as continued grab branches, post-collision takeover branches, and pre-collision grab branches. The estimated completion time for the q-th candidate grab branch is:

[0069] In the formula, q represents the candidate crawling branch number, and n represents the path node sequence number. This represents the total number of path nodes, where h represents the joint number. Indicates the target angle of the joint. and These represent the maximum permissible speed and the maximum permissible acceleration, respectively. This indicates the upper limit of the closing time of the end effector.

[0070] The pre-collision grabbing branch is retained only if the expected completion time of grabbing is earlier than the difference between the collision pre-occupancy expiration time and the grabbing safety time margin. The grabbing safety time margin is obtained by adding the binocular vision acquisition cycle, the robot control cycle, the maximum measurement error of the closure duration, and the maximum response deviation of the joint movement.

[0071] S320: When S230 outputs a collision-free pre-occupancy state, only the fruit fly algorithm is used to perform single-path optimization on the continuing crawling branch. When a collision-triggered takeover branch exists, one continuing crawling branch and all collision-triggered takeover branches are combined into a branch path combination, and collaborative optimization is performed. A branch path combination refers to a set of paths that have a common approach path and allow switching to different crawling results.

[0072] A fruit fly individual is encoded with the intermediate path node positions in the common path segment before branching, the branch path node positions, the intermediate path node positions in each grasping branch, the end effector approach direction, the end effector attitude adjustment amount, and the joint speed limit ratio. Using the executable path generated by S310 as the baseline fruit fly individual, a fruit fly population is generated according to the perturbation amplitude recorded in the process parameter table; ineffective fruit fly individuals that do not meet the joint constraints, grasping safe area constraints, or completion time constraints are removed.

[0073] The path cost of the 0th effective fly is ; In the formula, o represents the individual fruit fly number. Indicates the value of the path. Indicates the estimated time required to complete the data crawling process. Indicates the minimum safe distance. This represents the normalized value of the end effector attitude adjustment. This indicates the maximum percentage of joint velocity utilization. This represents the proportion of the length of the common path segment before the fork to the total length of the combined branch paths. and These represent the reference capture time and the reference safe distance, respectively. This prevents positive numbers with a denominator of 0. to This represents the weight. In single-path optimization, it will... Set to 0.

[0074] The above parameters are determined by the grasping verification test during the equipment commissioning phase and written into the process parameter table. In a preferred specific implementation scenario, a grasping time weight can be set. =0.4, safety distance weight =0.2, Attitude Adjustment Weight =0.15, Joint velocity weight =0.15, Public Path Proportional Weight =0.10; The population size of the fruit fly algorithm can be set to 50, the maximum number of iterations is 100, and the perturbation amplitude is 5% of the current variable's allowable range.

[0075] Will As the odor concentration evaluation value, the fruit fly individual with the largest odor concentration evaluation value is selected as the center fruit fly individual for the next round; when the difference in cost value between two consecutive optimal paths is not greater than the termination threshold or the maximum number of iterations is reached, the iteration stops and the optimized candidate grasping branch set is output.

[0076] S330. Starting from the current position of the industrial robot, align the path nodes in different gripping branches according to the cumulative path length. Path nodes whose cumulative path length difference is not greater than one path node spacing are identified as corresponding path nodes. When the position deviation of the corresponding path node is not greater than the common path position tolerance, the maximum joint angle deviation is not greater than the common path joint tolerance, and the completion time limit, joint constraints, and gripping safe area constraints are still met after switching to any gripping branch, it is identified as a common path node.

[0077] Consecutive common path nodes are arranged to form a pre-fork common path segment, and the last common path node is determined as the fork path node. The common path position tolerance and common path joint tolerance are determined by the path tracking error and joint tracking error allowed by the robot controller and are written into the process parameter table.

[0078] For example, if the positional deviation of two grabbing branches at a certain path node is 3.74mm and the maximum joint angle deviation is 0.8°, while the common path position tolerance is 5mm and the common path joint tolerance is 1°, and the grabbing pose can still be reached on time after switching, then this path node belongs to the common path node. Corresponding path nodes in two grabbing branches do not need to have completely identical node coordinates.

[0079] When multiple pre-collision capture branches exist that meet the pre-collision capture safety margin requirements, the branch with the lowest path cost is selected; if the path costs are the same, the branch with the earlier expected capture completion time is selected. If an executable pre-collision capture branch exists, it is designated as the initial execution path; otherwise, the common path segment before the fork is designated as the initial execution path. In the no-collision pre-occupancy state, the optimized continuation capture branch is designated as the initial execution path. If the common path segment before the fork cannot be extracted, the path from the current position to the safe waiting position is designated as the initial execution path.

[0080] Input the initial execution path, the optimized continuation fetch branch, the optimized collision takeover branch, the fork path node, and the safe wait position into S410-S430.

[0081] S4 specifically includes the following sub-steps: S410 reads the initial execution path, optimized continuation grasping branch, optimized post-collision takeover branch, forked path nodes, and safe waiting positions output by S330. It resamples the path nodes in the initial execution path according to the robot control cycle, generates the initial path joint control command sequence, and sends it to the industrial robot controller. The industrial robot controller reads feedback data from the joint encoders and servo drives in each control cycle to form the real-time robot execution status.

[0082] The real-time robot execution status includes at least the current joint angle, current joint velocity, current end effector pose, current path node, remaining distance from the current path node to the branch path node, and corresponding timestamp; if there is no branch path node, the remaining distance is recorded as 0.

[0083] The binocular vision acquisition unit synchronously acquires newly added left and right view images according to the acquisition cycle in S110, and repeatedly executes visual detection, image recognition, binocular matching, and 3D motion state calculation in S110-S120. Using the target tracking identifier as the associated index, the newly added 3D motion state record is appended to the neighborhood dynamic target pose sequence. For the same acquisition moment, the real-time robot execution state with a timestamp difference not exceeding one robot control cycle is read, ensuring that the visual data corresponds to the robot execution state.

[0084] A rolling observation window is set up. The rolling observation window refers to the data records corresponding to the most recent H valid acquisition moments used to determine in real time whether a collision has occurred; H represents the number of valid acquisition moments included in the rolling observation window, and its value is determined by the collision recognition verification test during the equipment debugging phase; in this embodiment, it is set to 3. When the 3D position of a neighboring dynamic target is missing only one acquisition moment, it is supplemented according to the linear interpolation rule in S130; when two consecutive acquisition moments are missing, the corresponding record is marked as an invalid record. The updated neighborhood dynamic target pose sequence and the real-time robot execution state are input into S420.

[0085] S420. Based on the acquisition time corresponding to the newly added left and right view images, read the expected 3D position, expected velocity vector, and expected 3D attitude angle vector of the target to be captured from the non-collision expected movement trajectory generated in S210 to form a non-collision reference state. Read the non-collision allowable deviation parameter table. The non-collision allowable deviation parameter table is established during the equipment debugging phase according to the target category, target specification identifier, and conveyor speed range, recording the non-collision allowable position deviation, non-collision allowable velocity deviation, and non-collision allowable attitude deviation.

[0086] For the k-th acquisition sequence number, calculate the collision deviation evaluation value:

[0087] In the formula, This indicates a deviation from the evaluation value in the collision assessment. , and These represent the actual 3D position, actual velocity vector, and actual 3D attitude angle vector of the target to be captured, respectively. Parameters with a superscript of 0 indicate a non-collision reference state. , and These represent the allowable position deviation, allowable velocity deviation, and allowable attitude deviation without collision, respectively.

[0088] When H consecutive valid acquisition times within the rolling observation window all meet the requirements When at least one valid collision transfer path retained by S230 shows evidence of actual contact, it is confirmed that the target to be captured has been affected by a collision. Evidence of actual contact refers to the actual distance between adjacent target nodes in the collision transfer path not exceeding the sum of the sum of their spatial outer contour feature radii and the contact confirmation distance margin, and the two nodes moving in the direction of mutual approach. The contact confirmation distance margin is obtained by adding the maximum error of binocular measurement and the maximum calculation error of the spatial outer contour dimension. For indirect collision transfer paths, actual contact evidence is checked level by level according to the transfer sequence.

[0089] After confirming the impact of the collision, calculate the branch matching value for the r-th candidate motion branch after the collision:

[0090] In the formula, r represents the candidate motion branch number after the collision. This represents the branch matching value. Parameters with superscript caps indicate the predicted parameters of the corresponding candidate motion branches after the collision. , and These represent the weights corresponding to position deviation, velocity deviation, and attitude deviation, respectively, and the sum of the three is 1. The weights, branch matching threshold, and branch discrimination margin are determined by the branch recognition verification test during the equipment commissioning phase and written into the process parameter table.

[0091] For example, when performing branch switching matching, position deviation weights can be set. =0.6, speed deviation weight =0.3, Attitude Deviation Weight =0.1; the branch matching threshold is set to 0.15, and the branch differentiation margin is set to 0.08.

[0092] A candidate motion branch after a collision is identified as an actual motion branch only if the minimum branch matching value is not greater than the branch matching threshold, and the difference between the second smallest branch matching value and the minimum branch matching value is not less than the branch discrimination margin, and these conditions are met for two consecutive valid acquisition times. If these conditions are not met, the branch is marked as pending confirmation; if all branches exceed the branch matching threshold, the actual motion branch is marked as not covered. For example, if the branch matching values ​​of two candidate motion branches after a collision are 0.62 and 0.68 respectively, and the branch discrimination margin is 0.10, the actual motion branch is not confirmed at this time.

[0093] S430: Read the real-time robot execution status and S420 output results. If the current execution is on a pre-collision capture branch and the collision impact has not been confirmed, maintain that branch; if the collision impact occurs prematurely, terminate that branch. If the current execution is on a common path segment before a fork and the collision impact has not been confirmed, continue running until a rolling observation window after the collision pre-occupancy expires; if the collision impact is still not confirmed, switch to continuing capture of the branch.

[0094] If the currently optimized continuation grasping branch is being executed and a collision is subsequently detected, S210-S330 are re-executed. If the actual motion branch has been confirmed, the system switches to the corresponding post-collision takeover branch. If the robot is in a branch-pending confirmation state and has reached a fork path node, or is in a state where the actual motion branch is not covered, a safe exit path is executed. The safe exit path refers to a path generated based on the robot's work area map, from the current position to a safe waiting position, avoiding prohibited areas.

[0095] Before switching to the post-collision takeover branch, perform a branch switching executability check:

[0096] In the formula, This represents the remaining execution time required to switch from the current position to the r-th target fetch branch and complete the fetch. Indicates the branch switching response time. Indicates the expected capture time. Indicates the current data collection time.

[0097] The branch switching response time is obtained by adding the robot control cycle, the control command transmission time, and the servo driver's maximum response time. Simultaneously, the joint angle, joint velocity, joint acceleration, and link motion envelope are checked after the switch. If the check fails, a safe exit path is executed; after reaching the safe waiting position, the newly added left and right view images are acquired, and S210-S330 are re-executed.

[0098] The current action type is determined as follows: maintain the pre-collision grabbing branch, switch to the continued grabbing branch, switch to the post-collision takeover branch, execute a safe exit path, or replan. The current action type, target grabbing branch number, path switching start node, path node sequence, end effector approach direction, end effector attitude adjustment amount, joint speed limit value, whether to execute a safe exit, and whether to replan are combined into a grabbing branch switching result, and then input into S510.

[0099] S5 specifically includes the following sub-steps: S510 reads the gripping branch switching result output by S430, the real-time robot execution status output by S410, and the robot work area map and gripping parameter table called by S310. If the current execution action type in the gripping branch switching result is executing a safe exit path, a joint control command sequence is generated based on the safe exit path, and the industrial robot is controlled to move to a safe waiting position. If the current execution action type is replanning, S210-S330 are re-executed. If the current execution action type is maintaining or switching to the target gripping branch, the target gripping control of this step continues.

[0100] The industrial robot controller resamples the path nodes in the target grasping branch according to the internally stored robot control cycle, forming control cycle path nodes. Control cycle path nodes refer to the records of joint target angles, joint target velocities, joint target accelerations, and arrival times arranged according to the robot control cycle.

[0101] The nth path node to the nth path node The execution time between each path node is:

[0102] In the formula, The values ​​represent the execution time of adjacent path nodes, q represents the target fetch branch number, n represents the path node sequence number, and h represents the joint number. Indicates the target angle of the joint. Indicates the joint speed limit value. This represents the maximum permissible acceleration. The maximum value among the execution times of each joint is taken because when multiple joints move synchronously, the arrival time of a path node is limited by the joint with the longest execution time.

[0103] Based on the execution duration of adjacent path nodes, linear interpolation is performed on the joint target angle, and the joint target velocity and joint target acceleration corresponding to the path node in the control cycle are calculated. Each item is checked to ensure that the joint target angle is within the allowable rotation range, the joint target velocity does not exceed the joint velocity limit, the joint target acceleration does not exceed the maximum allowable acceleration, and the linkage motion envelope is within the gripping safety area.

[0104] If the verification passes, the control cycle path nodes are combined into a joint control command sequence and sent to the industrial robot controller; if the verification fails, the current grasping action is terminated, a safe exit path is executed, and S210-S330 are re-executed.

[0105] S520: The industrial robot approaches the target to be grasped according to the joint control command sequence. After the end effector enters the grasping safety area, it continues to acquire new left and right view images and updates the actual 3D position and actual velocity vector of the target to be grasped according to the processing method in S410. Since the target to be grasped will still move during the end effector closure, the predicted position at the end of the closure is calculated according to the following formula:

[0106] In the formula, This indicates the predicted end-of-close position of the target object when the end effector completes the closure. Indicates the current actual three-dimensional position. This represents the current actual velocity vector. This indicates the upper limit of the closing time of the end effector. The upper limit of the closing time is written into the capture parameter table after repeated closing tests are performed for different target categories and target specifications during the equipment commissioning phase.

[0107] Read the grasp pose offset, target grasp attitude, allowable grasp position error, allowable grasp attitude error, maximum position correction per cycle, and maximum attitude correction per cycle from the grasp parameter table. The relative position deviation and relative attitude deviation are respectively:

[0108]

[0109] In the formula, Indicates relative positional deviation. Indicates the actual position of the end effector. This represents the grab pose offset. This indicates the relative attitude deviation. This represents the actual three-dimensional attitude angle vector of the end effector. This represents the three-dimensional attitude angle vector corresponding to the target's grasping posture.

[0110] For each robot control cycle, a limiting correction is performed based on the maximum position correction and the maximum attitude correction within a single cycle. Limiting correction restricts the position and attitude corrections within a single robot control cycle to within the safe range for the industrial robot. The corrected end effector pose still needs to be verified through joint constraints and gripping safety area constraints.

[0111] The end effector is controlled to close only if both the relative position deviation and relative attitude deviation do not exceed the corresponding allowable errors, and this holds true for two consecutive valid acquisition moments. If the deviation fails to converge after three consecutive robot control cycles, or if further correction would cause the linkage motion envelope to enter a prohibited area, the current grasping action is terminated and a safe exit path is executed.

[0112] For example, if the actual speed of the target to be grasped is 0.40 m / s and the upper limit of the end effector's closing time is 0.08 s, the target will still move 32 mm during the closing period. Therefore, the closing action should be triggered based on the predicted closing position, rather than based on the current three-dimensional position of the target. After the end effector completes the closing, it reads the finger spacing, drive current, and closing status indicator from the electric gripper driver to form gripping status data, which is then input into S530.

[0113] S530. Generate a verification lifting path segment based on the robot's work area map and gripping parameter table. The verification lifting path segment refers to the path the end effector moves at low speed away from the surface of the conveyor after completing its closure. The lifting distance, lifting speed, and verification duration are determined during the equipment commissioning phase based on the target size, conveyor height, and gripping safety area, and are written into the gripping parameter table. During the execution of the verification lifting path segment, left and right view images are continuously acquired, and gripping status data is read synchronously to form a gripping verification window.

[0114] The synchronization error between two adjacent acquisition times is:

[0115] In the formula, This represents the difference in displacement vectors between the target to be captured and the end effector at adjacent acquisition times; and These represent the actual 3D positions of the target to be captured at the current acquisition time and the previous acquisition time, respectively. and These represent the actual positions of the end effector at the current acquisition time and the previous acquisition time, respectively.

[0116] A successful grasp is determined only when the finger spacing is within the allowable range of the target clamping width, the drive current is within the stable clamping current range, the verification lifting path segment has been completed, the synchronous movement error of three consecutive valid acquisition moments is not greater than the synchronous movement error threshold, and the image recognition result shows that the target to be grasped has not left the clamping area. The above thresholds are all determined by the grasping verification test during the equipment debugging phase and written into the grasping parameter table.

[0117] When the end effector's finger obscures the target to be grasped, causing the effective contour area to fall below the minimum retention ratio of the target image area, the current grasping task is not immediately classified as a grasping failure. After completing the verification and lifting path segment, if the effective contour area recovers, image recognition and synchronous movement error calculation continue. If it still has not recovered, the robot moves to the preset verification position recorded on the robot's work area map and reacquires the verification left and right view images. For example, if only 18% of the effective contour area is retained, which is lower than the minimum retention ratio of 30%, the lifting verification is performed first, and then the grasping result is judged.

[0118] If a stable drive current is not generated after the end effector closes, the synchronous movement error continuously exceeds the limit, or the target to be grasped leaves the gripping area, the grasping is determined to have failed, and the industrial robot is controlled to execute a safe exit path. If the target to be grasped is still in the common field of view, the number of failures has not reached the maximum number of retries, and the re-grasping meets the safety constraints, S110 is executed again; otherwise, the current grasping task is terminated. The grasping task number, target tracking identifier, target category, target specification identifier, actual target grasping branch number, grasping completion time, finger spacing, drive current, maximum synchronous movement error, grasping success identifier, failure type, and subsequent processing method are combined into grasping result data.

[0119] All the above formulas are performed using dimensionless numerical calculations; the relevant formulas are based on empirical models that approximate the real situation, obtained through extensive data collection and software simulation fitting. The preset parameters and thresholds involved in the formulas can be conventionally set and adjusted by those skilled in the art according to the physical constraints of the actual application scenario.

[0120] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0122] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic target grasping method for industrial robots based on binocular vision collaboration, characterized in that, Includes the following steps: S1. Simultaneously acquire left and right view images of the target to be captured and nearby dynamic targets through the binocular vision acquisition unit. After visual detection, image recognition and binocular matching, calculate the three-dimensional position, motion direction, motion speed and attitude change of each target to generate a neighborhood dynamic target pose sequence. S2. Calculate the expected contact relationship between targets based on the neighborhood dynamic target pose sequence, construct a neighborhood collision transfer graph, and generate a collision impact pre-occupancy state that includes the collision transfer path, the collision pre-occupancy expiration time, and the candidate motion branches after the collision. Specifically, this includes: Read the dynamic target pose sequence in the neighborhood, extrapolate the expected three-dimensional position of each target according to the predicted step size, generate the expected motion envelope, and form a set of expected contact relationships in the neighborhood based on the distance between targets, the overlapping state of bounding boxes and the approach direction; Using each target as the target node and the expected contact relationship as the directed connection relationship, read the velocity transfer coefficient range, attitude deflection range and response time from the collision response parameter table to construct a neighborhood collision transfer graph; The direct and indirect collision transmission paths with the target to be captured are filtered, the arrival time of the collision impact is calculated recursively, the collision pre-occupancy expiration time is determined, and candidate motion branches after the collision are generated and the collision impact pre-occupancy state is formed. S3. Based on the collision impact pre-occupancy state, the robot's current joint state, the end effector posture, and the grasping safety area, generate a pre-collision grasping branch, a continued grasping branch, and a post-collision takeover branch. Optimize using the fruit fly algorithm and extract the common path segments before the forks, specifically including: Read the collision impact pre-occupation state and robot current state data, combine the robot's work area map and grasping parameter table, extract the expected grasping time along the non-collision expected movement trajectory and the candidate motion branch after the collision, and generate the pre-collision grasping branch, the continued grasping branch and the post-collision takeover branch. The continuous crawling branch and the collision takeover branch are combined into a branch path combination. The fruit fly algorithm is used to optimize the candidate crawling branches in a single path or in a collaborative manner, and the optimized candidate crawling branch set is output. S4. Control the robot to move along the common path segment before the collision grabbing branch or fork, continuously acquire new left and right view images, update the dynamic target pose sequence in the neighborhood, confirm whether the collision effect has occurred and identify the actual movement branch. S5. Generate the grasping branch switching result based on the actual motion branch, switch the robot to the continuing grasping branch or the corresponding collision takeover branch, generate the joint control command sequence, and control the end effector to complete tracking, alignment, grasping and grasping result verification.

2. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 1, characterized in that, S1 specifically includes: The binocular vision acquisition unit simultaneously acquires left and right view images, extracts the target image region through visual detection and image recognition, determines the target category, target specification markings and surface features, and determines the target to be captured and assigns target tracking markings according to the capture task table. Based on the calibration parameter file, the left and right view images are corrected and matched by binoculars. The three-dimensional position, direction of motion, velocity, velocity vector, three-dimensional attitude angle vector and spatial outer contour size of each target are calculated to form a three-dimensional motion state record.

3. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 2, characterized in that, Also includes: The neighborhood filtering distance is calculated with the target to be captured as the center, nearby dynamic targets are filtered, the three-dimensional motion state records are time-aligned and invalid records are removed, and a neighborhood dynamic target pose sequence is generated.

4. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 1, characterized in that, S3 also includes: aligning path nodes in different crawling branches, extracting common path segments before the fork and fork path nodes, and determining the initial execution path based on the preemptive safe time margin.

5. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 1, characterized in that, S4 specifically includes: Based on the initial execution path, an initial path joint control command sequence is generated to control the robot's movement, and new left and right view images are continuously acquired to update the neighborhood dynamic target pose sequence and the real-time robot execution status. The collision deviation evaluation value is calculated based on the updated neighborhood dynamic target pose sequence. The collision effect is confirmed by combining actual contact evidence. The actual motion branch or the uncovered actual motion branch is identified based on the branch matching value.

6. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 5, characterized in that, Also includes: Based on the actual motion branch and the real-time robot execution state, the executability of the branch switching is verified, and the switching result is generated, which is to switch to the continuing grasping branch, the corresponding collision takeover branch, or to execute the corresponding safe exit path.

7. The method for dynamic target grasping of industrial robots based on binocular vision collaboration according to claim 1, characterized in that, S5 specifically includes: Read the results of the grasping branch switching and the real-time robot execution status, resample the path nodes in the target grasping branch for control cycles, and generate a joint control command sequence; The robot is controlled to approach the target to be grasped according to the joint control command sequence, calculate the predicted position of the closing end, the relative position deviation and the relative posture deviation, perform amplitude limiting correction, and control the end effector to close when the closing trigger condition is met, thus forming the gripping state data; Based on the verification enhancement path segment, obtain the verification left and right view images, combine the clamping status data and synchronous movement error to verify the grasping results, generate grasping result data, and re-execute when grasping fails and the retry conditions are met.

Images