Control method and control system of flexible high-precision dual-arm body intelligent robot

Abstract

This invention discloses a control method and control system for a flexible, high-precision dual-arm omnidirectional intelligent robot. The control method includes: acquiring 3D point cloud data collected by a vision sensor; performing preprocessing, feature extraction, and target localization through a vision processing module to identify the spatial pose information of the task target and the end effector at the end of the second robotic arm; mapping the spatial pose information of the task target under the vision sensor to the work space of the second robotic arm to obtain the mapped pose; calculating the target pose and constraints of the two robotic arms according to a preset task type and the mapped pose; performing inverse kinematics calculations on the two robotic arms based on the target pose and constraints to generate the follow-up observation trajectory of the first robotic arm and the work trajectory of the second robotic arm; and synchronously sending control commands to the drive units of the first and second robotic arms to drive the two robotic arms to begin coordinated movement and implement the target task.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot control technology, and in particular to a control method and control system for a flexible, high-precision dual-armed intelligent robot. Background Technology

[0002] With the rapid development of industrial automation and intelligent robot technology, dual-armed intelligent robots, possessing collaborative operation capabilities similar to human arms, are increasingly widely used in various production and service scenarios. Existing dual-armed intelligent robot systems are typically equipped with visual perception modules to identify and locate the working environment and target workpieces. However, these visual perception modules are often fixedly installed, such as on a fixed bracket of the robot body or at a fixed position around the work scene, or on a motion mechanism with one or two degrees of freedom. For example, patent CN 111687885 A discloses an intelligent dual-armed robot system and visual guidance method for assembling disordered parts. This patent fixes a 3D vision structured light device on the top of the robot bracket, acquiring high-precision two-dimensional and three-dimensional data of the parts by projecting phase-shifting fringe structured light, and calculating the six-degree-of-freedom pose of the parts using a CAD model, thereby guiding the dual arms to grasp and assemble. This installation method limits the observation angle of the visual sensor, easily leading to blind spots when facing complex workpieces, resulting in insufficient positioning accuracy.

[0003] Among other existing technologies, patent CN207757622U discloses an assembly robot with binocular vision, in which the end effector and binocular camera are installed at the end of the same robot arm, using an eye-following-manual mode. In this mode, the field of view of the binocular camera is limited, and there is still a large blind spot. Patent CN211967519U provides a dual-arm robot assembly control system based on machine vision recognition. In this patent, a vision mechanism is installed at the end of the dual-arm assembly (on axis four), and the deviation value is calculated by taking pictures and calibrating with a camera and recognizing feature points, thereby guiding the dual arms to accurately install the parts. Although a vision mechanism is installed at the end of the robot arm in this technology, its control method is still limited to the eye-following-manual "picture-calculate deviation-teach assembly" mode of patent CN207757622U. It fails to define one of the robot arms as a high-degree-of-freedom mobile observation platform and does not give full play to the role of the vision mechanism at the end of the robot arm. Patent CN120816484A employs a configuration of a head-mounted global camera and left and right end-effector local cameras. It utilizes the head-mounted global camera and the local cameras installed at the ends of the working arm to acquire image streams and trains a generative strategy model through human demonstration data to achieve task generalization. However, it also adopts an "eye-follows-hand" mode for the end-effector local cameras, which is prone to visual blind spots due to end effector occlusion in complex assembly. Furthermore, this system mainly addresses the learning of operation strategies in unstructured environments and lacks the deterministic 3D visual servoing and accuracy compensation mechanism of this application, which utilizes an independent observation arm for active perspective optimization and precise tasks. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the shortcomings of the existing technology, the present invention provides a control method and control system for a flexible and high-precision dual-arm omnidirectional intelligent robot. It aims to solve the problems of limited observation position and angle, insufficient flexibility of visual perception, and inability to fully utilize the performance of high-precision sensors caused by the installation of visual sensors on a finite-degree-of-freedom motion mechanism, which in turn leads to low precision in dual-arm operation.

[0005] Technical Solution: To achieve the above objectives, the present invention provides a control method for a flexible, high-precision dual-arm omnidirectional intelligent robot. The dual-arm robot includes a first robotic arm, a second robotic arm, an end effector, and a vision sensor, wherein the vision sensor is a high-precision 3D binocular vision sensor. The end effector is mounted at the end of the second robotic arm, and the vision sensor is mounted at the end of the first robotic arm. Specifically, an end effector mounting bracket is mounted at the end of the first robotic arm, and the vision sensor is connected to the first robotic arm through the end effector mounting bracket. The control method is implemented by a robot controller, which includes a host computer and a slave computer. The control method includes:

[0006] The 3D point cloud data collected by the vision sensor is acquired, and preprocessed, feature extracted and target located by the vision processing module to identify the spatial pose information of the work target and the end effector at the end of the second robotic arm.

[0007] Based on the first transformation matrix between the base coordinate systems of the first robotic arm and the second robotic arm And the second transformation matrix between the base coordinates of the vision sensor and the first robotic arm. The spatial pose information of the target under the vision sensor is mapped to the working space of the second robotic arm to obtain the mapped pose.

[0008] Based on the preset task type and the mapped pose, the target poses and constraints of the two robotic arms are calculated: including the target pose, approach vector, and work path constraints of the end effector at the end of the second robotic arm, and the target observation pose of the vision sensor at the end of the first robotic arm; wherein, the work path constraint is the path that the end effector must take to complete a specific task, for example, if the end effector is a gripping mechanism, it needs to approach the target workpiece from a specific gripping direction along a specific path. The target observation pose is to ensure that the work area is at the optimal field of view depth and that the vision sensor is facing the target point.

[0009] Based on the target pose and constraints of the two robotic arms, inverse kinematics calculations are performed on the two robotic arms, the motion nodes of the two are aligned on the time axis, and interference from the first robotic arm and vision sensor on the operation of the second robotic arm is prevented, generating the follow-up observation trajectory of the first robotic arm and the operation trajectory of the second robotic arm.

[0010] Control commands are synchronously sent to the drive units of the first and second robotic arms to drive them to begin coordinated movement and perform the target task. During the coordinated movement, the pose nodes of the two robotic arms on the timeline are strictly followed to prevent self-collision.

[0011] Furthermore, before acquiring the 3D point cloud data collected by the visual sensor, the method further includes:

[0012] The global position of the workpiece to be operated is obtained by a 2D vision module deployed in the working environment, and the second robotic arm is guided to move with the end effector to a position close to the work target.

[0013] The first robotic arm is controlled to move the vision sensor closer to the target work point, so that the target work point is within the field of view of the vision sensor.

[0014] Furthermore, the generation of the target observation pose by the first robotic arm end-effector vision sensor includes:

[0015] The system analyzes the acquired 3D point cloud data in real time, calculates the completeness of the extracted key features of the target, and evaluates whether the current observation view meets the pre-set positioning threshold of the task. The key features of the target are features that can be directly used by the end effector and / or used as a positioning reference. The key features of the target are different for different end effectors.

[0016] When the evaluation result is lower than the positioning threshold, the optimal spatial observation pose that can cover the feature-deficient area is calculated and / or searched as the target observation pose, using the center of the target as a reference point. When searching for the optimal spatial observation pose, a dynamic scanning trajectory is generated based on the current pose, and the first robotic arm is controlled to move so that the vision sensor orbits the target. During the trajectory movement, an axial pointing constraint is applied so that the optical axis of the vision sensor always intersects the center of the target, ensuring that the target remains within the optimal field of view depth range during the follow-up scanning process.

[0017] Furthermore, the second transformation matrix The calculation formula is:

[0018] ;

[0019] in, The homogeneous transformation matrix between the end-effector mounting bracket and the base coordinate system of the first robotic arm; A fixed transformation matrix is ​​used to mount the visual sensor to the end sensor.

[0020] Furthermore, the process of driving the two robotic arms to begin coordinated movement includes:

[0021] Obtain the homogeneous coordinates of the task target and the end effector in the coordinate system of the vision sensor, respectively. as well as And calculate the spatial deviation between the two. ;

[0022] The subsequent movement trajectory of the second robotic arm is corrected in real time based on the spatial deviation.

[0023] Furthermore, when calculating the target pose and constraint conditions of the two robotic arms, both the first robotic arm assembly consisting of the first robotic arm and the vision sensor and the second robotic arm assembly consisting of the second robotic arm and the end effector are considered as dynamic obstacles.

[0024] When the two robotic arms start to move in coordination, the minimum spatial distance between the first robotic arm assembly and the second robotic arm assembly is calculated in real time. When the distance is lower than the safety threshold, the observation posture of the first robotic arm is adjusted first to avoid the working path of the second robotic arm, while maintaining the field of view of the vision sensor covering the target point.

[0025] A control system for a flexible, high-precision dual-armed intelligent robot, used to implement the aforementioned control method for the flexible, high-precision dual-armed intelligent robot, the control system comprising:

[0026] The visual data processing module is used to acquire 3D point cloud data collected by the visual sensor, perform preprocessing, feature extraction and target localization through the visual processing module, and identify the spatial pose information of the work target and the end effector at the end of the second robotic arm.

[0027] The pose conversion module is based on the first transformation matrix between the base coordinate systems of the first robotic arm and the second robotic arm. And the second transformation matrix between the base coordinates of the vision sensor and the first robotic arm. The spatial pose information of the target under the vision sensor is mapped to the working space of the second robotic arm to obtain the mapped pose.

[0028] The first calculation module, based on a preset task type and the mapped pose, calculates the target pose and constraints of the two robotic arms. These include the target pose, approach vector, and work path constraints of the end effector at the end of the second robotic arm, and the target observation pose of the vision sensor at the end of the first robotic arm. The work path constraint is the path the end effector must traverse to complete a specific task. For example, if the end effector is a gripping mechanism, it needs to approach the target workpiece from a specific gripping direction along a specific path. The target observation pose is designed to ensure the work area is at the optimal field of view depth and that the vision sensor is facing the target point.

[0029] The second calculation module performs inverse kinematics calculations on the two robotic arms based on their target poses and constraints, aligns their motion nodes on the time axis, and prevents the first robotic arm and vision sensors from interfering with the operation of the second robotic arm, generating the follow-up observation trajectory of the first robotic arm and the operation trajectory of the second robotic arm.

[0030] The collaborative control module synchronously sends control commands to the drive units of the first and second robotic arms, driving the two robotic arms to begin collaborative movement and perform the target task. During the collaborative movement, strictly adhering to the pose nodes of the two robotic arms on the timeline prevents self-collision.

[0031] Beneficial Effects: The control method and control system for the flexible, high-precision dual-armed intelligent robot of the present invention have the following beneficial effects:

[0032] (1) The present invention installs a high-precision 3D vision sensor at the end of the first robotic arm. The observation position and angle can be flexibly adjusted with the movement of the first robotic arm. This breaks through the problem of limited observation range of vision sensors under traditional fixed installation or limited degree of freedom installation methods. It can adjust the visual perception angle in real time according to the position, shape and progress of the work target, effectively avoid visual blind spots, realize all-round and flexible perception of the work environment and target, and greatly improve the flexibility and adaptability of visual perception.

[0033] (2) The hierarchical guidance mechanism combines global guidance through 2D vision module with local fine scanning through 3D vision sensor, which greatly shortens the search cycle in complex scenarios and ensures the accuracy of key workstations. It effectively solves the contradiction between perception range and operation accuracy in large-scale space, and improves the success rate and environmental adaptability of the system in irregular part operation.

[0034] (3) Through the visual servo function, the position and attitude of the second robotic arm can be calculated based on the target recognition results and the end effector recognition results, thereby avoiding the loss of accuracy due to calibration or the absolute position accuracy of the robotic arm itself, and further improving the accuracy of the operation.

[0035] (4) Through the flexible adjustment of visual perception and high-precision positioning capability, the present invention can adapt to different shapes and sizes of work targets and different layouts of work scenarios. There is no need to reinstall or adjust the visual sensor for specific tasks, which reduces the system's usage cost and debugging difficulty, and expands the application scope of the dual-arm body intelligent robot. Attached Figure Description

[0036] Figure 1 A structural diagram of a flexible, high-precision, dual-armed, body-worn intelligent robot.

[0037] Figure 2 A flowchart illustrating the control method for a flexible, high-precision dual-armed intelligent robot;

[0038] Figure 3 This is a schematic diagram of the control system for a flexible, high-precision, dual-armed, body-worn intelligent robot. Detailed Implementation

[0039] The invention will now be further described with reference to the accompanying drawings.

[0040] like Figure 1The control method for a flexible, high-precision dual-armed intelligent robot shown herein includes a torso 6, a first robotic arm 1, a second robotic arm 2, an end effector 3, and a vision sensor 4. The vision sensor 4 is a high-precision 3D binocular vision sensor. The first robotic arm 1 and the second robotic arm 2 are positioned on opposite sides of the torso 6. The end effector 3 is mounted at the end of the second robotic arm 2, and the vision sensor 4 is mounted at the end of the first robotic arm 1. Specifically, an end sensor mounting bracket 5 is mounted at the end of the first robotic arm 1, and the vision sensor 4 is connected to the first robotic arm 1 through the end sensor mounting bracket 5. The vision sensor 4 can be fixed to the end sensor mounting bracket 5 using bolts, clips, or other fixing methods. The vision sensor 4 employs structured light, time-of-flight (ToF), or binocular stereo vision 3D imaging technology, achieving a measurement accuracy of ±0.2mm or higher, which can meet the target positioning accuracy requirements of high-precision operation scenarios. The vision sensor 4 can detect not only the position and orientation of the work target (i.e., the workpiece being assembled, the parts being gripped, etc.), but also the position and orientation of the end effector 3 and the workpiece operated on the end effector 3.

[0041] The control method is implemented by the robot's controller, which includes a host computer and a slave computer. The controller has vision processing and dual-arm collaborative control functions. Specifically, the controller can receive 3D point cloud data transmitted by a high-precision 3D vision sensor, and perform preprocessing (such as noise reduction and filtering), feature extraction (such as workpiece edges, holes, contours, etc.), target localization, and posture recognition on the data to obtain the precise spatial position and posture information of the target. In addition, based on the target position and posture information output by the vision processing module, combined with the current position and motion parameters (such as speed, acceleration, joint angles, etc.) of the first and second robotic arms, it can generate the collaborative motion trajectory of the first and second robotic arms, and send control commands to the drive units of the first and second robotic arms to control the two robotic arms to collaboratively complete the task.

[0042] like Figure 2 As shown, the control method includes the following steps S101-S104:

[0043] Step S101: Acquire the 3D point cloud data collected by the vision sensor 4, perform preprocessing, feature extraction and target localization through the vision processing module, and identify the spatial pose information of the work target and the end effector 3 at the end of the second robotic arm 2.

[0044] Step S102, based on the first transformation matrix between the base coordinate systems of the first robotic arm 1 and the second robotic arm 2. And the second transformation matrix between the base coordinates of the vision sensor 4 and the first robotic arm 1. The spatial pose information of the target under the vision sensor 4 is mapped to the working space of the second robotic arm 2 to obtain the mapped pose.

[0045] Step S103: Based on the preset task type and the mapped pose, calculate the target pose and constraints of the two robotic arms. This includes the target pose, approach vector, and work path constraints of the end effector 3 at the end of the second robotic arm 2, and the target observation pose of the end vision sensor 4 at the end of the first robotic arm 1. The work path constraint is the path the end effector 3 must take to complete a specific task. For example, if the end effector 3 is a gripping mechanism, it needs to approach the target workpiece from a specific gripping direction along a specific path. The target observation pose is determined to ensure the work area is at the optimal field of view depth and that the vision sensor 4 is facing the target point.

[0046] Step S104: Based on the target pose and constraints of the two robotic arms, perform inverse kinematics calculation on the two robotic arms, align the motion nodes of the two on the time axis, and prevent the first robotic arm 1 and the vision sensor 4 from interfering with the operation of the second robotic arm 2, and generate the follow-up observation trajectory of the first robotic arm 1 and the operation trajectory of the second robotic arm 2.

[0047] In step S105, control commands are synchronously sent to the drive units of the first robotic arm 1 and the second robotic arm 2 to drive the two robotic arms to begin coordinated movement and perform the target task. During the coordinated movement, strictly adhering to the pose nodes of the two robotic arms on the timeline can prevent self-collision.

[0048] This control method integrates a high-precision 3D vision sensor into the end effector of the first robotic arm 1, enabling flexible adjustment of the observation angle as the operation progresses, effectively eliminating blind spots inherent in traditional fixed installation methods. By establishing a dual-arm collaborative strategy and unified coordinate mapping, precise hand-eye coordination is achieved. This not only leverages the performance advantages of the high-precision sensor but also ensures operational accuracy in complex scenarios, significantly improving the adaptability and success rate of the dual-arm robot in scenarios such as assembling irregularly shaped components.

[0049] Preferably, before acquiring the 3D point cloud data collected by the vision sensor 4 in step S101 above, the method further includes steps S201-S202:

[0050] Step S201: Obtain the global position of the workpiece to be operated by the 2D vision module deployed in the working environment, and guide the second robotic arm 2 with the end effector 3 to move to a position close to the work target.

[0051] Step S202: Control the first robotic arm 1 to move the vision sensor 4 closer to the target work point, so that the target work point is within the field of view of the vision sensor 4.

[0052] The aforementioned hierarchical guidance mechanism combines global guidance via a 2D vision module with local fine scanning via a 3D vision sensor, significantly shortening the search cycle in complex scenarios and ensuring the accuracy of key workstations. It effectively resolves the contradiction between the difficulty in balancing perception range and operational accuracy in large-scale spaces, and improves the system's success rate and environmental adaptability in irregular part operations.

[0053] Preferably, the generation of the target observation pose of the end vision sensor 4 of the first robotic arm 1 in step S103 includes the following steps S301-S302:

[0054] Step S301: Analyze the acquired 3D point cloud data in real time, calculate the completeness of the extraction of key target features, and evaluate whether the current observation view meets the preset positioning threshold of the task; where key target features are features that can be directly used by the end effector 3 and / or used as positioning references. The key target features are different for different end effectors 3.

[0055] Step S302: When the evaluation result is lower than the positioning threshold, using the center of the target as a reference point, calculate and / or find the optimal spatial observation pose that can cover the feature-deficient area as the target observation pose. When searching for the optimal spatial observation pose, generate a dynamic scanning trajectory based on the current pose, and control the first robotic arm 1 to move the vision sensor 4 around the target. During the trajectory movement, apply an axial pointing constraint that ensures the optical axis of the vision sensor 4 always intersects the target center, ensuring that the target is always within the optimal field of view depth range during the follow-up scanning process.

[0056] The above method can effectively solve the problem of recognition failure caused by occlusion or blind spots in complex working environments by introducing an adaptive observation mechanism based on point cloud quality assessment. By superimposing axial pointing constraints in the scanning path, it ensures that the vision sensor always maintains high-precision focusing on the core features of the target during multi-angle follow-up.

[0057] Preferably, in step S102 above, the second transformation matrix The calculation formula is:

[0058] ;

[0059] in, The homogeneous transformation matrix between the end sensor mounting bracket 5 and the base coordinate system of the first robotic arm 1; A fixed transformation matrix is ​​used to mount the visual sensor 4 to the end sensor mounting bracket 5.

[0060] Furthermore, the homogeneous transformation matrix between the end effector 3 and the base coordinate system of the second robotic arm is: The coordinate transformation relationship between the target point operated by the second robotic arm 2 and the second robotic arm 2 is as follows: ,in Let be the homogeneous coordinates of a fixed target in the coordinate system of visual sensor 4.

[0061] Preferably, the step S105 described above, which involves driving the two robotic arms to begin coordinated movement, includes the following steps S401-S402:

[0062] Step S401: Obtain the homogeneous coordinates of the task target and the end effector 3 in the coordinate system of the vision sensor 4. as well as And calculate the spatial deviation between the two. ;

[0063] Step S402: Correct the subsequent motion trajectory of the second robotic arm 3 in real time based on the spatial deviation.

[0064] Through the visual servo function, the position and posture of the second robotic arm 2 can be calculated based on the target recognition results and the end effector 3, according to the deviation results. This avoids the loss of accuracy caused by calibration or the absolute position accuracy of the robotic arm itself, and further improves the operation accuracy.

[0065] Furthermore, the controller directs the first robotic arm 1 to point the vision sensor 4 towards a preset feature area of ​​the second robotic arm 2 or a marker point on the end effector 3, acquiring its real-time 3D pose data in the visual coordinate system. The pose data fed back by the vision sensor 4 in real time is compared with the theoretical kinematic forward pose calculated based on the feedback from the joint encoder of the second robotic arm 2. The kinematic residual matrix of the two is calculated in a unified spatial coordinate system. Using this kinematic residual matrix, combined with the least squares method or Kalman filtering algorithm, the first transformation matrix between the base coordinate systems of the first and second robotic arms is corrected online. Simultaneously, the DH parameter table of the robotic arm is updated. Subsequently, based on the updated kinematic model, predictive compensation is performed on the dynamic trajectory planning in step S104 to eliminate static and dynamic mapping errors caused by mechanical structure deformation or environmental factors. By updating the kinematic model online instead of relying on preset fixed parameters, the system can maintain long-term high-precision operation stability, significantly reducing the cost of manual re-calibration. From the underlying algorithm architecture, it ensures the absolute positioning accuracy of the embodied intelligent robot in complex working conditions.

[0066] Preferably, when calculating the target pose and constraint conditions of the two robotic arms in step S103 above, both the first robotic arm assembly consisting of the first robotic arm 1 and the vision sensor 4 and the second robotic arm assembly consisting of the second robotic arm 2 and the end effector 3 are regarded as dynamic obstacles.

[0067] When driving the two robotic arms to start coordinating motion as described in step S105 above, the minimum spatial distance between the first robotic arm assembly and the second robotic arm assembly is calculated in real time. When the distance is lower than the safety threshold, the observation posture of the first robotic arm is adjusted first to avoid the working path of the second robotic arm, while maintaining the field of view of the vision sensor covering the target point.

[0068] The above method effectively solves the self-collision problem that may occur in high-degree-of-freedom dual-arm collaborative operations by treating the two arms and their end effectors as dynamic obstacles and calculating the shortest distance in space in real time. By prioritizing the adjustment of the observation posture of the first robotic arm as a dynamic avoidance strategy, the system automatically avoids the working trajectory of the second robotic arm while ensuring that the visual field always locks onto the target. This significantly enhances the system safety and smoothness of dual-arm collaboration in complex and confined spaces, and ensures continuous perception.

[0069] This invention also provides a control system for a flexible, high-precision dual-armed, sculpted intelligent robot. The control system may include or be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete this invention and implement the aforementioned control method for the flexible, high-precision dual-armed, sculpted intelligent robot. The program module referred to in this embodiment of the invention refers to a series of computer program instruction segments capable of performing specific functions, which are more suitable than the program itself for describing the execution process of the control method for the flexible, high-precision dual-armed, sculpted intelligent robot in the storage medium. The following description will specifically introduce the functions of each program module in this embodiment: such as... Figure 3 As shown, the control system includes:

[0070] The visual data processing module 510 is used to acquire 3D point cloud data collected by the visual sensor 4, perform preprocessing, feature extraction and target localization through the visual processing module, and identify the spatial pose information of the work target and the end effector 3 at the end of the second robotic arm 2.

[0071] The pose conversion module 520 is based on the first transformation matrix between the base coordinate systems of the first robotic arm 1 and the second robotic arm 2. And the second transformation matrix between the base coordinates of the vision sensor 4 and the first robotic arm 1. The spatial pose information of the target under the vision sensor 4 is mapped to the working space of the second robotic arm 2 to obtain the mapped pose.

[0072] The first calculation module 530 calculates the target poses and constraints of the two robotic arms based on a preset task type and the mapped poses. This includes the target pose, approach vector, and work path constraints of the end effector 3 at the end of the second robotic arm 2, and the target observation pose of the end vision sensor 4 at the end of the first robotic arm 1. The work path constraint is the path the end effector 3 must take to complete a specific task. For example, if the end effector 3 is a gripping mechanism, it needs to approach the target workpiece from a specific gripping direction along a specific path. The target observation pose is designed to ensure the work area is at the optimal field of view depth and that the vision sensor 4 is facing the target point.

[0073] The second calculation module 540 performs inverse kinematics calculations on the two robotic arms based on their target poses and constraints, aligns their motion nodes on the time axis, and prevents the first robotic arm 1 and the vision sensor 4 from interfering with the operation of the second robotic arm 2, generating the follow-up observation trajectory of the first robotic arm 1 and the operation trajectory of the second robotic arm 2.

[0074] The collaborative control module 550 synchronously sends control commands to the drive units of the first robotic arm 1 and the second robotic arm 2, driving the two robotic arms to begin collaborative movement and perform the target task. During the collaborative movement, strictly adhering to the pose nodes of the two robotic arms on the timeline can prevent self-collision.

[0075] Other aspects of implementing the above control method based on the above control system have been described in detail in the previous embodiments. Please refer to the corresponding content in the previous embodiments. They will not be repeated here.

[0076] The technical solution of the present invention will be described below with reference to examples:

[0077] The flexible, high-precision dual-arm omnidirectional intelligent robot system is used to complete the loading operation of small-sized automotive parts on the pre-electrophoresis suspension chain (parts with φ8mm hanging holes and hook end cross-sections of 4mm×4mm or 6mm×6mm). The specific implementation process is as follows:

[0078] 1. System Setup: The robot body adopts, for example... Figure 1 The industrial-grade dual-arm robot platform shown has two robotic arms, the first and the second, both of which are 6-DOF robotic arms with a repeatability of ±0.05mm. The vision sensor is a structured light 3D sensor with a measurement accuracy better than ±0.2mm in depth map, and is fixed to the end of the first robotic arm by a customized sensor mounting bracket. The controller is an industrial PC with built-in OpenCV-based vision processing algorithms and dual-arm collaborative control algorithms, and interacts with the first robotic arm, the second robotic arm, and the high-precision 3D vision sensor via Ethernet.

[0079] 2. Work preparation: Place the automotive parts to be shelved on the work platform, which has a 2D vision positioning module; initialize the control system, and after 2D vision positioning, control the second robotic arm to move above the parts and adjust the gripping angle to grip the workpiece.

[0080] 3. Target localization and recognition: The first robotic arm equipped with a vision sensor is controlled to move near the hanger, take a picture to identify the hook of the hanger, and the vision processing module of the control system performs noise reduction and filtering on the 3D point cloud data of the hook, extracts the outline features and entry point position features of the hook, determines the current position (X1,Y1,Z1) and attitude (rotation angle α1,β1,γ1) of the hook, and sends the positioning results to the dual-arm collaborative control module.

[0081] 4. Dual-arm collaborative gripping: The dual-arm collaborative control module generates the motion trajectory of the second robotic arm based on the positioning information of the hook, and controls the second robotic arm to move the gripped part above the hook in a specified posture and accurately insert it into the hook; at the same time, it controls the vision sensor of the first robotic arm to observe the hook from the side to ensure successful completion.

[0082] 5. Repeat steps 2-4 above until the entire rack loading operation is completed.

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A control method for a flexible, high-precision dual-armed robotic body, the dual-armed robot comprising a first robotic arm, a second robotic arm, an end effector, and a vision sensor; wherein the end effector is mounted at the end of the second robotic arm, characterized in that: The vision sensor is mounted on the end of the first robotic arm; the control method includes: The 3D point cloud data collected by the vision sensor is acquired, and preprocessed, feature extracted and target located by the vision processing module to identify the spatial pose information of the work target and the end effector at the end of the second robotic arm. Based on the first transformation matrix between the base coordinate systems of the first robotic arm and the second robotic arm And the second transformation matrix between the base coordinates of the vision sensor and the first robotic arm. The spatial pose information of the target under the vision sensor is mapped to the working space of the second robotic arm to obtain the mapped pose. Based on the preset task type and combined with the mapped pose, the target pose and constraints of the two robotic arms are calculated: including the target pose, proximity vector and work path constraints of the end effector of the second robotic arm, and the target observation pose of the end vision sensor of the first robotic arm. Based on the target pose and constraints of the two robotic arms, inverse kinematics calculations are performed on the two robotic arms to generate the follow-up observation trajectory of the first robotic arm and the operation trajectory of the second robotic arm. Control commands are sent synchronously to the drive units of the first and second robotic arms to drive the two robotic arms to begin coordinated movement and carry out the target task.

2. The control method for the flexible, high-precision dual-armed intelligent robot according to claim 1, characterized in that, Before acquiring the 3D point cloud data collected by the vision sensor, the method further includes: The global position of the workpiece to be operated is obtained by a 2D vision module deployed in the working environment, and the second robotic arm is guided to move with the end effector to a position close to the work target. The first robotic arm is controlled to move the vision sensor closer to the target work point, so that the target work point is within the field of view of the vision sensor.

3. The control method for the flexible, high-precision dual-armed intelligent robot according to claim 1, characterized in that, The generation of the target observation pose by the first robotic arm end-effector vision sensor includes: The system analyzes the acquired 3D point cloud data in real time, calculates the completeness of the extraction of key target features, and assesses whether the current observation view meets the pre-set positioning threshold of the task. When the evaluation result is lower than the positioning threshold, the optimal spatial observation pose that can cover the feature-deficient area is calculated and / or searched as the target observation pose, with the center of the operation target as the reference point.

4. The control method for the flexible, high-precision dual-armed intelligent robot according to claim 1, characterized in that, The second transformation matrix The calculation formula is: ； in, The homogeneous transformation matrix between the end-effector mounting bracket and the base coordinate system of the first robotic arm; A fixed transformation matrix is ​​used to mount the visual sensor to the end sensor.

5. The control method for the flexible, high-precision dual-armed intelligent robot according to claim 1, characterized in that, The process of driving the two robotic arms to begin coordinated movement includes: Obtain the homogeneous coordinates of the task target and the end effector in the coordinate system of the vision sensor, respectively. as well as And calculate the spatial deviation between the two. ; The subsequent movement trajectory of the second robotic arm is corrected in real time based on the spatial deviation.

6. The control method for the flexible, high-precision dual-armed intelligent robot according to claim 1, characterized in that, When calculating the target pose and constraint conditions of the two robotic arms, both the first robotic arm assembly consisting of the first robotic arm and the vision sensor and the second robotic arm assembly consisting of the second robotic arm and the end effector are considered as dynamic obstacles. When the two robotic arms start to move in coordination, the minimum spatial distance between the first robotic arm assembly and the second robotic arm assembly is calculated in real time. When the distance is lower than the safety threshold, the observation posture of the first robotic arm is adjusted first to avoid the working path of the second robotic arm, while maintaining the field of view of the vision sensor covering the target point.

7. A control system for a flexible, high-precision dual-armed intelligent robot, used to implement the control method for the flexible, high-precision dual-armed intelligent robot according to any one of claims 1-6, characterized in that, The control system includes: The visual data processing module is used to acquire 3D point cloud data collected by the visual sensor, perform preprocessing, feature extraction and target localization through the visual processing module, and identify the spatial pose information of the work target and the end effector at the end of the second robotic arm. The pose conversion module is based on the first transformation matrix between the base coordinate systems of the first robotic arm and the second robotic arm. And the second transformation matrix between the base coordinates of the vision sensor and the first robotic arm. The spatial pose information of the target under the vision sensor is mapped to the working space of the second robotic arm to obtain the mapped pose. The first calculation module calculates the target poses and constraints of the two robotic arms based on the preset task type and the mapped poses: including the target pose, proximity vector and work path constraints of the end effector of the second robotic arm, and the target observation pose of the end vision sensor of the first robotic arm. The second calculation module performs inverse kinematics calculations on the two robotic arms based on their target poses and constraints, generating the follow-up observation trajectory of the first robotic arm and the operation trajectory of the second robotic arm. The collaborative control module synchronously sends control commands to the drive units of the first and second robotic arms, driving the two robotic arms to begin collaborative movement and carry out the target task.

Images