A pose loop closure planning system and method based on PLC six-face vision quality inspection

Abstract

The application discloses a kind of pose closed loop planning system and method based on PLC six visual quality inspection, it is related to industrial robot quality inspection technical field, including: visual perception module, for obtaining RGB-D image;Pose closed loop control module, for receiving RGB-D image and output 6D pose data;Motion planning module, for generating double-arm motion trajectory;Flip optimization module, for generating optimal flip sorting set;Sorting execution module, for completing sorting and discharging action according to quality inspection result and control instruction;Robot controller, for data interaction and issue control instruction.The application does not need to rely on artificial flip or multiple fixtures, reduce the failure rate caused by incoming material position deviation etc., through double-arm alternate operation and Monte Carlo optimization flip sorting, realize PLC device six automatic continuous detection, with the combination of coarse positioning, fine positioning of double RGB-D camera, combined with the output 6D pose data of deep learning pose estimation model, effectively avoid perspective distortion and miss detection, false detection.

Description

Technical Field

[0001] This invention relates to the field of industrial robot quality inspection technology, specifically a pose closed-loop planning system and method based on PLC six-sided vision quality inspection. Background Technology

[0002] Currently, the appearance quality inspection of PLCs (Programmable Logic Controllers) in the 3C industry mainly adopts the following two solutions: First, manual flipping combined with a single-sided fixed camera. Workers flip the PLC six sides in sequence and take a picture of each side at a fixed station. This method is inefficient and may miss some inspections, significantly increasing the labor intensity. Second, a single-arm SCARA / DELTA robot is used with a special fixture. A two-finger gripper is installed on the robot's end effector, and 1-2 industrial cameras are used to complete the "grab-move-photograph" operation. However, SCARA only has 4 degrees of freedom, which requires the design of multiple flipping fixtures to complete the 6-degree-of-freedom inspection operation. The changeover time is long, and the single-arm robot cannot achieve gravity self-balancing when flipping heavy PLCs (>1.2kg), resulting in image jitter and blurry focus, which limits the inspection accuracy.

[0003] Based on this, a pose closed-loop planning system and method based on PLC six-sided visual quality inspection is provided, which can eliminate the drawbacks of existing technical solutions. Summary of the Invention

[0004] The purpose of this invention is to provide a pose closed-loop planning system and method based on PLC six-sided visual quality inspection, so as to solve the problems of long inspection time, difficulty in full six-sided coverage and poor size adaptability in the quality inspection process in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A pose closed-loop planning system based on PLC six-sided visual quality inspection includes:

[0007] The visual perception module is used to capture scene images of the material receiving platform and acquire RGB-D images containing the target PLC device. The target PLC device is placed on the material receiving platform. The visual perception module includes a head RGB-D camera and a wrist RGB-D camera. The head RGB-D camera is deployed at the center of the head of the dual-arm robot, and the wrist RGB-D camera is deployed on the side of the gripper at the end of the dual-arm robot's end effector.

[0008] The pose closed-loop control module is used to receive RGB-D images and output 6D pose data of the target PLC device;

[0009] The motion planning module is used to generate the motion trajectory of the robot's two arms using a collision-free path planning algorithm based on the pose data output by the pose closed-loop control module and the optimal flipping sort set generated by the flipping optimization module, while simultaneously performing vibration suppression control in conjunction with vibration sensor data.

[0010] The flipping optimization module is used to obtain the size, mass parameters and current pose data of the target PLC device, and generate the optimal flipping sort set by combining the evaluation function and Monte Carlo sampling mechanism.

[0011] The sorting execution module is used to control the end effector to complete the sorting and unloading of good or defective target PLC devices according to the quality inspection results and control instructions.

[0012] The robot controller, built into the dual-arm robot, is used to interact with various modules and end effectors via a network and to issue control commands.

[0013] The posture closed-loop control module, motion planning module, flipping optimization module, and sorting execution module are all integrated inside the dual-arm robot and interact with the vision perception module via a network. The sorting execution module is connected to the end effector via a CAN bus, and the end effector receives sorting instructions and performs unloading actions.

[0014] Furthermore, each of the two robotic arms of the dual-arm robot includes a shoulder rotation joint, an elbow rotation joint, and a wrist rotation joint. The robotic arm bases are symmetrically mounted on both sides of the material receiving platform, and the center distance of the robotic arm bases is adapted to the size of the material receiving platform, so that the working space of the dual arms covers the entire detection area. A vibration sensor is installed on the gripper at the end of the end effector. The output end of the vibration sensor is connected to the end effector, and the vibration data collected in real time is transmitted to the robot controller through the end effector.

[0015] Furthermore, the pose closed-loop control module is equipped with a deep learning pose estimation model. The input of the deep learning pose estimation model is an RGB-D image acquired by an RGB-D camera, and the output is 6D pose data of the target PLC device. The 6D pose data includes position coordinates and attitude angles.

[0016] Furthermore, the robot controller has a predefined database built-in, which is used to store the specifications of the target PLC device, standard six-sided image templates, detection ROI parameters, defect judgment standard thresholds, joint motion parameters of the dual-arm robot, RGB-D camera calibration parameters, and historical quality inspection data.

[0017] A pose closed-loop planning method based on PLC six-sided visual quality inspection is applied to a pose closed-loop planning system based on PLC six-sided visual quality inspection, and specifically includes the following steps:

[0018] Step S1: Perform coarse positioning operation for incoming materials: The head RGB-D camera scans the incoming material platform to obtain an RGB-D image containing the target PLC device, and outputs the initial 6D pose data of the target PLC device through a deep learning pose estimation model.

[0019] Step S2, Arm Selection and Coarse Path Planning: The motion planning module calculates the load margin index of the two arms based on the mass of the target PLC device and the radius of the envelope ball. Based on the calculation results, the robot arm with the larger load margin index value is selected as the first grasping arm. The RRT-Connect algorithm is used to generate a collision-free path, and the collision-free path is transmitted to the robot controller and the end effector.

[0020] Step S3: Perform precise grasping pose estimation operation: The wrist RGB-D camera scans the target PLC device, acquires the RGB-D image of the target PLC device, and outputs the precise 6D pose data of the target PLC device through the deep learning pose estimation model;

[0021] Step S4: Perform clamping and vibration suppression operation: The gripper gradually approaches the target PLC device. When the contact force reaches the preset contact threshold, it switches to force holding mode. The vibration sensor detects vibration data in real time. If the vibration amplitude exceeds the preset vibration threshold, the corresponding robot arm performs active reverse acceleration compensation.

[0022] Step S5: Perform single-sided detection operation: The first grasping arm moves the target PLC device to the shooting area in front of the head RGB-D camera, calculates the relative pose by combining the current 6D pose data and the execution pose data, and generates a smooth trajectory through a secondary planning algorithm to complete the shooting of the surface to be tested.

[0023] Step S6: Perform Monte Carlo flipping optimization operation: Using the maximum load margin index value, the fewest number of double arm alternations, and the shortest path as the evaluation function, obtain the optimal flipping sort set of the remaining five untested faces through the Monte Carlo sampling mechanism;

[0024] Step S7: Perform the alternating flipping operation with both arms: Perform the arm switching operation according to the optimal flipping sort set. The idle robot arm and the first gripping arm complete the handover at the preset arm switching point. The idle robot arm clamps the next test surface of the target PLC device and repeats the operation of step S5 to align the next test surface with the shooting area in front of the head RGB-D camera.

[0025] Step S8: Perform sorting and unloading operation: Repeat steps S5 to S7 until the six sides of the target PLC device are photographed. The sorting execution module will then send the target PLC device to the corresponding good or bad product area based on the quality inspection results after comparing the six-sided images of the target PLC device with the standard six-sided image template, thus completing the visual quality inspection process of the target PLC device.

[0026] Furthermore, step S5 specifically includes:

[0027] The first gripper moves the target PLC device to the shooting area in front of the head RGB-D camera. The head RGB-D camera scans the target PLC device and acquires RGB-D images. The current 6D pose data of the target PLC device is output through a deep learning pose estimation model.

[0028] Based on the joint angles of the robotic arm, the execution pose data of the end effector is calculated using positive kinematics, and the relative pose between the current 6D pose data and the execution pose data is calculated.

[0029] Based on relative pose and positive kinematics, the target pose data required when the normal vector of the surface to be tested coincides with the optical axis of the head RGB-D camera is calculated, so that the target PLC device surface and the head RGB-D camera's requested positioning accuracy are within the preset detection distance tolerance range. The preset detection distance ensures that the captured image meets the defect judgment standard threshold requirements.

[0030] A smooth trajectory is generated using a quadratic programming algorithm to complete the imaging of the surface to be measured.

[0031] Furthermore, the formula for calculating the load margin index in step S2 is as follows:

[0032] ;

[0033] in, This is the load margin index value. For robotic arms The maximum torque of the joint, For robotic arms Jacobian transpose matrix, It is a load force vector and satisfies , For the quality of the target PLC device, This is the acceleration due to gravity.

[0034] Furthermore, the calculation formula for the evaluation function in step S6 is as follows:

[0035] ;

[0036] in, For the evaluation function, This represents the average load margin index value for both arms. The number of times the arms alternate. This is the total path length. This serves as the baseline value for path length. , , The weighting coefficients are satisfied. .

[0037] Furthermore, the preset contact threshold in step S4 is 30N, and the preset vibration threshold is 0.1mm.

[0038] Furthermore, the preset detection distance tolerance in step S5 is ±3mm.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] 1. This pose closed-loop planning system based on PLC six-sided visual quality inspection does not rely on manual flipping or multiple sets of special flipping fixtures, reducing the grasping failure rate caused by material position deviation and robot absolute positioning error. Through the alternating operation of dual-arm robots and Monte Carlo optimization of flipping sorting, it realizes automated continuous detection of PLC devices on six sides, ensuring that the vibration amplitude meets the preset requirements when the camera is shooting.

[0041] 2. This invention utilizes a combination of coarse and fine positioning using dual RGB-D cameras on the head and wrist, combined with a deep learning pose estimation model to output 6D pose data, thereby achieving precise alignment between the surface to be measured and the camera optical axis, controlling the detection distance, and effectively avoiding viewpoint distortion, missed detections, and false detections.

[0042] 3. This invention selects the optimal gripping arm by calculating the load margin index, and with the alternating flipping of the two arms, it can adapt to PLC devices of different length, width, height and weight without changing the fixture. This solves the problem that single-arm robots cannot balance the gravity of flipping heavy PLC devices and reduces the gripping failure rate caused by the deviation of the incoming material size. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the pose closed-loop planning system of the present invention.

[0044] Figure 2 This is a schematic diagram illustrating the steps of the pose closed-loop planning method of the present invention.

[0045] Figure labeling: Visual perception module 10, pose closed-loop control module 20, motion planning module 30, flipping optimization module 40, sorting execution module 50, robot controller 60. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0047] Example 1

[0048] In this embodiment, as Figure 1 As shown, this invention provides a pose closed-loop planning system based on PLC six-sided visual quality inspection, comprising:

[0049] The vision perception module 10 is used to capture scene images of the material receiving platform and acquire RGB-D images containing the target PLC device. The target PLC device is placed on the material receiving platform. The vision perception module 10 includes a head RGB-D camera and a wrist RGB-D camera. The head RGB-D camera is deployed at the center of the head of the dual-arm robot, and the wrist RGB-D camera is deployed on the side of the gripper at the end of the end effector of the dual-arm robot. The head RGB-D camera is responsible for global coarse positioning and can quickly scan the entire material receiving platform to obtain the overall position information of the target PLC device. The wrist RGB-D camera moves with the end effector to scan the target PLC device at close range and acquire higher resolution local images to ensure the accuracy of the grasping pose calculation. The two cameras work together to ensure positioning efficiency while also taking into account positioning accuracy.

[0050] The pose closed-loop control module 20 is used to receive RGB-D images and output 6D pose data of the target PLC device. In step S1, it receives RGB-D images from the head RGB-D camera and outputs the initial 6D pose data of the target PLC device through a deep learning pose estimation model. The data is sent to the motion planning module 30 for coarse path planning. In step S3, the RGB-D image from the wrist RGB-D camera is received, and the precise 6D pose data is output through the same deep learning pose estimation model. The data is sent to the motion planning module 30 for precise grasping, and the current 6D pose data is received in step S4. And robot joint angles, calculate execution pose data and relative pose ;

[0051] The motion planning module 30 is used to generate the motion trajectory of the robot's two arms using a collision-free path planning algorithm based on the pose data output by the pose closed-loop control module 20 and the optimal flipping sort set generated by the flipping optimization module 40. Simultaneously, it performs vibration suppression control by combining vibration sensor data. The motion planning module 30 is based on the initial 6D pose data. Generate a collision-free coarse path based on precise 6D pose data. Generate a precise grasping path based on the target pose data. A quadratic smooth trajectory is generated. In step S4, the load margin calculation result of the flipping optimization module 40 is received. The robot arm with a larger load margin is selected to perform the first grasp. Real-time vibration data (RMS value) of the vibration sensor is received. When RMS>0.1mm, a reverse acceleration compensation command is generated to control the end effector to suppress vibration and ensure that the vibration amplitude at the moment of shooting is <0.05mm.

[0052] The flipping optimization module 40 is used to obtain the size, mass parameters and current pose data of the target PLC device, and generate the optimal flipping sort set through the Monte Carlo sampling mechanism in combination with the evaluation function. In step S2, the size and weight parameters of the target PLC device are read from the database of the robot controller 60, and the current joint status and load margin data of the two arms are obtained. In step S6, the optimal flipping sort set is filtered by the evaluation function through 5000 Monte Carlo samplings. In step S7, the optimal flipping sort set and the alternation command of the two arms are output to the motion planning module 30 to avoid extreme extension and collision.

[0053] The sorting execution module 50 is used to control the end effector to complete the sorting and unloading of good or bad target PLC devices according to the quality inspection results and control instructions. It sends clamping / releasing instructions to the end effector via CAN bus, controls the target PLC devices to be placed in place, receives the position feedback signal from the photoelectric sensor of the material box and sends it back to the robot controller 60.

[0054] The robot controller 60, built into the dual-arm robot, is used to interact with various modules and end effectors via the network, issue control commands, synchronize the working sequence of each module, and handle abnormal situations.

[0055] The posture closed-loop control module 20, motion planning module 30, flipping optimization module 40 and sorting execution module 50 are all integrated inside the dual-arm robot and interact with the vision perception module 10 through the network. The sorting execution module 50 is connected to the end effector through the CAN bus. The end effector receives sorting instructions and performs unloading actions.

[0056] In this embodiment, each of the two robotic arms of the dual-arm robot includes a shoulder rotation joint, an elbow rotation joint, and a wrist rotation joint. The robotic arm bases are symmetrically installed on both sides of the material receiving platform, and the center distance of the robotic arm bases is adapted to the size of the material receiving platform, so that the working space of the dual arms covers the entire detection area. A vibration sensor is installed on the gripper at the end of the end effector. The output end of the vibration sensor is connected to the end effector, and the vibration data collected in real time is transmitted to the robot controller 60 through the end effector.

[0057] In this embodiment, the pose closed-loop control module 20 is equipped with a deep learning pose estimation model (FFB6D). The input of the deep learning pose estimation model is an RGB-D image acquired by an RGB-D camera, and the output is 6D pose data of the target PLC device. The 6D pose data includes position coordinates and attitude angles.

[0058] In this embodiment, the robot controller 60 has a predefined database built-in. The database is used to store the specifications of the target PLC device, standard six-sided image templates, detection ROI parameters, defect judgment standard thresholds, joint motion parameters of the dual-arm robot, RGB-D camera calibration parameters, and historical quality inspection data. The specifications of the target PLC device include, but are not limited to, the model, length, width, height, weight, center of gravity coordinates, and shell material properties of several PLC devices. The defect judgment standard thresholds include, but are not limited to, scratch length and color difference value. The joint motion parameters of the dual-arm robot include, but are not limited to, the maximum torque of the dual-arm joints, the Jacobian matrix reference value, repeatability, and workspace boundary coordinates. The RGB-D camera calibration parameters include, but are not limited to, the camera intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (transformation matrix with robot coordinate system). The historical quality inspection data includes, but is not limited to, the detection time, PLC device model, detection result (good / defective product), defect type / location / size, and operator information.

[0059] Example 2

[0060] The difference from Example 1 is that, as Figure 2 As shown, the present invention also provides a pose closed-loop planning method based on PLC six-sided visual quality inspection, applied to the above-mentioned pose closed-loop planning system, specifically including the following steps:

[0061] Step S1: Perform coarse positioning operation for incoming materials: The head RGB-D camera scans the incoming material platform to obtain an RGB-D image containing the target PLC device, and outputs the initial 6D pose data of the target PLC device through a deep learning pose estimation model.

[0062] Step S2, Arm Selection and Coarse Path Planning: The motion planning module 30 calculates the load margin index of the two arms based on the mass of the target PLC device and the radius of the envelope sphere. According to the calculation results, the robot arm with the larger load margin index value is selected as the first grasping arm. The RRT-Connect algorithm is used to generate a collision-free path and transmits the collision-free path to the robot controller 60 and the end effector. The RRT-Connect algorithm is an improved version of the RRT (Rapid Expanding Random Tree) algorithm and is a classic algorithm for robot path planning, which is the existing technology.

[0063] Step S3: Perform precise grasping pose estimation operation: The wrist RGB-D camera scans the target PLC device to obtain the RGB-D image of the target PLC device, and outputs the precise 6D pose data of the target PLC device through the deep learning pose estimation model.

[0064] Step S4: Perform clamping and vibration suppression operation: The gripper gradually approaches the target PLC device. When the contact force reaches the preset contact threshold, it switches to force holding mode. The vibration sensor detects the vibration data in real time. If the vibration amplitude exceeds the preset vibration threshold, the corresponding robot arm performs active reverse acceleration compensation. The preset contact threshold is 30N and the preset vibration threshold is 0.1mm.

[0065] Step S5: Perform single-sided inspection operation: The gripping arm moves the target PLC device to the shooting area in front of the head RGB-D camera. The relative pose is calculated by combining the current 6D pose data and the execution pose data. A smooth trajectory is generated by the quadratic planning algorithm to complete the shooting of the surface to be tested.

[0066] Step S6: Perform Monte Carlo flipping optimization operation: Using the maximum load margin index value, the fewest number of double arm alternations, and the shortest path as the evaluation function, obtain the optimal flipping sort set of the remaining five untested faces through the Monte Carlo sampling mechanism;

[0067] Step S7: Perform the alternating flipping operation with both arms: Perform the arm switching operation according to the optimal flipping sort set. The idle robot arm waits at the arm switching point until the first gripping arm moves the target PLC device to the preset arm switching point. The idle robot arm clamps the next test surface of the target PLC device. The first gripping arm releases the target PLC device. The idle robot arm and the first gripping arm complete the handover at the preset arm switching point. The idle robot arm repeats the operation of step S5 to align the next test surface with the shooting area in front of the head RGB-D camera.

[0068] Step S8: Perform sorting and unloading operation: Repeat steps S5 to S7 until the six sides of the target PLC device are photographed. The sorting execution module 50 sends the target PLC device to the corresponding good or bad product area based on the quality inspection result after comparing the six sides of the target PLC device with the standard six sides image template, thus completing the visual quality inspection process of the target PLC device.

[0069] Specifically, the process of comparing the six-sided image of the target PLC device with the standard six-sided image template includes:

[0070] The head-mounted RGB-D camera captures six-sided images of the target PLC device and transmits them to the back-end terminal (a computing device that communicates with the robot controller 60). The back-end terminal is equipped with a standard image preprocessing module and a defect detection module. After noise reduction, enhancement, and correction operations by the image preprocessing module, the six-sided images are input to the defect detection module. Detection of defect types includes scratches / dents, missing materials / damage, blurred / misprinted characters, color differences / stains, and assembly defects (such as misaligned interfaces). Defects are detected through edge detection, morphological operations, template matching, area calculation, and color... Color space analysis and geometric measurement are used to detect six-sided images. The detection results are compared with the judgment standard template built into the back-end terminal, and the conclusion of "good product" or "defective product" is output. At the same time, the location, size and type of defects are output. The detection results are fed back to the robot controller 60 in real time via network signal, and then transmitted to the sorting execution module 50. According to the feedback results, the sorting execution module 50 puts the corresponding PLC device into the corresponding bin (good product area / defective product area). The photoelectric sensor of the bin confirms that it is placed in place, and the entire quality inspection process is completed.

[0071] In this embodiment, step S5 specifically includes:

[0072] The first gripper moves the target PLC device to the shooting area in front of the head RGB-D camera. The head RGB-D camera scans the target PLC device and acquires RGB-D images. The current 6D pose data of the target PLC device is output through a deep learning pose estimation model.

[0073] Based on the joint angles of the robotic arm, the execution pose data of the end effector is calculated using forward kinematics. The relative pose between the current 6D pose data and the execution pose data is then calculated. Forward kinematics is the process of solving the pose of the end effector (gripper) in the base coordinate system given the joint angles of the robot, and it is the existing technology.

[0074] Based on relative pose and forward kinematics, the target pose data required for the normal vector of the surface under test to coincide with the optical axis of the head RGB-D camera is calculated. This ensures that the target PLC device surface and the head RGB-D camera's required positioning accuracy are within the preset detection distance tolerance range. The preset detection distance ensures that the captured image meets the defect judgment standard threshold requirement. The preset detection distance is 3mm.

[0075] A smooth trajectory is generated using a quadratic programming algorithm to complete the imaging of the surface to be measured;

[0076] Specifically, this step overcomes the positioning accuracy bottleneck of traditional single-arm / fixture solutions, achieving precise alignment and stable imaging between the target PLC device's test surface and the head-mounted RGB-D camera. This invention acquires current 6D pose data through re-enhancing imaging using the head-mounted RGB-D camera. Combine joint angles to obtain execution pose data Then calculate the relative pose. ( , To execute the inverse matrix of pose data, thereby offsetting the absolute positioning error of the dual-arm robot, the residual deviation of the incoming material position, eliminating the pose offset caused by the size error of the target PLC device, and avoiding the accumulation of errors caused by single-segment pose estimation;

[0077] Furthermore, this invention also utilizes relative pose. Inverse kinematics of target pose data Force the normal vector of the test surface of the target PLC device to be... Align the optical axis with the head RGB-D camera to ensure distortion-free shooting angle. Strictly control the distance tolerance between the surface to be tested and the head RGB-D camera within ±3mm (±3mm is the visual inspection requirement for object distance) to ensure that defects can be clearly captured and avoid missed or false detections due to alignment deviations.

[0078] Furthermore, this invention generates a 50ms short-cycle smooth trajectory online using a quadratic programming algorithm (QP), instead of a traditional broken line trajectory. This ensures a continuous trajectory and controllable acceleration, avoiding impacts and vibrations during the movement of the dual-arm robot. Combined with the vibration suppression mechanism in step S4, it further ensures that the amplitude of the end-effector vibration is less than the preset vibration threshold at the moment of shooting. This solves the problem of image defocusing caused by vibration after the large-weight PLC (>1.2kg) is flipped, ensuring image clarity. The 50ms short-cycle trajectory ensures a compact detection cycle, avoiding slowing down the process due to excessive alignment time.

[0079] In this embodiment, the formula for calculating the load margin index in step S2 is as follows:

[0080] ;

[0081] in, The load margin index value ( Corresponding to the left arm, (Corresponding to the right arm), reflecting the redundancy capability of the current robotic arm when carrying the current target PLC device. For robotic arms The maximum torque of the joint is determined, and the minimum value among all the maximum torques of the joints is taken to ensure that the joint does not exceed the limit. For robotic arms The Jacobian transpose matrix is ​​used to establish the mapping relationship between the end load force and the joint torque. It is a load force vector and satisfies The first three elements of the vector correspond to the translational force components (X, Y, and Z axes) in a Cartesian coordinate system (the robot's base coordinate system), and the last three elements correspond to the rotational torque components (since the gravity of the target PLC device is a concentrated force parallel to the Z-axis, the rotational torque component is 0). The negative sign indicates that the direction of gravity is opposite to the positive direction of the Z-axis (the Z-axis is vertically upward, which is positive), ensuring that the direction of the force vector is consistent with the actual physical scene. For the quality of the target PLC device, This is the acceleration due to gravity, with a default value of 9.8 m / s². 2 It supports calibration based on usage scenarios. The L2 norm (Euclidean norm) of the vector is used to calculate the composite amplitude of the joint torque;

[0082] Specifically, the load margin index value The compatibility of the robotic arm with the target PLC device is quantified by the ratio of the difference between the maximum load capacity and the current actual load requirement. When the torque margin of the robotic arm joints approaches 1, the motion stability is high and the vibration risk is low, making it the preferred initial gripping arm. When the load margin is close to zero, the robotic arm approaches its load limit, which can easily lead to problems such as joint over-limit and increased end-effector vibration. Therefore, this arm should not be selected as the main gripping arm. The motion planning module 30 compares the load margin index values ​​of the two arms. Select the robotic arm with the larger value to perform the first grasp;

[0083] Envelope radius The dimensions (length, width, and height) of the target PLC device are retrieved from the database and derived accordingly. , , , These are the length, width, and height of the target PLC device, respectively. These values ​​are used to assist in correcting the Jacobian transpose matrix. The radius of the envelope sphere... It can reflect the spatial dimensions of the target PLC device, when the radius of the envelope sphere... When the value is large, the change in the lever arm length between the end effector of the robotic arm and the center of gravity of the target PLC device needs to be addressed through… The position parameters in the Jacobian matrix are corrected to ensure that the calculated joint torque is more consistent with the actual load-bearing scenario.

[0084] In this embodiment, the calculation formula for the evaluation function in step S6 is:

[0085] ;

[0086] in, For the evaluation function, The smaller the value, the better the flip sequence. This represents the average load margin index value for both arms. The number of times the arms alternate (e.g., left arm → right arm → left arm) is used. (2) The total path length is the sum of the joint space path lengths for all flipping movements. The baseline value for path length (the average path length of all possible sequences, used for normalization). , , The weighting coefficients are satisfied. The weighting coefficients can be dynamically adjusted according to actual production needs;

[0087] Specifically, the Monte Carlo sampling mechanism generates a large number of possible flipping sequences through 5,000 random samplings. Each sequence must meet the constraints of no collision in the dual-arm working space and accurate alignment of the test surface. The optimal flipping sequence set selected by this mechanism can minimize the number of arm switching operations and movement paths while ensuring load safety, avoid invalid actions, and further compress the quality inspection cycle.

[0088] In summary, this invention constructs an automated, high-precision, and highly adaptable PLC six-sided visual quality inspection solution through dual-camera collaborative localization in the visual perception module 10, deep learning-based accurate estimation in the pose closed-loop control module 20, collision-free path generation in the motion planning module 30, Monte Carlo sequence optimization in the flipping optimization module 40, and collaborative operation of a dual-arm robot. This solution solves the problems of low efficiency and high risk of missed inspection in traditional manual quality inspection, as well as the dependence on fixtures and poor size adaptability of single-arm robots. It can effectively improve quality inspection efficiency and quality, reduce production costs, and has good application prospects.

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A pose closed-loop planning system based on PLC six-sided visual quality inspection, characterized in that, include: The visual perception module (10) is used to capture scene images of the material receiving platform and obtain RGB-D images containing the target PLC device. The target PLC device is placed on the material receiving platform. The visual perception module (10) includes a head RGB-D camera and a wrist RGB-D camera. The head RGB-D camera is deployed at the center of the head of the dual-arm robot, and the wrist RGB-D camera is deployed on the side of the gripper at the end of the end effector of the dual-arm robot. The pose closed-loop control module (20) is used to receive RGB-D images and output 6D pose data of the target PLC device; The motion planning module (30) is used to generate the motion trajectory of the robot's two arms using a collision-free path planning algorithm based on the pose data output by the pose closed-loop control module (20) and the optimal flipping sort set generated by the flipping optimization module (40), and to perform vibration suppression control in conjunction with vibration sensor data. The flipping optimization module (40) is used to obtain the size, mass parameters and current pose data of the target PLC device. It combines the evaluation function of the maximum load margin index value, the minimum number of double arm alternations and the shortest path, and generates the optimal flipping sort set through the Monte Carlo sampling mechanism. The sorting execution module (50) is used to control the end effector to complete the sorting and unloading of good or bad target PLC devices according to the quality inspection results and control instructions. The robot controller (60) is built into the dual-arm robot and is used to interact with each module and end effector via the network and issue control commands. The posture closed-loop control module (20), motion planning module (30), flipping optimization module (40) and sorting execution module (50) are all integrated inside the dual-arm robot and interact with the vision perception module (10) via the network. The sorting execution module (50) is connected to the end effector via the CAN bus. The end effector receives the sorting instructions and performs the unloading action.

2. The pose closed-loop planning system based on PLC six-sided visual quality inspection according to claim 1, characterized in that, The two arms of the dual-arm robot each include a shoulder rotation joint, an elbow rotation joint, and a wrist rotation joint. The arm bases are symmetrically installed on both sides of the material receiving platform, and the center distance of the arm bases is adapted to the size of the material receiving platform, so that the working space of the two arms covers the entire detection area. The end effector has a vibration sensor installed on the gripper at the end. The output end of the vibration sensor is connected to the end effector, and the vibration data collected in real time is transmitted to the robot controller (60) through the end effector.

3. The pose closed-loop planning system based on PLC six-sided visual quality inspection according to claim 1, characterized in that, The pose closed-loop control module (20) is equipped with a deep learning pose estimation model. The input of the deep learning pose estimation model is an RGB-D image acquired by an RGB-D camera, and the output is 6D pose data of the target PLC device. The 6D pose data includes position coordinates and attitude angles.

4. The pose closed-loop planning system based on PLC six-sided visual quality inspection according to claim 1, characterized in that, The robot controller (60) has a predefined database built in, which is used to store the specifications of the target PLC device, standard six-sided image templates, detection ROI parameters, defect judgment standard thresholds, joint motion parameters of the dual-arm robot, RGB-D camera calibration parameters and historical quality inspection data.

5. A pose closed-loop planning method based on PLC six-sided visual quality inspection, applied to the pose closed-loop planning system based on PLC six-sided visual quality inspection according to any one of claims 1-4, characterized in that, Specifically, the following steps are included: Step S1: Perform coarse positioning operation for incoming materials: The head RGB-D camera scans the incoming material platform to obtain an RGB-D image containing the target PLC device, and outputs the initial 6D pose data of the target PLC device through a deep learning pose estimation model. Step S2, Arm Selection and Coarse Path Planning: The motion planning module (30) calculates the load margin index of the two arms based on the mass of the target PLC device and the radius of the envelope ball. Based on the calculation results, the robot arm with the larger load margin index value is selected as the first grasping arm. The RRT-Connect algorithm is used to generate a collision-free path and transmits the collision-free path to the robot controller (60) and the end effector. Step S3: Perform precise grasping pose estimation operation: The wrist RGB-D camera scans the target PLC device, acquires the RGB-D image of the target PLC device, and outputs the precise 6D pose data of the target PLC device through the deep learning pose estimation model; Step S4: Perform clamping and vibration suppression operation: The gripper gradually approaches the target PLC device. When the contact force reaches the preset contact threshold, it switches to force holding mode. The vibration sensor detects the vibration data in real time. If the vibration amplitude exceeds the preset vibration threshold, the corresponding robot arm performs active reverse acceleration compensation. Step S5: Perform single-sided detection operation: The first grasping arm moves the target PLC device to the shooting area in front of the head RGB-D camera, calculates the relative pose by combining the current 6D pose data and the execution pose data, and generates a smooth trajectory through a secondary planning algorithm to complete the shooting of the surface to be tested. Step S6: Perform Monte Carlo flipping optimization operation: Using the maximum load margin index value, the fewest number of double arm alternations, and the shortest path as the evaluation function, obtain the optimal flipping sort set of the remaining five untested faces through the Monte Carlo sampling mechanism; Step S7: Perform the alternating flipping operation with both arms: Perform the arm switching operation according to the optimal flipping sort set. The idle robot arm and the first gripping arm complete the handover at the preset arm switching point. The idle robot arm clamps the next test surface of the target PLC device and repeats the operation of step S5 to align the next test surface with the shooting area in front of the head RGB-D camera. Step S8: Perform sorting and unloading operation: Repeat steps S5 to S7 until the six sides of the target PLC device are photographed. Then, the sorting execution module (50) sends the target PLC device to the corresponding good or bad product area based on the quality inspection results after comparing the six-sided image of the target PLC device with the standard six-sided image template, thus completing the visual quality inspection process of the target PLC device.

6. The pose closed-loop planning method based on PLC six-sided visual quality inspection according to claim 5, characterized in that, Step S5 specifically includes: The first gripper moves the target PLC device to the shooting area in front of the head RGB-D camera. The head RGB-D camera scans the target PLC device and acquires RGB-D images. The current 6D pose data of the target PLC device is output through a deep learning pose estimation model. Based on the joint angles of the robotic arm, the execution pose data of the end effector is calculated using positive kinematics, and the relative pose between the current 6D pose data and the execution pose data is calculated. Based on relative pose and positive kinematics, the target pose data required when the normal vector of the surface to be tested coincides with the optical axis of the head RGB-D camera is calculated, so that the target PLC device surface and the head RGB-D camera's requested positioning accuracy are within the preset detection distance tolerance range. The preset detection distance ensures that the captured image meets the defect judgment standard threshold requirements. A smooth trajectory is generated using a quadratic programming algorithm to complete the imaging of the surface to be measured.

7. The pose closed-loop planning method based on PLC six-sided visual quality inspection according to claim 5, characterized in that, The formula for calculating the load margin index in step S2 is as follows: ； in, This is the load margin index value. For robotic arms The maximum torque of the joint, For robotic arms Jacobian transpose matrix, It is a load force vector and satisfies , For the quality of the target PLC device, This is the acceleration due to gravity.

8. The pose closed-loop planning method based on PLC six-sided visual quality inspection according to claim 5, characterized in that, The calculation formula for the evaluation function in step S6 is as follows: ； in, For the evaluation function, This represents the average load margin index value for both arms. The number of times the arms alternate. This is the total path length. This serves as the baseline value for path length. , , The weighting coefficients are and satisfy the following conditions: .

9. The pose closed-loop planning method based on PLC six-sided visual quality inspection according to claim 5, characterized in that, The preset contact threshold in step S4 is 30N, and the preset vibration threshold is 0.1mm.

10. The pose closed-loop planning method based on PLC six-sided visual quality inspection according to claim 6, characterized in that, The preset detection distance tolerance in step S5 is ±3mm.

Images