Pose closed-loop planning system and method based on PLC six-surface visual quality inspection

Abstract

The invention discloses a pose closed-loop planning system and method based on PLC six-surface visual quality inspection, and relates to the technical field of industrial robot quality inspection, and the system comprises a visual perception module which is used for obtaining an RGB-D image; the pose closed-loop control module is used for receiving the RGB-D image and outputting 6D pose data; the motion planning module is used for generating a double-arm motion track; the turn-over optimization module is used for generating an optimal turn-over sorting set; the sorting execution module is used for completing sorting and discharging actions according to the quality inspection result and the control instruction; and the robot controller is used for data interaction and control instruction issuing. According to the method, manual overturning or multiple sets of jigs are not needed, the grabbing failure rate caused by incoming material position deviation and the like is reduced, automatic and continuous six-face detection of the PLC device is achieved through double-arm alternate operation and Monte Carlo optimized overturning sorting, 6D pose data are output by means of coarse positioning and fine positioning combination of double RGB-D cameras and in combination with a deep learning pose estimation model, and the accuracy of six-face detection of the PLC device is improved. And visual angle distortion, missing detection and false detection are effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial robot quality inspection, and particularly relates to a pose closed-loop planning system and method based on PLC six-surface vision quality inspection. BACKGROUND

[0002] Currently, the PLC (Programmable Logic Controller) appearance quality inspection in the 3C industry mainly adopts the following two schemes: one is manual overturning combined with single-surface fixed cameras, workers overturn the PLC six surfaces in turn, and each surface is photographed once at a fixed station, this method is low in efficiency and has the possibility of missing detection, and significantly increases the labor intensity; the other is to use a single-arm SCARA / DELTA robot combined with a special fixture, install a two-finger gripper on the end effector of the robot, and complete the "grabbing-moving-photographing" operation in cooperation with 1-2 industrial cameras, but the SCARA has only 4 degrees of freedom, so a plurality of sets of overturning fixtures need to be additionally designed to complete the 6-degree-of-freedom detection operation, the changeover time is long, and the single-arm robot cannot realize gravity self-balancing when overturning a large weight PLC (>1.2 kg), resulting in image shaking and out-of-focus conditions, which limits the detection accuracy.

[0003] Based on this, the present application provides a pose closed-loop planning system and method based on PLC six-surface vision quality inspection, which can eliminate the drawbacks of the prior art. SUMMARY

[0004] The purpose of the present application is to provide a pose closed-loop planning system and method based on PLC six-surface vision quality inspection to solve the problems of long quality inspection time, difficulty in six-surface full coverage, and poor size adaptability in the quality inspection process in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] A pose closed-loop planning system based on PLC six-surface vision quality inspection, comprising: a vision perception module, configured to photograph a scene image of a feeding platform, and acquire an RGB-D image containing a target PLC device, the target PLC device being placed on the feeding platform, the vision perception module comprising a head RGB-D camera and a wrist RGB-D camera, the head RGB-D camera being arranged at the center of the head of a dual-arm robot, and the wrist RGB-D camera being arranged on the side of a gripper at the end of an end effector of the dual-arm robot; a pose closed-loop control module, configured to receive the RGB-D image and output 6D pose data of the target PLC device; a motion planning module, configured to generate a motion trajectory of the dual arms of the robot by using a collision-free path planning algorithm according to the pose data output by the pose closed-loop control module and an optimal overturning sorting set generated by an overturning optimization module, and to execute vibration suppression control in combination with vibration sensor data. a flip optimization module, configured to acquire size and quality parameters and current pose data of the target PLC device, combine an evaluation function, and generate an optimal flip sorting set through a Monte Carlo sampling mechanism; a sorting execution module, configured to control the end effector to complete a good or bad product sorting and discharging action of the target PLC device according to the quality inspection result and the control instruction; a robot controller, built in the dual-arm robot, configured to interact with each module and the end effector through a network and issue a control instruction; The pose closed-loop control module, the motion planning module, the flip optimization module and the sorting execution module are integrated in the dual-arm robot and interact with the visual perception module through a network, the sorting execution module is connected with the end effector through a CAN bus, and the end effector receives a sorting instruction and executes a discharging action.

[0007] Further, the two robot arms of the dual-arm robot each include a shoulder rotation joint, an elbow rotation joint and a wrist rotation joint, a robot arm base is symmetrically installed on both sides of the incoming material platform, and the center distance of the robot arm base is adapted to the size of the incoming material platform, so that the dual-arm working space covers the entire detection area, a vibration sensor is installed on the gripper at the end of the end effector, an output end of the vibration sensor is connected with the end effector, and real-time collected vibration data is transmitted to the robot controller through the end effector.

[0008] Further, the pose closed-loop control module is equipped with a deep learning pose estimation model, the deep learning pose estimation model takes an RGB-D image collected by an RGB-D camera as an input, and outputs 6D pose data of the target PLC device, the 6D pose data includes position coordinates and attitude angles.

[0009] Further, the robot controller is built in a predefined database, the database is used to store specification parameters of the target PLC device, a standard six-face image template, detection ROI parameters, flaw determination standard thresholds, dual-arm robot joint motion parameters, RGB-D camera calibration parameters and historical quality inspection data.

[0010] A pose closed-loop planning method based on PLC six-face visual quality inspection, applied to a pose closed-loop planning system based on PLC six-face visual quality inspection, specifically including the following steps: Step S1, performing incoming material coarse positioning operation: the head RGB-D camera scans the incoming material platform to acquire an RGB-D image containing the target PLC device, and outputs initial 6D pose data of the target PLC device through a deep learning pose estimation model; Step S2, performing arm selection and rough path planning operation: the motion planning module calculates the dual-arm load margin index according to the mass and envelope sphere radius size of the target PLC device, and selects the machine arm with a larger load margin index value as the first grabbing arm according to the calculation result, generates a collision-free path using the RRT-Connect algorithm, and transmits the collision-free path to the robot controller and the end effector; Step S3, performing accurate grasping pose estimation operation: the wrist RGB-D camera scans the target PLC device to obtain an RGB-D image of the target PLC device, and outputs accurate 6D pose data of the target PLC device through a deep learning pose estimation model; Step S4, performing clamping and vibration suppression operation: the gripper gradually approaches the target PLC device, switches to force holding mode when the contact force reaches the preset contact threshold, and the vibration sensor detects vibration data in real time. If the vibration amplitude exceeds the preset vibration threshold, the corresponding machine arm is actively accelerated to compensate; Step S5, performing single-face detection operation: the first grabbing arm moves the target PLC device to the shooting area in front of the head RGB-D camera, calculates the relative pose by combining the current 6D pose data and the execution pose data, generates a smooth trajectory through a quadratic programming algorithm, and completes the shooting of the measured surface; Step S6, performing Monte Carlo surface flipping optimization operation: taking the maximum load margin index value, the minimum number of dual-arm alternations, and the shortest path as the evaluation function, the optimal surface flipping sorting set of the remaining five unmeasured surfaces is obtained through the Monte Carlo sampling mechanism; Step S7, performing dual-arm alternating surface flipping operation: according to the optimal surface flipping sorting set, the arm exchange operation is performed, the idle machine arm and the first grabbing arm complete the handover at the preset arm exchange point, the idle machine arm clamps the next measured surface of the target PLC device, and the next measured surface is aligned with the shooting area in front of the head RGB-D camera. The operation of step S5 is repeated; Step S8, performing sorting and discharging operation: steps S5-S7 are repeated until the six-surface shooting of the target PLC device is completed, and the sorting execution module sends the target PLC device to the corresponding good or non-good area according to the quality inspection result after comparing the six-surface images of the target PLC device with the standard six-surface image template, and completes the visual quality inspection process of the target PLC device.

[0011] Further, the step S5 specifically includes: The first grabbing arm moves the target PLC device to the shooting area in front of the head RGB-D camera, the head RGB-D camera scans the target PLC device and obtains an RGB-D image, and outputs the current 6D pose data of the target PLC device through a deep learning pose estimation model; According to the joint angle of the robot arm, the execution pose data of the end effector is calculated based on forward kinematics, and the relative pose of the current 6D pose data and the execution pose data is calculated; According to the relative pose and the forward kinematics, target pose data required when the normal vector of the surface to be measured coincides with the optical axis of the head RGB-D camera is calculated, so that the target PLC device surface and the head RGB-D camera request positioning accuracy is lower than the preset detection distance, and the preset detection distance makes the photographed image meet the flaw judgment standard threshold requirement; A smooth trajectory is generated by a quadratic programming algorithm to complete the shooting of the surface to be measured.

[0012] Further, the calculation formula of the load margin index in step S2 is: ; Among them, is the load margin index value, is the maximum torque of the joint of the robot arm , is the Jacobian transpose matrix of the robot arm , is the load force vector and satisfies , is the mass of the target PLC device, is the acceleration of gravity.

[0013] Further, the calculation formula of the evaluation function in step S6 is: ; Among them, is the evaluation function, is the average load margin index value of the dual-arm, is the number of dual-arm alternations, is the total path length, is the path length reference value, , , is the weight coefficient and satisfies .

[0014] Further, the preset contact threshold in step S4 is 30N, and the preset vibration threshold is 0.1mm.

[0015] Further, the preset detection distance in step S5 is 3mm.

[0016] Compared with the prior art, the beneficial effects of the present application are as follows: 1. This pose closed-loop planning system based on PLC six-sided visual quality inspection does not rely on manual flipping or multiple sets of special flipping fixtures, reducing the grasping failure rate caused by material position deviation and robot absolute positioning error. Through the alternating operation of dual-arm robots and Monte Carlo optimization of flipping sorting, it realizes automated continuous detection of PLC devices on six sides, ensuring that the vibration amplitude meets the preset requirements when the camera is shooting. 2. This invention utilizes a combination of coarse and fine positioning using dual RGB-D cameras on the head and wrist, combined with a deep learning pose estimation model to output 6D pose data, thereby achieving precise alignment between the surface to be measured and the camera optical axis, controlling the detection distance, and effectively avoiding viewpoint distortion, missed detections, and false detections. 3. This invention selects the optimal gripping arm by calculating the load margin index, and with the alternating flipping of the two arms, it can adapt to PLC devices of different length, width, height and weight without changing the fixture. This solves the problem that single-arm robots cannot balance the gravity of flipping heavy PLC devices and reduces the gripping failure rate caused by the deviation of the incoming material size. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the pose closed-loop planning system of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the steps of the pose closed-loop planning method of the present invention.

[0019] Figure labeling: Visual perception module 10, pose closed-loop control module 20, motion planning module 30, flipping optimization module 40, sorting execution module 50, robot controller 60. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0021] Example 1 In this embodiment, as Figure 1 As shown, this invention provides a pose closed-loop planning system based on PLC six-sided visual quality inspection, comprising: The visual perception module 10 is used to shoot the scene image of the incoming material platform, and obtain the RGB-D image containing the target PLC device placed on the incoming material platform. The visual perception module 10 includes a head RGB-D camera and a wrist RGB-D camera. The head RGB-D camera is arranged at the center of the head of the dual-arm robot, and the wrist RGB-D camera is arranged on the side of the gripper at the end of the end effector of the dual-arm robot. The head RGB-D camera is responsible for global coarse positioning, can quickly scan the entire incoming material platform, and obtain the overall position information of the target PLC device. The wrist RGB-D camera moves with the end effector, scans the target PLC device at a close distance, and obtains a higher resolution local image to ensure the calculation accuracy of the precise grasping pose. The two cameras work together to ensure positioning efficiency while taking into account positioning accuracy. The pose closed-loop control module 20 is used to receive the RGB-D image and output the 6D pose data of the target PLC device. In step S1, the RGB-D image of the head RGB-D camera is received, and the initial 6D pose data of the target PLC device is output by the deep learning pose estimation model , and is sent to the motion planning module 30 for rough path planning. In step S3, the RGB-D image of the wrist RGB-D camera is received, and the accurate 6D pose data is output by the same deep learning pose estimation model , and is sent to the motion planning module 30 for precise grasping. In step S4, the current 6D pose data and the joint angle of the robot are received, and the execution pose data and the relative pose are calculated. The motion planning module 30 is used to generate the motion trajectory of the robot dual-arm according to the pose data output by the pose closed-loop control module 20 and the optimal flipping sequence set generated by the flipping optimization module 40, using a collision-free path planning algorithm, and simultaneously performs vibration suppression control combined with the vibration sensor data. The motion planning module 30 generates a collision-free rough path according to the initial 6D pose data , generates a precise grasping path according to the accurate 6D pose data , generates a twice-planned smooth trajectory according to the target pose data , receives the load margin calculation result of the flipping optimization module 40 in step S4, selects the robot arm with greater load margin to perform the first grasping, and receives the real-time vibration data (RMS value) of the vibration sensor. When RMS>0.1mm, a reverse acceleration compensation instruction is generated to control the end effector to suppress vibration, and ensure that the vibration amplitude at the shooting moment is less than 0.05mm. The turning optimization module 40 is configured to obtain the size and mass parameters of the target PLC device and current pose data, combine an evaluation function, and generate an optimal turning sequence set through a Monte Carlo sampling mechanism. In step S2, the size and mass parameters of the target PLC device in the database of the robot controller 60 are read, and the current joint state and load margin data of the dual-arm robot are obtained. In step S6, the optimal turning sequence set is filtered through 5000 times of Monte Carlo sampling and the evaluation function. In step S7, the optimal turning sequence set and the alternating instructions of the dual-arm robot are output to the motion planning module 30 to avoid extreme stretching and collision. The sorting execution module 50 is configured to control the end effector to complete the sorting and discharging operation of the target PLC device according to the quality inspection result and the control instruction. The module sends the clamping / unclamping instruction to the end effector through the CAN bus, controls the target PLC device to be placed in position, receives the in-position feedback signal of the material box photoelectric sensor, and returns the signal to the robot controller 60. The robot controller 60 is built-in in the dual-arm robot, configured to interact with each module and the end effector through a network, issue control instructions, synchronize the working time sequence of each module, and handle abnormal situations.

[0022] In the embodiment, the pose closed-loop control module 20, the motion planning module 30, the turning optimization module 40, and the sorting execution module 50 are integrated in the dual-arm robot and interact with the visual perception module 10 through a network. The sorting execution module 50 is connected with the end effector through the CAN bus, and the end effector receives the sorting instruction and performs the discharging operation.

[0023] In the embodiment, each of the two robot arms of the dual-arm robot includes a shoulder rotation joint, an elbow rotation joint, and a wrist rotation joint. The robot arm base is symmetrically installed on both sides of the incoming material platform, and the center distance of the robot arm base is matched with the size of the incoming material platform, so that the dual-arm working space covers the entire detection area. A vibration sensor is installed on the gripper at the end of the end effector. The output end of the vibration sensor is connected with the end effector, and the end effector transmits the real-time collected vibration data to the robot controller 60. In the embodiment, the pose closed-loop control module 20 is equipped with a deep learning pose estimation model (FFB6D). The input of the deep learning pose estimation model is the RGB-D image collected by the RGB-D camera, and the output is the 6D pose data of the target PLC device, including the position coordinates and the attitude angle. In this embodiment, the robot controller 60 has a predefined database built-in. The database is used to store the specifications of the target PLC device, standard six-sided image templates, detection ROI parameters, defect judgment standard thresholds, joint motion parameters of the dual-arm robot, RGB-D camera calibration parameters, and historical quality inspection data. The specifications of the target PLC device include, but are not limited to, the model, length, width, height, weight, center of gravity coordinates, and shell material properties of several PLC devices. The defect judgment standard thresholds include, but are not limited to, scratch length and color difference value. The joint motion parameters of the dual-arm robot include, but are not limited to, the maximum torque of the dual-arm joints, the Jacobian matrix reference value, repeatability, and workspace boundary coordinates. The RGB-D camera calibration parameters include, but are not limited to, the camera intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (transformation matrix with robot coordinate system). The historical quality inspection data includes, but is not limited to, the detection time, PLC device model, detection result (good / defective product), defect type / location / size, and operator information.

[0024] Example 2 The difference from Example 1 is that, as Figure 2 As shown, the present invention also provides a pose closed-loop planning method based on PLC six-sided visual quality inspection, applied to the above-mentioned pose closed-loop planning system, specifically including the following steps: Step S1: Perform coarse positioning operation for incoming materials: The head RGB-D camera scans the incoming material platform to obtain an RGB-D image containing the target PLC device, and outputs the initial 6D pose data of the target PLC device through a deep learning pose estimation model. Step S2: Perform arm selection and coarse path planning: The motion planning module 30 calculates the load margin index of the two arms based on the mass of the target PLC device and the radius of the envelope ball. Based on the calculation results, the robot arm with the larger load margin index value is selected as the first grasping arm. The RRT-Connect algorithm is used to generate a collision-free path, and the collision-free path is transmitted to the robot controller 60 and the end effector. The RRT-Connect algorithm is an improved version of the RRT (Rapid Expanding Random Tree) algorithm and is a classic algorithm for robot path planning, which is the existing technology. Step S3: Perform precise grasping pose estimation operation: The wrist RGB-D camera scans the target PLC device to obtain the RGB-D image of the target PLC device, and outputs the precise 6D pose data of the target PLC device through the deep learning pose estimation model. Step S4: Perform clamping and vibration suppression operation: The gripper gradually approaches the target PLC device. When the contact force reaches the preset contact threshold, it switches to force holding mode. The vibration sensor detects the vibration data in real time. If the vibration amplitude exceeds the preset vibration threshold, the corresponding robot arm performs active reverse acceleration compensation. The preset contact threshold is 30N and the preset vibration threshold is 0.1mm. Step S5: Perform single-sided inspection operation: The gripping arm moves the target PLC device to the shooting area in front of the head RGB-D camera. The relative pose is calculated by combining the current 6D pose data and the execution pose data. A smooth trajectory is generated by the quadratic planning algorithm to complete the shooting of the surface to be tested. Step S6: Perform Monte Carlo flipping optimization operation: Using the maximum load margin index value, the fewest number of double arm alternations, and the shortest path as the evaluation function, obtain the optimal flipping sort set of the remaining five untested faces through the Monte Carlo sampling mechanism; Step S7: Perform the alternating flipping operation with both arms: Perform the arm switching operation according to the optimal flipping sort set. The idle robot arm waits at the arm switching point until the first gripping arm moves the target PLC device to the preset arm switching point. The idle robot arm clamps the next test surface of the target PLC device. The first gripping arm releases the target PLC device. The idle robot arm and the first gripping arm complete the handover at the preset arm switching point. The idle robot arm repeats the operation of step S5 to align the next test surface with the shooting area in front of the head RGB-D camera. Step S8: Perform sorting and unloading operation: Repeat steps S5 to S7 until the six sides of the target PLC device are photographed. The sorting execution module 50 sends the target PLC device to the corresponding good or bad product area based on the quality inspection result after comparing the six sides of the target PLC device with the standard six sides image template, thus completing the visual quality inspection process of the target PLC device. Specifically, the process of comparing the six-sided image of the target PLC device with the standard six-sided image template includes: The head-mounted RGB-D camera captures six-sided images of the target PLC device and transmits them to the back-end terminal (a computing device that communicates with the robot controller 60). The back-end terminal is equipped with a standard image preprocessing module and a defect detection module. After noise reduction, enhancement, and correction operations by the image preprocessing module, the six-sided images are input to the defect detection module. Detection of defect types includes scratches / dents, missing materials / damage, blurred / misprinted characters, color differences / stains, and assembly defects (such as misaligned interfaces). Defects are detected through edge detection, morphological operations, template matching, area calculation, and color... Color space analysis and geometric measurement are used to detect six-sided images. The detection results are compared with the judgment standard template built into the back-end terminal, and the conclusion of "good product" or "defective product" is output. At the same time, the location, size and type of defects are output. The detection results are fed back to the robot controller 60 in real time via network signal, and then transmitted to the sorting execution module 50. According to the feedback results, the sorting execution module 50 puts the corresponding PLC device into the corresponding bin (good product area / defective product area). The photoelectric sensor of the bin confirms that it is placed in place, and the entire quality inspection process is completed.

[0025] In this embodiment, step S5 specifically includes: The first gripper moves the target PLC device to the shooting area in front of the head RGB-D camera. The head RGB-D camera scans the target PLC device and acquires RGB-D images. The current 6D pose data of the target PLC device is output through a deep learning pose estimation model. Based on the joint angles of the robotic arm, the execution pose data of the end effector is calculated using forward kinematics. The relative pose between the current 6D pose data and the execution pose data is then calculated. Forward kinematics is the process of solving the pose of the end effector (gripper) in the base coordinate system given the joint angles of the robot, and it is the existing technology. Based on relative pose and forward kinematics, the target pose data required for the normal vector of the surface under test to coincide with the optical axis of the head RGB-D camera is calculated. This results in the target PLC device surface being positioned below the required RGB-D camera head positioning accuracy below the preset detection distance. The preset detection distance ensures that the captured image meets the defect judgment standard threshold requirement. The preset detection distance is 3mm. A smooth trajectory is generated using a quadratic programming algorithm to complete the imaging of the surface to be measured; Specifically, this step overcomes the positioning accuracy bottleneck of traditional single-arm / fixture solutions, achieving precise alignment and stable imaging between the target PLC device's test surface and the head-mounted RGB-D camera. This invention acquires current 6D pose data through re-enhancing imaging using the head-mounted RGB-D camera. Combine joint angles to obtain execution pose data Then calculate the relative pose. ( , To execute the inverse matrix of pose data, thereby offsetting the absolute positioning error of the dual-arm robot, the residual deviation of the incoming material position, eliminating the pose offset caused by the size error of the target PLC device, and avoiding the accumulation of errors caused by single-segment pose estimation; Furthermore, this invention also utilizes relative pose. Inverse kinematics of target pose data Force the normal vector of the test surface of the target PLC device to be... Align the optical axis with the head RGB-D camera to ensure distortion-free shooting angle. Strictly control the distance tolerance between the surface to be tested and the head RGB-D camera within ±3mm (±3mm is the visual inspection requirement for object distance) to ensure that defects can be clearly captured and avoid missed or false detections due to alignment deviations. Furthermore, this invention generates a 50ms short-cycle smooth trajectory online using a quadratic programming algorithm (QP), instead of a traditional broken line trajectory. This ensures a continuous trajectory and controllable acceleration, avoiding impacts and vibrations during the movement of the dual-arm robot. Combined with the vibration suppression mechanism in step S4, it further ensures that the amplitude of the end-effector vibration is less than the preset vibration threshold at the moment of shooting. This solves the problem of image defocusing caused by vibration after the large-weight PLC (>1.2kg) is flipped, ensuring image clarity. The 50ms short-cycle trajectory ensures a compact detection cycle, avoiding slowing down the process due to excessive alignment time.

[0026] In this embodiment, the formula for calculating the load margin index in step S2 is as follows: ; in, The load margin index value ( Corresponding to the left arm, (Corresponding to the right arm), reflecting the redundancy capability of the current robotic arm when carrying the current target PLC device. For robotic arms The maximum torque of the joint is determined, and the minimum value among all the maximum torques of the joints is taken to ensure that the joint does not exceed the limit. For robotic arms The Jacobian transpose matrix is ​​used to establish the mapping relationship between the end load force and the joint torque. It is a load force vector and satisfies The first three elements of the vector correspond to the translational force components (X, Y, and Z axes) in a Cartesian coordinate system (the robot's base coordinate system), and the last three elements correspond to the rotational torque components (since the gravity of the target PLC device is a concentrated force parallel to the Z-axis, the rotational torque component is 0). The negative sign indicates that the direction of gravity is opposite to the positive direction of the Z-axis (the Z-axis is vertically upward, which is positive), ensuring that the direction of the force vector is consistent with the actual physical scene. For the quality of the target PLC device, This is the acceleration due to gravity, with a default value of 9.8 m / s². 2 It supports calibration based on usage scenarios. The L2 norm (Euclidean norm) of the vector is used to calculate the composite amplitude of the joint torque; Specifically, the load margin index value The compatibility of the robotic arm with the target PLC device is quantified by the ratio of the difference between the maximum load capacity and the current actual load requirement. When the torque margin of the robotic arm joints approaches 1, the motion stability is high and the vibration risk is low, making it the preferred initial gripping arm. When the load margin is close to zero, the robotic arm approaches its load limit, which can easily lead to problems such as joint over-limit and increased end-effector vibration. Therefore, this arm should not be selected as the main gripping arm. The motion planning module 30 compares the load margin index values ​​of the two arms. Select the robotic arm with the larger value to perform the first grasp; Envelope radius The dimensions (length, width, and height) of the target PLC device are retrieved from the database and derived accordingly. , , , These are the length, width, and height of the target PLC device, respectively. These values ​​are used to assist in correcting the Jacobian transpose matrix. The radius of the envelope sphere... It can reflect the spatial dimensions of the target PLC device, when the radius of the envelope sphere... When the value is large, the change in the lever arm length between the end effector of the robotic arm and the center of gravity of the target PLC device needs to be addressed through… The position parameters in the Jacobian matrix are corrected to ensure that the calculated joint torque is more consistent with the actual load-bearing scenario.

[0027] In this embodiment, the calculation formula for the evaluation function in step S6 is: ; in, For the evaluation function, The smaller the value, the better the flip sequence. This represents the average load margin index value for both arms. The number of times the arms alternate (e.g., left arm → right arm → left arm) is used. (2) The total path length is the sum of the joint space path lengths for all flipping movements. The baseline value for path length (the average path length of all possible sequences, used for normalization). , , The weighting coefficients are satisfied. The weighting coefficients can be dynamically adjusted according to actual production needs; Specifically, the Monte Carlo sampling mechanism generates a large number of possible flipping sequences through 5,000 random samplings. Each sequence must meet the constraints of no collision in the dual-arm working space and accurate alignment of the test surface. The optimal flipping sequence set selected by this mechanism can minimize the number of arm switching operations and movement paths while ensuring load safety, avoid invalid actions, and further compress the quality inspection cycle.

[0028] In summary, this invention constructs an automated, high-precision, and highly adaptable PLC six-sided visual quality inspection solution through dual-camera collaborative localization in the visual perception module 10, deep learning-based accurate estimation in the pose closed-loop control module 20, collision-free path generation in the motion planning module 30, Monte Carlo sequence optimization in the flipping optimization module 40, and collaborative operation of a dual-arm robot. This solution solves the problems of low efficiency and high risk of missed inspection in traditional manual quality inspection, as well as the dependence on fixtures and poor size adaptability of single-arm robots. It can effectively improve quality inspection efficiency and quality, reduce production costs, and has good application prospects.

[0029] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A pose loop-closure planning system based on PLC six-face vision quality inspection, characterized in that, The application relates to a double-arm robot system for PLC device quality inspection and sorting, which comprises the following components: a visual perception module (10) for shooting a scene image of a material receiving platform, acquiring an RGB-D image containing a target PLC device placed on the material receiving platform, the visual perception module (10) comprising a head RGB-D camera arranged at the center of the head of a double-arm robot and a wrist RGB-D camera arranged on the side of a gripper at the end of an end effector of the double-arm robot; a pose closed-loop control module (20) for receiving the RGB-D image and outputting 6D pose data of the target PLC device; a motion planning module (30) for generating a motion trajectory of the double-arm robot by using a collision-free path planning algorithm according to the pose data output by the pose closed-loop control module (20) and an optimal flipping sequence set generated by a flipping optimization module (40), and simultaneously executing vibration suppression control in combination with vibration sensor data; the flipping optimization module (40) for acquiring size and mass parameters and current pose data of the target PLC device, combining an evaluation function, and generating an optimal flipping sequence set through a Monte Carlo sampling mechanism; a sorting execution module (50) for controlling the end effector to complete a good or bad product sorting and discharging action of the target PLC device according to a quality inspection result and a control instruction; a robot controller (60) built in the double-arm robot for data interaction with the modules and the end effector through a network and issuing control instructions; wherein the pose closed-loop control module (20), the motion planning module (30), the flipping optimization module (40) and the sorting execution module (50) are all integrated in the double-arm robot and perform data interaction with the visual perception module (10) through a network, the sorting execution module (50) is connected with the end effector through a CAN bus, and the end effector receives sorting instructions and executes a discharging action.

2. The pose loop-closing planning system based on PLC six-face visual quality inspection according to claim 1, wherein, Both of the two robot arms of the double-arm robot comprise a shoulder rotation joint, an elbow rotation joint and a wrist rotation joint, robot arm bases are symmetrically installed on both sides of the material receiving platform, and the center distance of the robot arm bases is matched with the size of the material receiving platform, so that the double-arm working space covers the whole detection area, a vibration sensor is installed on the gripper at the end of the end effector, the output end of the vibration sensor is connected with the end effector, and the vibration sensor transmits real-time collected vibration data to the robot controller (60) through the end effector. 3.The pose loop-closure planning system based on PLC six-view visual quality inspection of claim 1, wherein, The pose closed-loop control module (20) is provided with a deep learning pose estimation model, the input of the deep learning pose estimation model is an RGB-D image collected by an RGB-D camera, and the output of the deep learning pose estimation model is 6D pose data of the target PLC device, the 6D pose data comprising position coordinates and attitude angles.

4. The pose loop-closing planning system based on PLC six-face visual quality inspection according to claim 1, wherein, The robot controller (60) is built in a pre-defined database, the database is used for storing specification parameters of the target PLC device, a standard six-image template, detection ROI parameters, flaw judgment standard threshold values, double-arm robot joint motion parameters, RGB-D camera calibration parameters and historical quality inspection data. 5.A pose loop-closing planning method based on PLC six-surface vision quality inspection, applied to the pose loop-closing planning system based on PLC six-surface vision quality inspection according to any one of claims 1-4, characterized in that, The application further discloses a double-arm robot system for PLC device quality inspection and sorting, which comprises the following steps: Step S1, performing a rough positioning operation: the head RGB-D camera scans the incoming platform to obtain an RGB-D image containing the target PLC device, and outputs initial 6D pose data of the target PLC device through a deep learning pose estimation model; Step S2, performing arm selection and rough path planning operation: the motion planning module (30) calculates the dual-arm load margin index according to the mass and envelope sphere radius size of the target PLC device, and selects the machine arm with larger load margin index value as the first grabbing arm according to the calculation result, generates a collision-free path using the RRT-Connect algorithm, and transmits the collision-free path to the robot controller (60) and the end effector; Step S3, performing accurate grasping pose estimation operation: the wrist RGB-D camera scans the target PLC device to obtain an RGB-D image of the target PLC device, and outputs accurate 6D pose data of the target PLC device through a deep learning pose estimation model; Step S4, performing clamping and vibration suppression operation: the gripper gradually approaches the target PLC device, and switches to force holding mode when the contact force reaches the preset contact threshold; the vibration sensor detects vibration data in real time, and if the vibration amplitude exceeds the preset vibration threshold, the corresponding machine arm is actively accelerated in the opposite direction to compensate; Step S5, performing single-face detection operation: the first grabbing arm moves the target PLC device to the shooting area in front of the head RGB-D camera, calculates the relative pose based on the current 6D pose data and the execution pose data, generates a smooth trajectory through a quadratic programming algorithm, and completes the shooting of the to-be-detected surface; Step S6, performing Monte Carlo surface flipping optimization operation: taking the maximum load margin index value, the minimum number of dual-arm alternations, and the shortest path as the evaluation function, the optimal surface flipping sorting set of the remaining five unmeasured surfaces is obtained through the Monte Carlo sampling mechanism; Step S7, performing dual-arm alternating surface flipping operation: the optimal surface flipping sorting set is used to perform arm switching operation, the idle machine arm and the first grabbing arm complete handover at the preset arm switching point, the idle machine arm clamps the next to-be-detected surface of the target PLC device, and the next to-be-detected surface is aligned with the shooting area in front of the head RGB-D camera by repeating the step S5 operation; Step S8, performing sorting and discharging operation: repeating steps S5-S7 until the six-surface shooting of the target PLC device is completed, and the sorting and executing module (50) sends the target PLC device to the corresponding good or non-good area according to the quality inspection result obtained by comparing the six-surface images of the target PLC device with the standard six-surface image template, and completes the visual quality inspection process of the target PLC device.

6. The pose loop-closing planning method based on PLC six-face visual quality inspection according to claim 5, characterized in that, The step S5 specifically comprises: The first grabbing arm moves the target PLC device to the shooting area in front of the head RGB-D camera, the head RGB-D camera scans the target PLC device and obtains an RGB-D image, and outputs the current 6D pose data of the target PLC device through a deep learning pose estimation model; According to the joint angle of the machine arm, the execution pose data of the end effector is calculated based on forward kinematics, and the relative pose of the current 6D pose data and the execution pose data is calculated; According to the relative pose and forward kinematics, the target pose data required when the normal vector of the surface to be measured coincides with the optical axis of the head RGB-D camera is calculated, so that the target PLC device surface and the head RGB-D camera request positioning accuracy is lower than the preset detection distance, and the preset detection distance makes the photographed image meet the flaw judgment standard threshold requirement; A smooth trajectory is generated by a quadratic programming algorithm to complete the shooting of the surface to be measured.

7. The pose loop-closing planning method based on PLC six-face visual quality inspection according to claim 5, characterized in that, The calculation formula of the load margin index in the step S2 is: ； wherein, is a load margin indicator value, is a maximum joint torque of the robot arm , is a Jacobian transpose matrix of the robot arm , is a load force vector and satisfies , is a mass of the target PLC device, is a gravitational acceleration.

8. The pose loop-closing planning method based on PLC six-face visual quality inspection according to claim 5, characterized in that, The calculation formula of the evaluation function in the step S6 is: ； wherein, is an evaluation function, is a double-arm average load margin index value, is a double-arm alternation number, is a total path length, is a path length reference value, , , is a weight coefficient and satisfies .

9. The pose loop-closing planning method based on PLC six-face visual quality inspection according to claim 5, wherein, The preset contact threshold in the step S4 is 30N, and the preset vibration threshold is 0.1mm.

10. The pose loop-closing planning method based on PLC six-face visual quality inspection according to claim 6, wherein, The preset detection distance in the step S5 is 3mm.

Images