A machine vision-based loader boom coaxiality detection device and method

Abstract

This invention discloses a machine vision-based device and method for detecting the coaxiality of a loading arm, belonging to the field of visual inspection technology. It includes: a cabinet; a positioning component detachably mounted on the upper part of the cabinet for gripping and positioning the loading arm; a single-cantilever robot mounted on the cabinet; a CCD camera rotatably mounted on the single-cantilever robot for acquiring data from the loading arm located on the positioning component; and a control component disposed within the cabinet for controlling the single-cantilever robot to move the CCD camera relative to the loading arm on the positioning component and controlling the CCD camera to acquire data from the loading arm. This invention allows for algorithm research on visual inspection devices in a laboratory environment, enabling preliminary algorithm verification, reducing the workload of R&D personnel and engineers, improving R&D efficiency, and providing a theoretical basis for the future construction of large-scale inspection devices in factories.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection technology, specifically relating to a machine vision-based device and method for detecting the coaxiality of a loading boom. Background Technology

[0002] The Shenyang Institute of Automation, Chinese Academy of Sciences, has invented a method suitable for geometric parameter detection of large products. By connecting a structured light measurement head to the end effector of a robot, the robot drives the scanning head to complete high-precision dimensional measurements of the product's shape. By scanning the circumference of an endpoint, point cloud data is obtained. This data is then fitted into an ellipse to obtain the center of the circle. Fitting the center of the circle yields a line segment. Projecting the other endpoint onto this line provides the model's coaxiality error. Figure 1 As shown.

[0003] Jilin University has invented a method for measuring the coaxiality and key position of rectangular spline shafts based on structured light vision. This method primarily utilizes machine vision and camera calibration to obtain camera parameters, the line structured light plane, and the spline shaft axis equation. By moving a laser along the axis, the equation of the line structured light plane and the image of the light stripe on the shaft are obtained at each station. Using the line structured light plane and the spline plane, the coaxiality of the spline shaft can be obtained according to the principle of minimum containment plane. Figure 2 As shown.

[0004] In addition, there are similar technologies, such as: bearing coaxiality detection system, automatic wheel detection device, and dynamic quantitative measurement device for coaxiality and flatness of screw conveyor system, etc.

[0005] Existing technologies include the following verification methods:

[0006] Simulation verification: The control algorithm is simulated on a computer using third-party simulation software (e.g., ADAMS, ROBOTICSYOOLBOX, ROS Gazebo, etc.). The simulation software is run to verify the vision-based loading boom detection device program algorithm in the constructed virtual environment.

[0007] 2. Physical verification: The control algorithm is written into the physical loading arm detection device based on machine vision. Engineers or R&D personnel send instructions to the device through a host computer to verify the vision detection device program algorithm.

[0008] 3. Detection and Comparison Verification: The data obtained by the vision inspection device is processed and compared with the measurement results of the contact robotic arm to verify the program algorithm of the vision inspection loading arm coaxiality device.

[0009] The above verification methods are summarized as follows:

[0010] 1. Simulation verification has the advantages of fast device response, high detection efficiency, and a wide range of adjustable parameters. However, the disadvantage is that simulation software on computers has fewer influencing factors and cannot compare to the real environment, which leads to a decrease in the reliability of the device.

[0011] 2. Physical verification has the advantage of allowing engineers or R&D personnel to verify the device algorithm indoors, taking into account specific issues such as drive, communication, and parameter adjustment. Physical verification in a laboratory environment is more reliable than simulation. The disadvantage is that the camera of the detection device is moved to the designated detection station by a single cantilever robot, which is fixed to a plane, requiring very high parallelism accuracy. Therefore, to obtain a relatively accurate coaxiality error, the precision of the measuring device must be quite high; otherwise, significant errors will occur. Due to the complexity of industrial inspection environments and the inability to effectively control lighting conditions, the image quality acquired by the camera will be affected, thus creating difficulties for subsequent visual algorithm processing.

[0012] 3. Comparative verification: The advantage is that by comparing the coaxiality results obtained with those obtained by a contact-based robotic arm, the coaxiality results obtained by the vision inspection device can be verified, thereby verifying the reliability of the model algorithm. Furthermore, vision inspection is more efficient than contact-based measurements. The disadvantage is that its accuracy is slightly lower than that of contact-based methods. Summary of the Invention

[0013] To solve the above problems, the present invention adopts the following technical solution:

[0014] A machine vision-based coaxiality detection device for a loading boom includes:

[0015] The cabinet is used for fixed support;

[0016] A positioning component, detachably mounted on the upper end of the cabinet, is used for clamping and positioning the loading boom;

[0017] A single-cantilever robot, which is mounted on the cabinet;

[0018] A CCD camera, rotatably mounted on the single cantilever robot, is used to acquire data from the loading arm located on the positioning assembly.

[0019] A control component, disposed within the cabinet, is used to control the single-cantilever robot to move the CCD camera relative to the loading arm located on the positioning component, and to control the CCD camera to collect data from the loading arm.

[0020] Furthermore, the single cantilever robot includes a support frame, an X-axis assembly, a Y-axis assembly, and a Z-axis assembly. The support frame is installed on the upper end of the cabinet. The Y-axis assembly is fixedly disposed on the upper end of the support frame. The X-axis assembly is slidably disposed on the Y-axis assembly along a first direction. The Z-axis assembly is slidably disposed on the X-axis assembly along a second direction.

[0021] Furthermore, the Y-axis assembly includes a Y-axis guide rail fixedly mounted on the support frame, a first slider that can slide relative to the Y-axis guide rail is provided on one side, and the X-axis assembly is fixedly mounted on the first slider; a first drive motor is provided at one end of the Y-axis guide rail, a first lead screw is provided at the output end of the first drive motor, the first lead screw is rotatably connected to the Y-axis guide rail, a first nut is provided inside the first slider, and the first lead screw cooperates with the first nut.

[0022] Furthermore, the X-axis assembly includes an X-axis guide rail fixedly mounted on the first slider via a first mounting bracket and a first drag chain disposed on one side of the X-axis guide rail. A second slider that can slide relative to the X-axis guide rail is disposed on the other side of the X-axis guide rail, and the Z-axis assembly is fixedly mounted on the second slider. A second drive motor is disposed at one end of the X-axis guide rail, and a second lead screw is disposed at the output end of the second drive motor. The second lead screw is rotatably connected to the X-axis guide rail. A second nut is disposed inside the second slider, and the second lead screw cooperates with the second nut.

[0023] Furthermore, the Z-axis assembly includes a Z-axis guide rail fixedly mounted on the second slider via a second mounting bracket and a second drag chain disposed on one side of the Z-axis guide rail. A rotating component is disposed at one end of the Z-axis guide rail, and the CCD camera is mounted on the rotating component.

[0024] Furthermore, the rotating assembly includes a mounting plate, a third drive motor, and a mounting base. The third drive motor is mounted on the mounting plate, and the mounting plate is fixedly mounted on the Z-axis guide rail. The drive end of the third drive motor passes through the mounting plate and is fixedly connected to the mounting base. The CCD camera is mounted on the mounting base.

[0025] Furthermore, the control components include a visual inspection human-machine interface disposed on one side of the upper end of the cabinet, and a motor driver and an industrial control computer disposed inside the cabinet. The industrial control computer includes a controller circuit responsible for the servo motion of the visual inspection boom coaxiality device and a motion module for driving the CCD camera to the designated shooting position.

[0026] The motor driver is electrically connected to the first drive motor, the second drive motor, and the third drive motor respectively, and is used to control the start and stop of the first drive motor, the second drive motor, and the third drive motor;

[0027] The controller circuit includes a main controller, a serial communication bus, and a servo motor controller, a digital-to-analog converter, a machine vision controller, and a camera rotation controller that are electrically connected to the main controller via the serial communication bus.

[0028] The motion module includes a motion board, which is electrically connected to the main controller of the industrial computer and the motor driver.

[0029] Furthermore, it also includes a laser tracker and several target spheres disposed around the positioning component, the laser tracker being used to measure the center of the target spheres.

[0030] A detection method for the coaxiality detection device of a loading boom based on machine vision, as described above, is characterized by comprising the following steps:

[0031] S10. Place several target balls near a certain positioning component, measure the center of the target balls with a laser tracker, and obtain the transformation matrix T0 between the laser tracker and the target ball coordinate system through a preset algorithm.

[0032] S20. A single cantilever robot drives a CCD camera to move together, and the CCD camera captures the target ball and obtains the transformation matrix T1 between the target ball coordinate system and the CCD camera coordinate system at that position.

[0033] S30. By performing matrix operations on T0 and T1, the coordinate transformation matrix T between the CCD camera coordinate system and the laser tracker is obtained, and then all shooting positions of the loading arm detection are calibrated to achieve global calibration.

[0034] S40. Use the CCD camera to take a picture of the loading boom with multiple circular holes, transmit the image to the industrial control computer, process the image through machine vision algorithm, and obtain the center pixel coordinates of the loading boom.

[0035] S50. The CCD camera arrives at the designated shooting position to acquire images. The obtained image pixel coordinates are transformed using the transformation matrix T to obtain the physical coordinates of the center of the circle, and then the center coordinates of multiple holes are obtained. The center coordinates of the multiple holes are transformed to the same coordinate system, and curve fitting is performed using the polyfit function in MATLAB to obtain the reference axis fitted by the center of the multiple holes. The coaxiality error is twice the distance between the center point of the multiple holes and the farthest point of the reference axis.

[0036] Beneficial effects:

[0037] This invention optimizes system equipment and processes, overcoming the shortcomings of existing systems and achieving process optimization. The technical effects are as follows:

[0038] 1. It allows for algorithm research on visual inspection devices in a laboratory environment, enabling preliminary algorithm verification, reducing the workload of R&D personnel and engineers, improving R&D efficiency, and providing a theoretical basis for the future construction of large-scale inspection devices in factories.

[0039] 2. By using a servo motor to control a single cantilever robot to drive a camera for shooting, rapid detection of the workpiece can be achieved. Compared with the method of shooting only at a fixed station, it is more efficient and has no obvious limitations on the shape of the workpiece, thus realizing the versatility of the detection device.

[0040] 3. You can freely switch between automatic detection and manual detection modes.

[0041] 4. Using machine vision for non-contact inspection avoids the drawbacks of contact measurement methods, such as damage to the workpiece surface and slow inspection speed, thus improving the efficiency of identification and inspection operations. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of a coaxiality detection method in the prior art;

[0043] Figure 2 This is a schematic diagram of the coaxiality detection of rectangular splines based on structured light vision in the existing technology.

[0044] Figure 3 This is a schematic diagram of the overall structure of the machine vision-based loading boom coaxiality detection device of the present invention.

[0045] Figure 4 This is a front view of the overall structure of the machine vision-based loading boom coaxiality detection device of the present invention.

[0046] Figure 5 This is a top view of the overall structure of the machine vision-based loading boom coaxiality detection device of the present invention.

[0047] Figure 6 This is a left view of the overall structure of the machine vision-based loading boom coaxiality detection device of the present invention.

[0048] Figure 7 A schematic diagram showing the connection between the main control unit and the motion board;

[0049] Figure 8 A schematic diagram showing the setup of the laser tracker, CCD camera, and target sphere;

[0050] Figure 9 This is a schematic diagram of the controller wiring.

[0051] Figure 10 This is a schematic diagram of coaxiality error measurement.

[0052] The components include: 1. Cabinet; 2. Visual inspection human-machine interface; 3. First cable chain; 4. X-axis guide rail; 5. First servo motor; 6. Second servo motor; 7. Second cable chain; 8. Z-axis guide rail; 9. Y-axis guide rail; 10. Third servo motor; 11. Support frame; 12. Loading boom; 13. Boom tooling fixture; 14. Radiator; 15. CCD camera; 16. Rotating assembly; 17. Industrial computer; 17.1. Main controller; 17.2. Serial communication bus; 17.3. Servo motor controller; 17.4. Digital-to-analog converter controller; 17.5. Machine vision controller; 17.6. Camera rotation controller; 18. Motion board; 19. Target ball; 20. Laser tracker. Detailed Implementation

[0053] Example 1

[0054] A machine vision-based coaxiality detection device for a loading boom includes:

[0055] Cabinet 1, used for fixed support;

[0056] A positioning component is detachably mounted on the upper end of the cabinet 1 for clamping and positioning the loading motorized arm 12.

[0057] A single-cantilever robot is mounted on cabinet 1.

[0058] CCD camera 15, which is rotatably mounted on a single cantilever robot, is used to acquire data from the loading arm 12 located on the positioning assembly.

[0059] The control component is located inside the cabinet 1 and is used to control the single cantilever robot to move the CCD camera 15 relative to the loading arm 12 located on the positioning component, and to control the CCD camera 15 to collect data from the loading arm 12.

[0060] In this embodiment, the single cantilever robot includes a support frame 11, an X-axis assembly, a Y-axis assembly, and a Z-axis assembly. The support frame 11 is mounted on the upper end of the cabinet 1. The Y-axis assembly is fixedly mounted on the upper end of the support frame 11. The X-axis assembly is slidably mounted on the Y-axis assembly along a first direction. The Z-axis assembly is slidably mounted on the X-axis assembly along a second direction.

[0061] The first direction is the X-axis direction in the XYZ coordinate system, and the second direction is the Z-axis direction in the XYZ coordinate system.

[0062] The Y-axis assembly includes a Y-axis guide rail 9 fixedly mounted on a support frame 11. A first slider that can slide relative to the Y-axis guide rail 9 is provided on one side. The X-axis assembly is fixedly mounted on the first slider. A first drive motor is provided at one end of the Y-axis guide rail 9. A first lead screw is provided at the output end of the first drive motor. The first lead screw is rotatably connected to the Y-axis guide rail 9. A first nut is provided inside the first slider. The first lead screw and the first nut are threadedly engaged.

[0063] In this embodiment, one of the first slider and the Y-axis guide rail is provided with a slide rail, and the other of the first slider and the Y-axis guide rail is provided with a slide groove, with the slide rail and the slide groove slidingly engaged.

[0064] The X-axis assembly includes an X-axis guide rail 4 fixedly mounted on a first slider via a first mounting bracket and a first drag chain 3 disposed on one side of the X-axis guide rail 4. A second slider that can slide relative to the X-axis guide rail 4 is disposed on the other side of the X-axis guide rail 4. The Z-axis assembly is fixedly mounted on the second slider. A second drive motor is disposed at one end of the X-axis guide rail 4. A second lead screw is disposed at the output end of the second drive motor. The second lead screw is rotatably connected to the X-axis guide rail 4. A second nut is disposed inside the second slider. The second lead screw and the second nut are threadedly engaged.

[0065] In this embodiment, one of the second slider and the X-axis guide rail is provided with a slide rail, and the other of the second slider and the X-axis guide rail is provided with a slide groove, with the slide rail and the slide groove slidingly engaged.

[0066] The Z-axis assembly includes a Z-axis guide rail 8 fixedly mounted on a second slider via a second mounting bracket and a second drag chain 7 disposed on one side of the Z-axis guide rail 8. A rotating component 16 is disposed at one end of the Z-axis guide rail 8, and a CCD camera 15 is mounted on the rotating component 16.

[0067] In this embodiment, the rotating assembly 16 includes a mounting plate, a third drive motor, and a mounting base. The third drive motor is mounted on the mounting plate, and the mounting plate is fixedly mounted on the Z-axis guide rail 8. The drive end of the third drive motor passes through the mounting plate and is fixedly connected to the mounting base. The CCD camera 15 is mounted on the mounting base.

[0068] In this embodiment, the positioning component includes a boom tooling fixture 13, which is mounted on a positioning hole on the cabinet 1.

[0069] In this embodiment, the control components include a visual inspection human-machine interface 2 disposed on one side of the upper end of the cabinet 1, a motor driver and an industrial control computer 17 disposed inside the cabinet 1. The industrial control computer 17 includes a controller circuit responsible for the servo motion of the visual inspection boom coaxiality device and a motion module for driving the CCD camera 15 to move to the designated shooting position; the industrial control computer 17 acts as a host computer to control the motor driver.

[0070] The motor driver is electrically connected to the first drive motor, the second drive motor, and the third drive motor respectively, and is used to control the start and stop of the first drive motor, the second drive motor, and the third drive motor.

[0071] In this embodiment, the first drive motor, the second drive motor, and the third drive motor are respectively the third servo motor 10, the first servo motor 5, and the second servo motor 6.

[0072] The controller circuitry includes a main controller 17.1, a serial communication bus 17.2, and a servo motor controller 17.3, a digital-to-analog converter controller 17.4, a machine vision controller 17.5, and a camera rotation controller 17.6 that are electrically connected to the main controller via the serial communication bus.

[0073] In this embodiment, the motion module includes a motion board 18, which is electrically connected to the main controller 17.1 of the industrial computer 17 and the motor driver. The servo motor controller 17.3 is electrically connected to the motor driver.

[0074] The machine vision-based loading arm coaxiality detection device provided in this embodiment also includes a target laser tracker 20 and several target balls 19 arranged around the positioning component. The target laser tracker 20 is used to measure the center of the target balls 19.

[0075] In this embodiment, a radiator 14 is provided on one side of the cabinet 1.

[0076] Example 2

[0077] This embodiment describes the detection method of the coaxiality detection device for a loading boom based on machine vision provided in Embodiment 1, including the following steps:

[0078] S10. Place several target balls 19 near a certain positioning component, use a target laser tracker 20 to measure the center of the target balls 19, and obtain the transformation matrix T0 between the target laser tracker 20 and the target ball 19 coordinate system through a preset algorithm.

[0079] S20. The CCD camera 15 is moved together by the single cantilever robot. The CCD camera 15 takes pictures of the target ball 19 and obtains the transformation matrix T1 between the coordinate system of the target ball 19 and the coordinate system of the CCD camera 15 at that position.

[0080] S30. By performing matrix operations on T0 and T1, the coordinate transformation matrix T between the coordinate system of CCD camera 15 and target laser tracker 20 is obtained, and then all shooting positions of the loading arm detection are calibrated to achieve global calibration.

[0081] S40. In this embodiment, the loading arm 12 is provided with four circular holes. A CCD camera 15 is used to take a picture of the loading arm 12 with four circular holes, and the image is transmitted to the industrial control computer 17. The image is processed by a machine vision algorithm to obtain the center pixel coordinates of the loading arm 12.

[0082] The S50 and CCD camera 15 arrive at the designated shooting position to acquire images. The obtained image pixel coordinates are transformed using the transformation matrix T to obtain the physical coordinates of the center of the circle, and then the center coordinates of the four holes are obtained. The center coordinates of the four holes are transformed to the same coordinate system, and curve fitting is performed using the polyfit function in MATLAB to obtain the reference axis fitted by the center of the four holes. The coaxiality error is twice the distance between the center of the four holes and the farthest point of the reference axis.

[0083] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A machine vision-based device for detecting the coaxiality of a loading boom, characterized in that, include: The cabinet is used for fixed support; A positioning component, detachably mounted on the upper end of the cabinet, is used for clamping and positioning the loading boom; A single-cantilever robot, which is mounted on the cabinet; A CCD camera, rotatably mounted on the single cantilever robot, is used to acquire data from the loading arm located on the positioning assembly. A control component, which is disposed in the cabinet, is used to control the single cantilever robot to move the CCD camera relative to the loading arm located on the positioning component, and to control the CCD camera to collect data from the loading arm. The detection method for the coaxiality detection device of the loading boom based on machine vision includes the following steps: S10. Place several target balls near a certain positioning component, measure the center of the target balls with a target laser tracker, and obtain the transformation matrix T0 between the target laser tracker and the target ball coordinate system through a preset algorithm. S20. The CCD camera moves together with the single cantilever robot, the CCD camera takes pictures of the target ball and obtains the transformation matrix T1 between the target ball coordinate system and the CCD camera coordinate system at this position. S30. By performing matrix operations on T0 and T1, the coordinate transformation matrix T between the CCD camera coordinate system and the target laser tracker is obtained, and then all shooting positions of the loading boom are calibrated to achieve global calibration. S40. The loading boom has four circular holes. A CCD camera is used to take a picture of the loading boom with four circular holes, and the image is transmitted to the industrial control computer. The image is processed by machine vision algorithm to obtain the center pixel coordinates of the loading boom. The S50 and CCD cameras arrive at the designated shooting position to acquire images. The obtained image pixel coordinates are transformed using the transformation matrix T to obtain the physical coordinates of the center of the circle, and then the center coordinates of the four holes are obtained. The center coordinates of the four holes are transformed to the same coordinate system, and curve fitting is performed using the polyfit function in MATLAB to obtain the reference axis fitted by the center of the four holes. The coaxiality error is twice the distance between the center of the four holes and the farthest point of the reference axis.

2. The machine vision-based loading boom coaxiality detection device according to claim 1, characterized in that, The single cantilever robot includes a support frame, an X-axis assembly, a Y-axis assembly, and a Z-axis assembly. The support frame is installed on the upper end of the cabinet; the Y-axis assembly is fixedly installed on the upper end of the support frame. The X-axis assembly is slidably disposed on the Y-axis assembly along a first direction; the Z-axis assembly is slidably disposed on the X-axis assembly along a second direction.

3. The machine vision-based loading boom coaxiality detection device according to claim 2, characterized in that, The Y-axis assembly includes a Y-axis guide rail fixedly mounted on the support frame. A first slider that can slide relative to the Y-axis guide rail is provided on one side. The X-axis assembly is fixedly mounted on the first slider. A first drive motor is provided at one end of the Y-axis guide rail. A first lead screw is provided at the output end of the first drive motor. The first lead screw is rotatably connected to the Y-axis guide rail. A first nut is provided inside the first slider. The first lead screw cooperates with the first nut.

4. The machine vision-based loading boom coaxiality detection device according to claim 3, characterized in that, The X-axis assembly includes an X-axis guide rail fixedly mounted on the first slider via a first mounting bracket and a first drag chain disposed on one side of the X-axis guide rail. A second slider that can slide relative to the X-axis guide rail is disposed on the other side of the X-axis guide rail. The Z-axis assembly is fixedly mounted on the second slider. A second drive motor is disposed at one end of the X-axis guide rail. A second lead screw is disposed at the output end of the second drive motor. The second lead screw is rotatably connected to the X-axis guide rail. A second nut is disposed inside the second slider. The second lead screw cooperates with the second nut.

5. The machine vision-based loading boom coaxiality detection device according to claim 4, characterized in that, The Z-axis assembly includes a Z-axis guide rail fixedly mounted on the second slider via a second mounting bracket and a second drag chain disposed on one side of the Z-axis guide rail. A rotating component is disposed at one end of the Z-axis guide rail, and the CCD camera is mounted on the rotating component.

6. The machine vision-based loading boom coaxiality detection device according to claim 5, characterized in that, The rotating assembly includes a mounting plate, a third drive motor, and a mounting base. The third drive motor is mounted on the mounting plate, and the mounting plate is fixedly mounted on the Z-axis guide rail. The drive end of the third drive motor passes through the mounting plate and is fixedly connected to the mounting base. The CCD camera is mounted on the mounting base.

7. The machine vision-based loading boom coaxiality detection device according to claim 6, characterized in that, The control components include a visual inspection human-machine interface set on one side of the upper end of the cabinet, and a motor driver and an industrial control computer set inside the cabinet. The industrial control computer includes a controller circuit responsible for the servo motion of the visual inspection boom coaxiality device and a motion module for driving the CCD camera to the designated shooting position. The motor driver is electrically connected to the first drive motor, the second drive motor, and the third drive motor respectively, and is used to control the start and stop of the first drive motor, the second drive motor, and the third drive motor; The controller circuit includes a main controller, a serial communication bus, and a servo motor controller, a digital-to-analog converter, a machine vision controller, and a camera rotation controller that are electrically connected to the main controller via the serial communication bus. The motion module includes a motion board, which is electrically connected to the main controller of the industrial computer and the motor driver.

Images