Product dimension measurement experiment platform based on machine vision
By integrating the ball screw into the robotic arm visual inspection platform in the item detection and sorting system, the object size measurement and automatic sorting are realized, solving the problem of insufficient complexity and accuracy of the existing system, and improving sorting accuracy and efficiency , and simplifies the operation process.
Patent Information
- Application Number
- CN202310527132.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-12
Smart Images

Figure CN118926119A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of object detection and sorting, and in particular to a product size measurement experimental platform based on machine vision. Background Art
[0002] With the development of the times, the accuracy and efficiency of the existing manual visual sorting of items to be inspected are often questioned. In some occasions where manual visual sorting is difficult to meet the requirements, machine visual sorting is often used to replace manual sorting. At the same time, in large-scale repetitive industrial production processes, the use of image processing detection methods can greatly improve the automation and efficiency of sorting. The visual recognition camera is used to identify the size of the items to be inspected, and the communication between the visual processing host computer and the ROS host computer is used to control the sorting operation of the corresponding robot arm and conveyor belt.
[0003] The existing object detection system is complex, has low accuracy and a low degree of integration. Summary of the invention
[0004] The purpose of the present invention is to provide a product size measurement experimental platform based on machine vision and a mechanical arm visual inspection platform with an integrated ball screw to improve sorting accuracy and efficiency.
[0005] The technical solution to achieve the purpose of the present invention is:
[0006] A product size measurement experimental platform based on machine vision, including:
[0007] Conveyor belts for transporting products to be sorted;
[0008] Industrial cameras for detecting products to be sorted;
[0009] The processing unit calibrates the industrial camera and processes the collected images to determine whether the product is qualified;
[0010] The robotic arm grabs and removes unqualified products from the conveyor belt;
[0011] The industrial camera and the robotic arm are respectively located on both sides of the conveyor belt and are respectively connected to the ball screw; the processing unit controls the movement of the lead screw to achieve the translational tracking target of the camera, and transmits the coordinates back to the robotic arm in real time for grasping according to the number of rotations and lead of the ball screw.
[0012] Compared with the prior art, the present invention has the following significant advantages:
[0013] (1) The conveyor belt is used for transmission using a ball screw. The robot arm and the camera detection platform are connected to both sides of the conveyor belt. The three are integrated into a single integrated product. The two conveyor belt transmission devices using ball screws can be flexibly adjusted according to the actual processing conditions, such as loading and unloading positions, material transportation direction, and workstation layout. When the ball screw transmission method is used, there is no need to repeatedly perform hand-eye calibration. After a hand-eye calibration, even if the relative position between the camera and the robot arm changes, the corresponding position information can still be obtained through the motor encoder of the ball screw mechanism for adjustment.
[0014] (2) In order to realize the function of automatically identifying the items to be sorted, the integrated ball screw can also make the overall operation simpler, facilitate the calculation of the relative position of the robot arm and the visual camera, and has high integration and simple operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 Schematic diagram of the overall layout of the product size measurement experimental platform based on machine vision.
[0016] Figure 2 Schematic diagram of the bottom connection of the conveyor belt based on machine vision. DETAILED DESCRIPTION
[0017] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0018] Combination Figure 1 , Figure 2 , a product size measurement experimental platform based on machine vision in this embodiment includes a conveyor belt 1, an industrial camera 2, a robot arm end effector 3, a robot arm movable base 4, a first ball screw motor 5, a second ball screw motor 6, a camera bracket movable base 7, a conveyor belt motor 8, and a visual processing host computer;
[0019] The conveyor belt motor 8 is used to drive the movement of the conveyor belt 1, and the conveyor belt 1 is used to transmit the products to be sorted. The industrial camera 2 is set on the movable base 7 of the camera bracket to collect images of the products. The visual processing host computer determines whether the product is qualified based on the image taken by the industrial camera (which is a prior art and not the focus of this patent), and controls the mechanical arm to grab unqualified products. The mechanical arm is set on the mechanical arm movable base 4, and the end of the mechanical arm is provided with a mechanical arm end effector 3. The mechanical arm end effector 3 can use a clamp or a suction cup to grab and detect unqualified products. Two identical ball screws are provided at the bottom of the conveyor belt 1, which are driven by the first ball screw motor 5 and the second ball screw motor 6 respectively; the two ball screws respectively drive the movement of the mechanical arm movable base 4 and the camera bracket movable base 7, and the movement direction is parallel to the transmission direction of the conveyor belt 1, and the position of the camera detection platform and the grabbing platform can be adjusted according to the product type. The robotic arm and the camera detection platform are connected to both sides of the conveyor belt 1, integrating the three into an integrated single product. The two conveyor belt transmission devices using ball screws can be flexibly adjusted according to actual processing conditions, such as loading and unloading positions, material transportation direction, and workstation layout. When using the ball screw transmission method, there is no need to repeatedly perform hand-eye calibration. After a hand-eye calibration, even if the relative position between the camera and the robotic arm changes, the corresponding position information can still be obtained through the motor encoder of the ball screw mechanism for adjustment.
[0020] The camera module establishes communication with the visual processing host computer through the USB interface, and the robotic arm establishes communication with the ROS host computer through serial communication. The robotic arm includes a solenoid valve and a vacuum suction cup, the vacuum suction cup is installed at the end of the robotic arm, and the vacuum suction cup conduit is fixed with the solenoid valve; the host computer controls the robotic arm to open the solenoid valve through serial communication, and removes defective products through the vacuum suction cup.
[0021] Before the system is operated, the lens used must be calibrated first, and a hand-eye calibration is sufficient. When the relative position between the camera and the robotic arm changes subsequently, the corresponding position information is obtained through the motor encoder of the ball screw mechanism for adjustment.
[0022] The items to be sorted are placed on conveyor belt 1 in advance. When conveyor belt 1 is controlled to transfer the items to be detected to the border of the camera's field of view, conveyor belt 1 stops and waits for the camera to perform machine vision detection steps. The machine vision detection used by the camera adopts the Canny operator and uses a Gaussian filter to smooth the image. The principle is to perform weighted averaging according to the grayscale values of the pixels to be filtered and their neighborhood points according to the parameter rules generated by the Gaussian formula. This can effectively filter out the high-frequency noise superimposed in the ideal image, calculate the gradient intensity and direction of each pixel, and perform non-maximum suppression on the gradient amplitude to make the fuzzy boundary clear. The dual threshold algorithm is used to detect and determine the real and potential edges. The edge detection is finally completed by suppressing isolated weak edges. After obtaining the boundary of the item to be detected, it is compared with the pre-set parameters to determine whether the detection is qualified. When the detection is unqualified, the visual processing host computer sends the coordinates of the unqualified item to the ROS host computer through the local area network, and the ROS host computer receives it through the local area network. After the coordinate information of the unqualified items to be sorted is obtained from the visual processing host computer, the coordinates are sent to the robotic arm through serial communication. The robotic arm picks up the items to be sorted according to the pre-set trajectory planning. The robotic arm consists of two key parts: a solenoid valve and a vacuum suction cup. When the robotic arm moves to the top of the coordinates of the item to be sorted, the robotic arm opens the solenoid valve to make the vacuum suction cup work. After the item to be sorted is sucked to the top of the designated area to be sorted, the main control board controls the solenoid valve to close, the vacuum suction cup stops working, and the item to be sorted falls into the designated area to be sorted. The robotic arm returns to the top of the conveyor belt module through trajectory planning, waiting for the visual processing host computer to send the position coordinates of the next item to be sorted.
[0023] The visual processing host computer and the ROS host computer are connected to each other through a local area network, and the machine vision processing results are used to control the robot arm grasping and qualified product detection. This can effectively sort the items to be sorted. Through machine vision and reflux sorting of items, the recognition accuracy and the speed of information transmission are greatly improved. It has the characteristics of simple structure, hygiene and convenience.
[0024] Under normal circumstances, the Zhang calibration method can be used to obtain the relationship between the camera coordinate system and the image pixel coordinate system, and then the nine-point calibration method can be used to obtain the relationship between the robot base coordinate system and the camera coordinate system.
[0025] First, assume that we have a two-dimensional pixel coordinate point X in the camera coordinate system before transformation. Camera = [x, y, 1], in order to correspond to the subsequent three-dimensional coordinate point, the third digit is padded with 1; the coordinates in the transformed robot arm coordinate system are X Robot =[x′,y′,z′], then according to the matrix transformation rule:
[0026]
[0027] The R matrix represents the rotation matrix, and the M matrix represents the displacement matrix. Since there is no rotation about the Z axis when modeling the coordinate system, their forms are as follows:
[0028]
[0029]
[0030] So the calibration conversion process is:
[0031] x'=r1x+r2y+r3+m1
[0032] y'=r4x+r5y+r6+m2
[0033] z'=m3
[0034] Among them, m3 is the value given when the coordinate system is modeled and is a constant;
[0035] Therefore, let:
[0036] a=r1 b=r2 c=r3+m1
[0037] d=r4 e=r5 f=r6+m2
[0038] The relationship matrix representing the robot working coordinate system to the camera coordinate system can be obtained as follows:
[0039]
[0040] According to the formula: TX Camera =X Robot The calculation process of converting the points in the two-dimensional camera coordinate system into the points in the robotic arm coordinate system is as follows:
[0041]
[0042] z'=m3
[0043] The expanded expression is:
[0044] ax+by+c=x'
[0045] dx+ey+f=y'
[0046] m3=z'(m3 is known)
[0047] At this point, we know that to obtain the transformation matrix from the camera to the robot working coordinate system to the camera coordinate system, six equations are required, that is, the coordinate values of three groups of points. In order to be more accurate in industry, the nine-point calibration method is often used, that is, using 9 groups of points and the least squares method to make the calibration value more accurate.
[0048] If you move the camera or the base of the robot after a calibration, the [a, b, c, d, e, f] parameters measured in the previous calibration will become invalid and need to be retested. If the project requires the camera to track the target in real time, it cannot be completed.
[0049] If the camera translates with the ball screw at this time, assuming that the lead of the ball screw is k and it rotates n times, then the camera translation distance is l, and the formula is as follows
[0050] l=n×k
[0051] The pixel point after the movement does not move relative to the camera, but moves relative to the robot arm, that is, the coordinates of the point in the robot arm coordinate system are [x'-l, y', 1]. According to the formula:
[0052] TX Camera =X Robot
[0053]
[0054] The expanded expression is:
[0055] ax+by+c=x'-n×k
[0056] dx+ey+f=y'
[0057] After the nine-point calibration, the parameters a, b, c, d, e, and f are all known, n can be measured according to the angle sensor of the ball screw, and the lead k is the factory parameter of the ball screw. In this way, the camera can track the target in translation and transmit the coordinates to the robot arm in real time for grabbing.
[0058] The present invention integrates the three into a single integrated product, and includes two conveyor belt transmission devices using ball screws, which can be flexibly adjusted according to actual processing conditions, such as loading and unloading positions, material transportation direction, work station layout, etc.; and when using the ball screw transmission method, there is no need to repeatedly perform hand-eye calibration. After a hand-eye calibration, even if the relative position between the camera and the robotic arm changes, the corresponding position information can still be obtained through the motor encoder of the ball screw mechanism for adjustment.
Claims
1. A product size measurement experimental platform based on machine vision, comprising: Conveyor belts for transporting products to be sorted; Industrial cameras for detecting products to be sorted; The processing unit calibrates the industrial camera and processes the collected images to determine whether the product is qualified; The robotic arm grabs and removes unqualified products from the conveyor belt; It is characterized in that The industrial camera and the robotic arm are respectively located on both sides of the conveyor belt and are respectively connected to the ball screw; the processing unit controls the movement of the lead screw to achieve the translational tracking target of the camera, and transmits the coordinates back to the robotic arm in real time for grasping according to the number of rotations and lead of the ball screw.
2. The product size measurement experimental platform based on machine vision according to claim 1 is characterized in that: The coordinate transformation of the camera and the robotic arm satisfies: ax+by+c=x'-n×k dx+ey+f=y' Among them, a, b, c, d, e, and f are the parameters obtained after nine-point calibration, [x, y] is the two-dimensional pixel coordinate point in the camera coordinate system before transformation, [x′, y′] is the coordinate in the robot coordinate system after transformation, n is the number of rotations of the ball screw, and k is the lead of the ball screw.
3. The product size measurement experimental platform based on machine vision according to claim 1 is characterized in that: The end effector of the mechanical arm is a clamping claw or a vacuum suction cup.
4. The product size measurement experimental platform based on machine vision according to claim 3 is characterized in that: The vacuum suction cup conduit is fixedly installed with an electromagnetic valve, and the processing unit controls the mechanical arm to open the electromagnetic valve through serial communication.
5. The product size measurement experimental platform based on machine vision according to claim 1 is characterized in that: The robotic arm, camera and conveyor belt are integrated together.
6. The product size measurement experimental platform based on machine vision according to claim 1 is characterized in that: The industrial camera is arranged on a movable base of a camera support, the mechanical arm is arranged on a movable base of the mechanical arm, and two ball screws respectively drive the movable base of the mechanical arm and the movable base of the camera support to move, and the moving direction is parallel to the transmission direction of the conveyor belt.
7. The product size measurement experimental platform based on machine vision according to claim 1 is characterized in that: The camera establishes communication with the visual processing host computer through the USB interface, and the robotic arm establishes communication with the ROS host computer through serial port communication.
Citation Information
Cited By
Object visual detection device and detection method
CN121231486A
