A product assembly method based on machine vision and SCARA manipulator

Abstract

The invention discloses a product assembly method based on machine vision and SCARA manipulator, and specifically relates to the field of automatic assembly technology. The position of an inner tray and a bottom shell of disorderly incoming materials is identified by vision, and coordinates including coordinate axis values ​​and rotation angles are obtained. The manipulator grabs the inner tray according to the visual coordinates and places it in the bottom shell. There are two ways to calibrate the visual coordinate system and the manipulator coordinate system, and accurate calibration can be achieved under the separation of working areas. Special clamps can be used for inner tray clamping to adapt to different sizes. Large model segmentation technology is used for visual detection, and the outer contours and coordinates of various materials can be detected. The method has the advantages of flexible configuration of vision and manipulator, unified and simple detection method, accurate coordinate system calibration, wide material detection range, etc., which effectively improves the assembly efficiency and quality, and is suitable for various industrial scenes requiring automated assembly.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic assembly, and more specifically, to a product assembly method based on machine vision and a SCARA manipulator. Background Art

[0002] In modern industrial production, automated assembly technology is of vital importance to improving production efficiency, reducing labor costs and improving product quality.

[0003] Traditional assembly methods often rely on manual operation, which has problems such as low efficiency, low precision and difficulty in adapting to complex and changing production tasks. With the continuous advancement of science and technology, vision technology and robotics technology have been widely used in the industrial field, but how to achieve precise coordination between vision and robotics, especially when the visual inspection area and the robotic work area are separated to ensure the accuracy of coordinate conversion, and how to efficiently and uniformly detect and locate different materials, has always been a challenge facing this field. For example, the traditional visual work area and the robotic work area need to partially overlap in order to calibrate the coordinate system, which limits the flexibility of the layout of the work area; when ordinary traditional vision inspects materials, complex image processing steps are required to manually search for stable feature areas based on material color, background color and shape, and the process is cumbersome and has limited adaptability.

[0004] Therefore, a product assembly method based on machine vision and SCARA manipulator is proposed. Summary of the invention

[0005] In order to overcome the above defects of the prior art, the present invention provides a product assembly method based on machine vision and SCARA manipulator to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a product assembly method based on machine vision and SCARA manipulator, comprising the following steps:

[0007] Use the machine vision recognition system to identify the positions of the inner trays and bottom shells of the disorderly incoming materials to obtain the position coordinates of the inner trays and bottom shells;

[0008] According to the position coordinates of the inner tray and the bottom shell given by the machine vision recognition system, the inner tray is grasped by a robot and placed in the bottom shell; wherein the robot working area and the visual inspection area are separated by 0-100m.

[0009] Preferably, before performing visual recognition, a calibration step of the visual coordinate system and the manipulator coordinate system is also included, specifically including:

[0010] Set two marking points on the plate and ;

[0011] Place the thin plate in the visual inspection area and in the visual coordinate system The coordinates of the two marked points detected on the thin plate are , record the wheel encoder value as ;

[0012] Then move the thin plate to the working area of ​​the robot, in the robot coordinate system Use the calibration cone to align the two marking points respectively, and record the coordinates of the robot arm for the two calibrations as follows: , record the wheel encoder value as ;but The coordinates of the two marked points in for:

[0013]

[0014]

[0015]

[0016]

[0017] set up and The relative angle is ,exist The direction offset is , The direction offset is ,Right now Counterclockwise rotation Trailing edge( )Move to get , calculated according to the following formula , :

[0018]

[0019]

[0020]

[0021]

[0022] Thus determining arrive Angle of rotation and the coordinate origin offset ( ).

[0023] Preferably, before performing visual recognition, a calibration step of the visual coordinate system and the manipulator coordinate system is also included, specifically including:

[0024] Install a marking pen at the center of the end of the robot gripper, move the blank sheet to the bottom of the robot, and Use a marker pen to circle two points on the thin plate. Draw a circle and record the value of the wheel encoder as ;

[0025] Then move the plate under the visual camera and record the value of the wheel encoder as , in the visual coordinate system In the visual detection marker coordinates ,but Central coordinates The calculation formula is:

[0026]

[0027]

[0028]

[0029]

[0030] set up and The relative angle is ,exist The direction offset is , The direction offset is ,Right now Counterclockwise rotation Trailing edge( )Move to get , calculated according to the following formula , :

[0031]

[0032]

[0033]

[0034]

[0035] Thus determining arrive Angle of rotation and the coordinate origin offset ( ).

[0036] Furthermore, two marking points are used to obtain the optimal position relationship between the visual coordinate system and the robot coordinate system, including:

[0037] from Pick any two points from the points and calculate a set of offsets to get Group offset( , , ), where i=i=1, 2,..., ;

[0038] Each group ( , , ) and the target value ( ) are: Deviation ,deviation ,deviation ;

[0039] The sum of squared deviations is , for the sum of squared deviations:

[0040] right Find the partial derivative and set it to zero, , we get α= ;

[0041] Take the partial derivative with respect to m and set it to zero, = -2 =0, we get m= ;

[0042] Take the partial derivative with respect to n and set it to zero, = -2 =0, we get n= ;

[0043] When α= ,m= ,n= , the positional relationship between the visual coordinate system and the robot coordinate system is optimal.

[0044] Furthermore, before the machine vision recognition system recognizes the position of the material, it also includes the initial recognition of the material, which specifically includes:

[0045] Take photos of the materials to obtain visual images of the materials;

[0046] Manually mark the center point and direction point in the visual image, and obtain the coordinate values ​​T1(c1, d1) and T2(c2, d2) of the center point and direction point in the visual coordinate system;

[0047] Use the image detection function to extract the material contour in the visual image. The material contour includes the coordinates of each point on the contour in the visual coordinate system, recorded as stdContour, and calculates the area center point (modelX, modelY) of the material contour and the contour angle .

[0048] Furthermore, the machine vision recognition system performs position recognition on the inner support and the bottom shell, specifically including:

[0049] Use the image detection function to extract the contour of the inner tray or the bottom shell, and record it as the contour curContour;

[0050] According to the matching function of the image contour, find the same contour as stdContour in curContour, and get the position of the center of the contour area at this time (curX, curY, curR), from stdContour to curContour:

[0051] The rotation angle is =curR-modelR;

[0052] The lateral translation offX = curX-modelX;

[0053] Vertical translation offY = curY-modelY;

[0054] The center point and direction point at this time are recorded as T3(c3,d3) and T4(c4,d4) respectively;

[0055] Rotation matrix rotation= ;

[0056] The offset matrix produced by the rotation:

[0057] rotationTrans= ;

[0058] Translation matrix translation= ;

[0059] = *rotation +rotationTrans +translation;

[0060] = *rotation +rotationTrans +translation;

[0061] Get the center point coordinates and direction coordinates T3(c3,d3), T4(c4,d4).

[0062] The product assembly method based on machine vision and SCARA manipulator proposed by the present invention has the following beneficial effects:

[0063] 1. The position of the inner tray and bottom shell of the disordered incoming materials is recognized by vision, and the robot is used to grab and place them according to the visual coordinates, realizing the automated assembly process and greatly improving the assembly efficiency. For example, in actual production, compared with traditional manual assembly, the number of assemblies per unit time can be increased several times. At the same time, precise coordinate positioning ensures the accuracy of the assembly of the inner tray and bottom shell, reduces assembly errors, and improves product quality.

[0064] 2. The visual camera working area and the robot working area can be separated. For example, calibration can be achieved even if they are hundreds of meters apart. This breaks the limitations of traditional configurations and makes the layout of the working area more flexible. The detection results of a visual camera can be used by multiple robots, and the robots can be matched to grab. For example, when some robots are repaired or debugged, other robots can continue to work, ensuring the continuity of production. For example, on large production lines, this flexible configuration can effectively respond to production needs during equipment failure or maintenance.

[0065] 3. In visual inspection, the inspection method for any material is unified and simple. For new materials, you only need to specify the center point and direction point when modeling to detect the coordinates. There is no need to perform complex image processing steps like traditional vision, such as image multiplication by averaging kernel, convolution kernel, edge line evaluation, contour extraction, and mutual application. This not only reduces the time cost of modeling and inspection, but also reduces the technical requirements for operators, and improves production efficiency and the versatility of inspection.

[0066] 4. The use of wheel encoders to record the encoder values ​​under the camera and the manipulator enables accurate coordinate system calibration and coordinate detection even when the camera working area and the manipulator working area are far apart. Whether the manipulator moves the calibration plate after visual detection of the marking point or the manipulator visually detects the marking point in the working area, the offset of the manipulator and the visual coordinate system can be accurately calculated, providing a reliable coordinate basis for the precise operation of the manipulator and ensuring high assembly accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0068] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0069] Example 1

[0070] like Figure 1 As shown, the robot automatic assembly method provided in this embodiment 1 is implemented in visual recognition and robot grasping and placing operations:

[0071] Visual identification of the inner support and bottom shell position

[0072] Use a visual system (such as an industrial camera, etc.) installed in a suitable position to take photos of the inner tray and bottom shell of the incoming materials in disorder. Through the image processing algorithm, the position of the inner tray and bottom shell in the visual coordinate system is identified, and their coordinate values ​​are obtained. The coordinate values ​​include the X and Y values ​​on the coordinate axis and the rotation angle R around the Z axis. For example, when inspecting the inner tray and bottom shell of a product, the visual system can accurately identify the center coordinates (X1, Y1) and rotation angle R1 of the inner tray, and the center coordinates (X2, Y2) and rotation angle R2 of the bottom shell.

[0073] Robot grabs the inner tray

[0074] The robot 1, which is connected to the visual system, receives the inner pallet coordinates given by the visual system and moves to the corresponding position to grab the inner pallet. The robot's working area is separated from the visual inspection area by a certain distance, such as 5m. This distance setting makes the layout of the working area more flexible and avoids mutual interference. During the grasping process, the robot's movement path is precisely planned to ensure accurate grasping of the inner pallet.

[0075] The robot places the inner support in the bottom shell

[0076] Robot 2 places the inner tray grabbed by robot 1 in the bottom shell according to the bottom shell coordinates given by the vision system. When placing, precise position adjustment is also performed based on the coordinate information to ensure that the inner tray can be placed in the bottom shell accurately. For example, when placing a certain type of inner tray, robot 2 places the inner tray steadily in the center of the bottom shell according to the bottom shell coordinates provided by the vision system.

[0077] Example 2

[0078] The robot automatic assembly method provided in this embodiment 2 is implemented in the calibration of the visual coordinate system and the robot coordinate system:

[0079] Mark point detection and coordinate sending

[0080] Mark two points mark1 and mark2 on a thin plate, and move the thin plate to the bottom of the visual camera through the assembly line belt. The visual system detects the coordinates of the mark points in the visual coordinate system coorA as p1(x1,y1) and p2(x2,y2), and sends these coordinates to the robot. For example, in the calibration operation on a production line, the visual system detects the coordinates of mark1 as (10,20) and the coordinates of mark2 as (30,40).

[0081] Marker board movement and distance recording

[0082] The robot moves the marking plate from the visual inspection area to the robot working area. During the movement, the wheel encoder records the distance of the calibration plate. Assume that when the marking plate moves from the visual area to the robot working area, the distance value recorded by the wheel encoder is 50mm (just an example).

[0083] Calibration point acquisition and offset calculation

[0084] The robot uses the tip of the calibration cone to align the marking points mar1 and mark2 in the working area, and records the coordinates of the robot as (rootX1, rootY1) and (rootX2, rootY2), and records the encoder value encode2 at this time. According to the above recorded data, the coordinates of the marking points p3 (x3, y3) and p4 (x4, y4) in the robot coordinate system coorB are calculated according to the formula. For example, the coordinates of p3 are calculated to be (60, 20) (assumed data). Then, according to the coordinate values ​​of the marking points in the two coordinate systems, a series of formulas are used to convert the offsets of the robot coordinate system and the visual coordinate system in x, y, and r. For example, the offset in the x direction is calculated to be 5mm, the offset in the y direction is 3mm, and the rotation angle α is 30° (assumed data), thereby determining the accurate relationship between the two coordinate systems, which provides a basis for the subsequent precise operation of the robot.

[0085] Example 3

[0086] The robot automatic assembly method provided in this embodiment 3 is implemented in the calibration of the visual coordinate system and the robot coordinate system. The difference from the embodiment 2 is that the calibration method is different:

[0087] Robot mark point operation and coordinate sending

[0088] Place the calibration plate under the robot's working area. The robot will poke several marking points in the calibration plate in the specified order, such as two points p1(x1,y1) and p2(x2,y2), and send the coordinate values ​​to the vision. At the same time, the marking pen (or stamp) installed in the center of the end of the robot's gripper leaves a mark on the thin plate, recording the encoder value encode1 at this time. Assume that the coordinates of the two points poked by the robot are (15,25) and (35,45), and the encoder value encode1 is 30mm (for example only).

[0089] Calibration plate movement and visual inspection

[0090] Move the calibration plate to the visual detection area, the encoder wheel records the movement distance, and the visual system detects the coordinates of the calibration point. When the calibration plate reaches the visual area, the visual system detects the coordinates of the marking point in the visual coordinate system (visionX1, visionY1) and (visionX2, visionY2). Assume that the detected coordinates are (45, 25) and (65, 45), and the encoder value encode2 is recorded as 80mm (assumed data).

[0091] Offset calculation

[0092] According to the coordinate values ​​of the mark sent by the robot and the coordinate values ​​of the visual inspection, the offsets of the robot and the visual coordinate system in x, y, and r are calculated according to the relevant formulas. The calculation shows that the offset in the x direction is 4mm, the offset in the y direction is 2mm, and the rotation angle α is 25° (assumed data), thus achieving accurate calibration of the two coordinate systems.

[0093] Example 4

[0094] The robot automatic assembly method provided in this embodiment 4 is implemented in the inner support clamping mode:

[0095] Inner support clamping and outer diameter adjustment

[0096] When the inner tray is clamped by a gripper with a telescopic cylinder, the gripper squeezes the outer edge of the inner tray to shrink the outer diameter of the inner tray during the process of grasping the inner tray. For example, when clamping an inner tray with a diameter of 100mm, the gripper squeezes to shrink its outer diameter to 98mm (hypothetical data). Even if the current coordinate position of the inner tray obtained by the manipulator has a small offset, such as 2mm, the inner tray will automatically be centered during the squeezing and shrinking process of the inner tray, ensuring the accuracy of the inner tray placement.

[0097] Insert the inner tray into the bottom shell

[0098] When the inner support is slightly larger than the bottom shell, such as the inner support diameter is 2mm larger than the bottom shell inner diameter, when the inner support is placed in the bottom shell, the outer diameter of the inner support after the clamping jaws are closed is significantly smaller than the bottom shell inner diameter, so it can be placed more easily. After the inner support is placed in the bottom shell, the clamping jaw cylinder is released, and the inner support automatically spreads out to fill the bottom shell, completing the assembly process. This clamping method effectively solves the assembly problem caused by the size difference between the inner support and the bottom shell, and improves assembly efficiency and accuracy.

[0099] Example 5

[0100] The robot automatic assembly method provided in this embodiment 5 is applied in the large model segmentation technology of visual inspection:

[0101] New Material Modeling Operations

[0102] For new materials, such as a new type of plastic box, it is necessary to take a photo under the camera for the first time as a template material. The operator manually marks the center point and direction point in the visual image, for example, the center point coordinates are (50,50) and the direction point coordinates are (60,55). Then the image detection function is used to extract the outer contour of the product at this time, recorded as stdContour, and the center point of the contour area modelX=52, modelY=53 (assumed data) and the contour angle modelR=arctan((60-50) / (55-50))=45° (assumed data) are calculated.

[0103] Coordinate acquisition when running detection

[0104] When the equipment runs the detection function, the visual system uses the image inspection function to extract the outer contour curContour of the product at this time. Through the image contour matching function, find the same contour as stdContour in curContour, and get the position of the center of the contour area at this time (curX, curY, curR), assuming curX=53, curY=54, curR=50°. Calculate the rotation angle α=curR-modelR=5° from stdContour to curContour, the lateral translation offX=curX-modelX=1mm, and the longitudinal translation offY=curY-modelY=1mm. Then calculate the new center point and direction point coordinates based on the rotation matrix, offset matrix, and translation matrix formulas, so as to accurately obtain the coordinate information of the material at different positions, and realize the precise positioning and detection of the material.

[0105] Example 6

[0106] The robot automatic assembly method provided in Example 6 is implemented in the camera coordinate system calibration:

[0107] Get the angle β between the camera and the belt

[0108] Place the thin plate below the camera working area, and the camera takes the first picture. The position of the thin plate is recorded as H0. For example, the coordinates of H0 are (10,10). Start the belt to drive the thin plate to move on the belt. The camera takes the second picture and the position of the thin plate is recorded as H1. Assume that the coordinates of H1 are (30,15). Then, according to the formula, the angle between the camera coordinate system and the belt is calculated as β=arctan((30-10) / (15-10))=63.43° (assumed data). This angle information is of great significance for accurately understanding the relative position relationship between the camera and the belt, and then optimizing the visual inspection and material transmission process. It can be used to predict and adjust the material movement trajectory in subsequent operations.

[0109] Example 7

[0110] The robot automatic assembly method provided in this embodiment 7 is implemented by obtaining the optimal position relationship through multiple marking points:

[0111] Mark point selection and offset calculation

[0112] Assume there are 5 markers (k = 5), pick any two numbers from them, such as marker 1 and marker 2, and bring them into the algorithm to calculate a set of offsets 1,m1,n1.

[0113] Taking other combinations, such as marker point 1 and marker point 3, we can get Group offset data. For example, a group of offsets calculated 1=10°, m1=2mm, n1=3mm (example only).

[0114] Mean calculation and error analysis

[0115] calculate ,m,n mean, , = = , the deviation between each set of offsets and the mean is calculated, such as the deviation ,deviation ,deviation ;

[0116] Minimize the sum of squared deviations: , by respectively , m, n calculate the partial derivatives and set them to zero, find the value that minimizes S. Finally, we get the optimized , m, and n values ​​are used to determine the optimal position relationship between the two coordinate systems, which improves the accuracy of coordinate system calibration and helps to improve the accuracy of robot operation.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A product assembly method based on machine vision and SCARA manipulator, characterized in that: The following steps are involved: Use the machine vision recognition system to identify the positions of the inner trays and bottom shells of the disorderly incoming materials to obtain the position coordinates of the inner trays and bottom shells; According to the position coordinates of the inner tray and the bottom shell given by the machine vision recognition system, the inner tray is grasped by the manipulator and placed in the bottom shell; wherein the manipulator working area and the visual inspection area are separated by 0-100m; in, Before visual recognition, the visual coordinate system and the robot coordinate system are calibrated, including: Two marking points mark1 and mark2 are set on the thin plate; Place the thin plate in the visual inspection area, detect the coordinates of the two marked points on the thin plate in the visual coordinate system coorA, and record the value of the wheel encoder as encode1; Then move the thin plate to the working area of ​​the robot, align the two marking points with the calibration cone in the robot coordinate system coorB, record the coordinates of the robot for the two calibrations as (rootX1,rootY1) and (rootX2,rootY2), and record the value of the wheel encoder as encode2; then the coordinates of the two marking points P3(x3,y3) and P4(x4,y4) in coorB are: x3=rootX1+encode2-encode1 x4=rootX2+encode2-encode1 y3=rootY1 y4=rootY2 Assume that the relative angle between coorA and coorB is α, the offset in the x direction is m, and the offset in the y direction is n, that is, coorA rotates counterclockwise by α and moves along (m, n) to obtain coorB. Calculate α, m, and n according to the following formula: x3=x1cos(α)+y1sin(α)-m y3=y1cos(α)-x1sin(α)-n x4=x2cos(α)+y2sin(α)-m y4=y2cos(α)-x2sin(α)-n Thus, the rotation angle α from coorA to coorB and the coordinate origin offset (m, n) are determined; or, The calibration steps of the visual coordinate system and the robot coordinate system are as follows: Install a marking pen at the center of the end of the robot gripper, move the blank sheet to the bottom of the robot, and use the marking pen to draw circles around the two points P'1 (x'1, y'1) and P'2 (x'2, y'2) on the sheet in the robot coordinate system coorB, and record the value of the wheel encoder as encode1'; Then move the thin plate under the visual camera and record the value of the wheel encoder as encode2'. In the visual coordinate system coorA, the visual detection mark coordinates Then the calculation formula for coordinates P'3 (x'3, y'3) and P'4 (x'4, y'4) in coorB is: y', = visionY1 y'4=visionY2 Assume that the relative angle between coorA and coorB is α', the offset in the x direction is m', and the offset in the y direction is n', that is, coorA rotates α' counterclockwise and moves along (m', n') to obtain coorB. Calculate α', m', and n' according to the following formula: x'3=x'1cos(α')+y'1sin(α')-m' y'3=y'1cos(α')-x'1sin(α')-n' x'4=x'2cos(α')+y'2sin(α')-m' y'4=y'2cos(α')-x'2sin(α')-n' Thus, the rotation angle α' from coorA to coorB and the coordinate origin offset (m', n') are determined.

2. A product assembly method based on machine vision and SCARA manipulator according to claim 1, characterized in that: Use two markers to obtain the optimal position relationship between the visual coordinate system and the robot coordinate system, including: Choose any two points from k points and calculate a set of offsets to get Group offset (α i ,m i ,n i ), where i=1, 2, ..., Each group (α i ,m i ,n i ) and the target value (α, m, n) are: Deviation α' i =α i -α, deviation m' i =m i -m, deviation n' i =n i -n; The sum of squared deviations is For the sum of squared deviations: Take the partial derivative with respect to α and set it to zero, get Take the partial derivative with respect to m and set it to zero, get Take the partial derivative with respect to n and set it to zero, get when The positional relationship between the visual coordinate system and the robot coordinate system is optimal.

3. A product assembly method based on machine vision and SCARA manipulator according to claim 1, characterized in that: Before the machine vision recognition system identifies the location of the material, it also includes the initial recognition of the material, including: Take photos of the materials to obtain visual images of the materials; Manually mark the center point and direction point in the visual image, and obtain the coordinate values ​​T1(c1, d1) and T2(c2, d2) of the center point and direction point in the visual coordinate system; Use the image detection function to extract the material contour in the visual image. The material contour includes the coordinates of each point on the contour in the visual coordinate system, recorded as stdContour, and calculates the area center point (modelX, modelY) of the material contour and the contour angle 4. A product assembly method based on machine vision and SCARA manipulator according to claim 3, characterized in that: The machine vision recognition system recognizes the position of the inner tray and the bottom shell, including: Use the image detection function to extract the contour of the inner tray or the bottom shell, and record it as the contour curContour; According to the matching function of the image contour, find the same contour as stdContour in curContour, and get the position of the center of the contour area at this time (curX, curY, curR), from stdContour to curContour: The rotation angle is α”=curR-modelR; Transverse translation offX = curX - modelX; Vertical translation offY = curY - modelY; The center point and direction point at this time are recorded as T3(c3,d3) and T4(c4,d4) respectively; Rotation Matrix The offset matrix produced by the rotation: Translation Matrix Get the center point coordinates and direction coordinates T3(c3,d3), T4(c4,d4).

Images