A method for realizing vehicle body positioning in an automobile production line

Abstract

The present invention discloses a method for realizing vehicle body positioning in an automobile production line, including: calibrating 4 cameras into the same coordinate system through external instruments; collecting images of 4 parts of the vehicle body at 4 angles of the cameras, and searching for the two-dimensional image coordinates of the inherent feature marking points that can be used for visual positioning of the vehicle body from these images; at the same time, according to the azimuth relationship between the cameras and the marking points, fitting multiple marking points in the 4 images through software modeling, calculating the spatial mathematical relationship with the reference marking points, determining the optimal vehicle body azimuth, calculating the correction vector between the current azimuth and the theoretical azimuth, and sending the correction vector to the robotic arm, and the robotic arm corrects the previously taught robotic arm path to perform operations such as gluing, welding, and spraying. The present invention realizes non-contact three-dimensional azimuth positioning to find out the workpiece offset; complete automation of production control, which can be triggered by receiving external signals; high positioning accuracy and good stability.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile automated production lines, and particularly relates to a method for realizing vehicle body positioning in an automobile production line. Background Art

[0002] With the development of society, various production lines have achieved extremely high automation, especially automobile production lines. In an automobile production line, usually a conveying device is used to transport an automobile body to a specified position, and after the conveying device stops, operations such as gluing, welding, and painting are performed on the automobile body by a robot.

[0003] Due to factors such as internal reasons in the conveying device, machine differences, and installation errors of the conveying device, performing operations on the vehicle body only based on the position reached by the conveying device stopping may cause the stopping position / pose of the conveyed workpiece not to be always constant, and the vehicle body workpiece may undergo parallel rotation in position. If the robot performs operations such as glue sealing, painting, and welding at this time, there will be a large deviation from the original taught robot program. In order to improve the positioning accuracy, high requirements are needed for the conveying device, resulting in increased costs. At the same time, the conveying device also has wear conditions. As the usage frequency increases, the conveying accuracy will also decrease, which will also affect the quality. Summary of the Invention

[0004] The purpose of the present invention is to provide a system and method for realizing vehicle body positioning in an automobile production line to solve the above-mentioned technical problems existing in the prior art.

[0005] To achieve the above purpose, an embodiment of the present invention provides a system for realizing vehicle body positioning in an automobile production line, including: at least 4 cameras, a vision analysis device for calibrating the 4 cameras, and at least 4 robotic arms.

[0006] Among them, four calibrated cameras are used to capture images of four parts of the vehicle body from four angles, and the shooting focus of the images corresponds to at least one preset feature hole on the vehicle body. The coordinates of the preset feature hole are coordinate points in the vehicle body coordinate system. According to the absolute position relationship of the coordinate points of the feature hole in the vehicle body coordinate system, the fixed conversion values (X, Y, Z, Rx, Ry, RZ) between the current vehicle body coordinate system and the "zero vehicle" coordinate system are calculated. The "zero vehicle" is defined as the position where the vehicle without offset is located, and the coordinate system of the "zero vehicle" is obtained through external instruments. The camera captures the preset feature hole, takes the center point of the captured circular image as the feature marking point, the camera searches for the feature marking point and obtains the two-dimensional image coordinates of the feature marking point. At the same time, according to the azimuth relationship between the cameras and the coordinates of the feature marking point in the vehicle body coordinate system, using the specific relationship existing in space, a correction vector is solved through mathematical operations, so as to position the vehicle body through the four cameras. Among them, the specific relationship existing in space refers to the original fixed coordinate points of the feature marking points of the four cameras in the vehicle body coordinate system;

[0007] The visual analysis device is used to calibrate the four cameras. The visual analysis device uses a laser calibrator to calibrate the four cameras, the four robotic arms, and the vehicle body into one coordinate system, calculates the fixed conversion value of each coordinate system relative to the world coordinate system, and realizes the conversion between any two coordinate systems;

[0008] After positioning the vehicle body, the positioning information of the vehicle body is sent to the robotic arm, and the robotic arm compensates the previously taught robotic arm path according to the positioning information, so as to perform automated operations on the vehicle body.

[0009] Furthermore, the robotic arm programs the process trajectory in the vehicle body coordinate system. After the four cameras take pictures of the vehicle body, the deviation value between the current vehicle body and the reference vehicle body is calculated. The robotic arm is corrected by the deviation value to ensure that the movement trajectory of the robotic arm is consistent with the process trajectory of the vehicle body, realizing visual compensation.

[0010] The present invention also provides a vehicle body positioning method for a system for realizing vehicle body positioning in an automobile production line, including:

[0011] Step 1: Calibrate four cameras into the same coordinate system through external instruments;

[0012] Step 2: Collect four images of four parts of the vehicle body from four angles, and search for the two-dimensional image coordinates of the feature marking points on the vehicle body to be visually positioned in the four images;

[0013] Step 3: According to the azimuth relationship between each camera and the feature marking points, through software modeling, fit the multiple feature marking points in the 4 images, and calculate the spatial mathematical relationship between the multiple marking points and the reference marking point;

[0014] Step 4: After determining the optimal vehicle body azimuth mathematically, calculate the correction vector (X, Y, Z, RX, RY, RZ) between the current azimuth and the theoretical azimuth, and send the correction vector to the robotic arm. The robotic arm corrects the previously taught robotic arm path for automated operation.

[0015] Further, step 1 is specifically to calibrate the 4 cameras using a fixed tooling or a laser calibration instrument, and determine the absolute position relationship of the 4 cameras in the world coordinate system through calibration.

[0016] Further, step 2 is specifically to select 4 specific positions on the vehicle body as feature marking points, and the specific positions have clear and definite image information or clear boundaries.

[0017] Further, step 3 generates a vehicle body model through software modeling, and determines the position and posture of the entire vehicle body in the space coordinate system according to the 4 feature marking points on the vehicle body model.

[0018] Further, the specific mathematical implementation method of step 4 is: Let the world coordinate system be Fw, the vehicle body coordinate system be F0, the coordinate systems of the 4 cameras be F1, F2, F3, and F4, and the 4 target areas be A, B, C, and D, and find the position (R, T) of F0 in Fw.

[0019] Further, the mathematical implementation method further includes: When each camera can only see one point within the target area, points a, b, c, and d are seen in the 4 target areas A, B, C, and D respectively. Let the coordinates of point a measured in camera 1 be (u, v), then point a must be on a straight line, and find the straight line equation of point a; Represent the straight line equation in the world coordinate system Fw, and find the equation of point a in FW; Similarly, find the equations of points b and c in FW. There are a total of 9 unknowns and 6 equations.

[0020] Further, the mathematical implementation method further includes: Use the "three-point method" (ICPψ method) to find the coordinate values (R, T), the fourth point is redundant, and find the least squares method; According to the above steps, arbitrarily select three points from points a, b, c, and d, that is, use three cameras to obtain the position and posture of the vehicle body, and use the remaining fourth point as a redundant point for mathematical optimization through the least squares method.

[0021] Further, step 4 is used to identify the characteristic marking points on the vehicle body to determine the current azimuth deviation vector of the vehicle body, that is, the coordinate transformation of six degrees of freedom. The offset vector is transmitted to the robotic arm, so that the robotic arm can accurately correct its motion trajectory to ensure that the trajectory of the robotic arm is spatially consistent with the gap of the vehicle body.

[0022] The method of the present invention has the following advantages:

[0023] (1) Finding the workpiece offset by non-contact three-dimensional azimuth positioning;

[0024] (2) Complete automation of production control, which can be triggered by receiving external signals;

[0025] (3) High positioning accuracy and good stability, with the accuracy controlled within 1 mm and the recognition time within 2 s. Description of the Drawings

[0026] Figure 1 Shows the composition of a vehicle body positioning system using four cameras in an automobile production line.

[0027] Figure 2 Shows the flowchart of the method for realizing vehicle body positioning using four cameras in an automobile production line;

[0028] Figure 3 Shows the coordinate systems of the four cameras and the position of F0 in Fw in the target areas A, B, C, and D.

[0029] In the figure, 1: camera; 2: robotic arm; 3: vehicle body. Detailed Embodiments

[0030] The technical solutions of the present invention will be clearly and completely described below in conjunction with specific implementation embodiments. However, those skilled in the art should understand that the implementation embodiments described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the implementation embodiments of the present invention, all other implementation embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0031] Due to the error of the conveying system, the vehicle body cannot stop at exactly the same azimuth every time, that is, there may be a deviation in the vehicle body coordinate system. At this time, the control system will automatically trigger the vision system to determine the current azimuth deviation of the vehicle body by identifying some inherent characteristic markings on the vehicle body, that is, the coordinate transformation of six degrees of freedom (X, Y, Z, RX, RY, RZ), and transmit the azimuth correction vector to the robot through the PLC, so that the robot can accurately "find" the vehicle body and perform the corresponding gluing application.

[0032] When the vehicle on the production line reaches the specified position through the conveyor line, the present invention uses four cameras to position the vehicle body when the position of the vehicle body is inaccurate. After positioning the vehicle body, the new position is sent to a robot or other actuators for automated operations such as gluing, welding, painting, and assembly.

[0033] An embodiment of the present invention provides a system for realizing vehicle body positioning in an automobile production line, as Figure 1 shown. The system includes at least 4 cameras 1, a visual analysis device, at least 4 robotic arms 2, and a vehicle body 3;

[0034] Among them, 4 calibrated cameras are used to capture images of 4 parts of the vehicle body from 4 angles, and the shooting focus of the images corresponds to at least one preset feature hole of the vehicle body. The coordinates of the preset feature hole are coordinate points in the vehicle body coordinate system. According to the absolute position relationship of the coordinate points of the preset feature hole in the vehicle body coordinate system, the fixed conversion values (X, Y, Z, Rx, Ry, RZ), that is, the position offset, between the current vehicle body coordinate system and the "zero vehicle" coordinate system are calculated. The coordinate system of the "zero vehicle" is obtained through external instruments, that is, the position offset (X, Y, Z, Rx, Ry, RZ) between the position of the current vehicle body and the "zero position vehicle body" coordinate system is obtained. Among them, the "zero vehicle" is defined as the position where the vehicle with no offset is located. The cameras search for inherent feature marker points and obtain the two-dimensional image coordinates of the inherent feature marker points. At the same time, according to the azimuth relationship between the cameras and the coordinates of the inherent feature marker points in the vehicle body coordinate system, using a specific relationship existing in space, that is, the original fixed coordinate points of the four feature marker points in the vehicle body coordinate system, a correction vector is solved through mathematical operations, so as to position the vehicle body through the 4 cameras;

[0035] The visual analysis device is used to calibrate the 4 cameras. Generally, a laser calibrator similar to that of the American company FARO is used for the visual analysis device. Through the laser calibrator, the 4 cameras, the robotic arms, and the vehicle body are all calibrated to a coordinate system, and the fixed conversion values of each coordinate system are calculated to realize the conversion between any two coordinate systems. In practical applications, the vehicle body coordinates are taken as the standard, and the robotic arms are programmed for the process trajectory with the vehicle body coordinate system. After the 4 cameras take pictures, the deviation value between the current vehicle body and the reference vehicle body is calculated. After the robotic arms are corrected using the deviation value, it can ensure that the movement trajectory of the robotic arms is consistent with the process trajectory required by the vehicle body, realizing the compensation function of vision; and

[0036] The robotic arm replaces manual work. After positioning the vehicle body, it sends the positioning information of the vehicle body to the robotic arm. The robotic arm compensates the previously taught path according to the positioning information, so as to perform automated operations such as gluing, welding, and spraying on the vehicle body. Therefore, the robotic arm 2 is the final work execution mechanism, used to complete automated operations such as vehicle body gluing and vehicle body spraying.

[0037] Another embodiment of the present invention provides a method for realizing vehicle body positioning in an automobile production line, as Figure 2 shown, this method includes:

[0038] Step 1: Calibrate 4 cameras to the same coordinate system through external instruments;

[0039] Generally, a fixed tooling or a laser calibration instrument is used to calibrate the 4 cameras. At this time, the absolute position relationship of the 4 cameras in the coordinate system is determined through calibration.

[0040] Step 2: Collect images of 4 parts of the vehicle body from 4 angles, and search for the two-dimensional image coordinates of the inherent feature marking points on the vehicle body in the images;

[0041] Generally, 4 specific positions on the vehicle body are selected, which have clear and definite image information. For example, the feature information of some round holes on the vehicle body, and the positions with clear boundaries of the round holes are used as the inherent feature marking points;

[0042] Step 3: At the same time, according to the azimuth relationship between each camera and the inherent feature marking points, the vehicle body model modeled by software is theoretically consistent with the actual shape of the vehicle. Each camera coordinate, each robot coordinate, and the vehicle body coordinate are unified to a coordinate system through external instruments. Therefore, the conversion relationship (x, y, z, Rx, Ry, Rz) of each coordinate system can be obtained. This is a mathematical model. Fit multiple marking points in the 4 images and calculate the spatial mathematical relationship with the reference marking points;

[0043] The model of the vehicle body is known. The position and attitude of the entire vehicle body in the space coordinate system are determined according to the four feature points;

[0044] Step 4: In this way, the optimal vehicle body orientation is determined mathematically, calculate the correction vector (X, Y, Z, RX, RY, RZ) between the current orientation and the theoretical orientation, and send the correction vector to the robotic arm. The robotic arm corrects the previously taught robotic arm path and performs automated operations such as gluing, welding, and spraying.

[0045] Specific mathematical implementation method:

[0046] As Figure 3As shown, the world coordinate system Fw, the vehicle body coordinate system F0, the coordinate systems F1, F2, F3, and F4 of the four cameras, and the 4 target areas A, B, C, and D. Find the position of F0 in Fw (R,

[0047] If each camera can only see one point within the area, points a, b, c, and d are seen in the 4 target areas A, B, C, and D respectively:

[0048] 1. Assume that the coordinates of point a measured in camera 1 are (u, v). Then point a must be on a straight line, and the equation of the straight line is expressed as:

[0049]

[0050] Among them, c 1 = 1, where fx, fy, cx, and cy are the internal parameters of the camera, and (x 1 , y 1 , z 1 ) are the coordinates of point a in the camera 1 coordinate system.

[0051] 2. Express the equation of the straight line in the world coordinate system Fw. Since

[0052] ( 1 Tw is known)

[0053] In the formula, (x 1w , y 1w , z 1w ) are the coordinates of point a in the world coordinate system, and X 1w = [x 1w , y 1w , z 1w , 1]’. Substituting it in, the straight line is expressed as:

[0054]

[0055] ( 1 Ti is 1 the i-th row of Tw)

[0056] Equation (1) has three unknowns and 2 linear equations of the first degree.

[0057] The value range of each parameter, 1 Tw is known, and its value satisfies the homogeneous transformation matrix specification. a 1 , b 1 , c 1 are also known real quantities.

[0058] 3. Similarly, list the equations of the straight lines where points b and c are located in FW. There are a total of 9 unknowns and 6 equations

[0059]

[0060]

[0061] There are three unknowns and two first-order linear equations in Equation (2). 2 Tw is known and its value satisfies the homogeneous transformation matrix specification, a 1 , b 1 , c 1 are also known real quantities.

[0062] There are three unknowns and two first-order linear equations in Equation (3). 3 Tw is known and its value satisfies the homogeneous transformation matrix specification, a 1 , b 1 , c 1 are also known real quantities.

[0063] 4. The relative relationships among points a, b, and c are known. Supplement three equations,

[0064] (x 1w - x 2w ) 2 +(y 1w - y 2w ) 2 +(z 1w - z 2w ) 2 = d 2 AB

[0065] (x 2w - x 3w ) 2 +(y 2w - y 3w ) 2 +(z 2w - z 3w ) 2 = d 2 BC

[0066] (x 1w - x 3w ) 2 +(y 1w - y 3w ) 2 +(z 1w - z 3w ) 2 = d 2 AC (4)

[0068] where the positive number dAB , d BC , d AC are the lengths of lines AB, BC, and AC respectively. For these 9 unknowns, 6 linear equations of the first degree, and 3 quadratic equations of the second degree, definite solutions can be obtained to find the coordinates of points a, b, and c in FW.

[0069] Intuitively understood from the above discussion, it is to find three points on three known skew lines that satisfy the relative distance constraints. The specific solution is to use the first 6 linear equations to express x 1w , y 1w , z 1w in terms of the expressions of x 2w , y 2w , z 2w , x 3w , y 3w , z 3w , and then substitute them into the last 3 quadratic equations to solve the system of three quadratic equations. This can be implemented using code or by calling matlab functions.

[0070] The solved values may have two sets of solutions. One set of solutions has the vehicle body flipped upside down, which can be excluded, leaving only the other set of solutions that meet the conditions.

[0071] 5. Using the "three-point method" (ICPψ method) can obtain the coordinate values (R, T):

[0072]

[0073] 2. t * = μ q - Rμ p

[0074] 6. The fourth point is redundant, and the least squares method can be used.

[0075] Based on the first three points a, b, and c, that is, the three cameras, the pose information of the vehicle body can be obtained. We use the fourth point d as a redundant point and perform mathematical optimization through the least squares method.

[0076] At the same time, it can prevent the determination of the vehicle body's pose information when one of the cameras fails.

[0077] For example, in the PVC robot gluing project in the painting workshop, the determination of the three-dimensional orientation of the vehicle body by the vision system formed by four cameras: The vehicle body is automatically transported on the production line through a skid or a spreader; when the vehicle body reaches the PVC gluing station, the robot will automatically apply PVC glue to it. Due to the error of the conveying system, the vehicle body cannot stop at exactly the same orientation every time, that is, there may be a deviation in the vehicle body coordinate system. At this time, the control system will automatically trigger the vision system to determine the current orientation deviation of the vehicle body by identifying some inherent feature marking points on the vehicle body, that is, the coordinate transformation of six degrees of freedom, and transmit this offset vector to the robotic arm of the robot, so that the robot can accurately correct its own motion trajectory, ensure that the trajectory of the robotic arm of the robot is spatially consistent with the gap of the vehicle body, and meet the production process requirements.

[0078] Although the present invention has been described in detail above with general descriptions and specific embodiments, on the basis of the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. A system for realizing vehicle body positioning in an automobile production line, comprising: at least 4 cameras, a visual analysis device for calibrating the 4 cameras, and at least 4 robotic arms; wherein, 4 calibrated cameras are used to capture images of 4 parts of the vehicle body from 4 angles, and the shooting focus of the images corresponds to at least one preset feature hole of the vehicle body. The coordinates of the preset feature hole are coordinate points in the vehicle body coordinate system. According to the absolute position relationship of the coordinate points of the preset feature hole in the vehicle body coordinate system, the fixed conversion values (X, Y, Z, Rx, Ry, RZ) between the current vehicle body coordinate system and the "zero vehicle" coordinate system are calculated. The "zero vehicle" is defined as the position where the vehicle without deviation is located, and the coordinate system of the "zero vehicle" is obtained through external instruments; the camera captures the preset feature hole, takes the center point of the captured circular image as the feature marking point, the camera searches for the feature marking point and obtains the two-dimensional image coordinates of the feature marking point. At the same time, according to the azimuth relationship between each camera and the coordinates of the feature marking point in the vehicle body coordinate system, using the specific relationship existing in space, a correction vector is solved through mathematical operations, so as to position the vehicle body through the 4 cameras; wherein, the specific relationship existing in space refers to the original fixed coordinate points of the feature marking points of the 4 cameras in the vehicle body coordinate system; the visual analysis device is used to calibrate the 4 cameras. The visual analysis device uses a laser calibrator to calibrate the 4 cameras, the 4 robotic arms, and the vehicle body to the same coordinate system, calculates the fixed conversion value of each coordinate system relative to the world coordinate system, and realizes the conversion between any two coordinate systems; after positioning the vehicle body, the positioning information of the vehicle body is sent to the robotic arm, and the robotic arm compensates the previously taught robotic arm path according to the positioning information, so as to perform automated operations on the vehicle body.

2. The system for realizing vehicle body positioning in an automobile production line according to claim 1, characterized in that, the robotic arm programs the process trajectory in the vehicle body coordinate system. After the 4 cameras take pictures of the vehicle body, the deviation value between the current vehicle body and the reference vehicle body is calculated; the robotic arm is corrected by the deviation value to ensure that the movement trajectory of the robotic arm is consistent with the process trajectory of the vehicle body, and visual compensation is realized.

3. A vehicle body positioning method using the system for realizing vehicle body positioning in an automobile production line according to any one of claims 1 or 2, comprising: Step 1: Calibrate 4 cameras to the same coordinate system through external instruments; Step 2: Collect 4 images of 4 parts of the vehicle body from 4 angles, and search for the two-dimensional image coordinates of the feature marking points on the vehicle body to be visually positioned in the 4 images; Step 3: According to the azimuth relationship between each camera and the feature marking points, through software modeling, fit multiple feature marking points in the 4 images, and calculate the spatial mathematical relationship between the multiple marking points and the reference marking point; Step 4: After determining the optimal vehicle body orientation mathematically, calculate the correction vector (X, Y, Z, RX, RY, RZ) between the current orientation and the theoretical orientation, and send the correction vector to the robotic arm. The robotic arm corrects the previously taught robotic arm path and performs automated operations.

4. The vehicle body positioning method according to claim 3, wherein, Step 1 is specifically to calibrate the 4 cameras using a fixed tooling or a laser calibration instrument, and determine the absolute position relationship of the 4 cameras in the world coordinate system through calibration.

5. The vehicle body positioning method according to claim 3, wherein, Step 2 is specifically to select 4 specific positions on the vehicle body as feature marking points, and the specific positions have clear and definite image information or clear boundaries.

6. The vehicle body positioning method according to claim 3, wherein, In Step 3, a vehicle body model is generated through software modeling, and the position and orientation of the entire vehicle body in the space coordinate system are determined according to the 4 feature marking points on the vehicle body model.

7. The vehicle body positioning method according to claim 3, wherein, The specific mathematical implementation method of Step 4 is as follows: Let the world coordinate system be Fw, the vehicle body coordinate system be F0, the coordinate systems of the 4 cameras be F1, F2, F3, and F4, and the 4 target areas be A, B, C, and D. Calculate the position (R, T) of F0 in Fw.

8. The vehicle body positioning method according to claim 7, wherein, The mathematical implementation method further includes: When each camera can only see one point within the target area, points a, b, c, and d are seen in the 4 target areas A, B, C, and D respectively. Let the coordinates of point a measured in camera 1 be (u, v), then point a must be on a straight line, and find the straight line equation of point a; Represent the straight line equation in the world coordinate system Fw and find the equation of point a in FW; Similarly, find the equations of points b and c in FW. There are a total of 9 unknowns and 6 equations.

9. The vehicle body positioning method according to claim 8, wherein, The mathematical implementation method further includes: Using the "three-point method" (ICPψ method) to find the coordinate values (R, T), the fourth point is redundant, and find the least squares method; According to the above steps, arbitrarily select three points from points a, b, c, and d, that is, use three cameras to obtain the position and orientation of the vehicle body, and use the remaining fourth point as a redundant point for mathematical optimization through the least squares method.

10. The vehicle body positioning method according to claim 3, wherein, Step 4 is used to identify the feature marking points on the vehicle body to determine the current orientation deviation vector of the vehicle body, that is, the coordinate transformation of 6 degrees of freedom. Transmit the offset vector to the robotic arm, so that the robotic arm can accurately correct its own motion trajectory and ensure that the trajectory of the robotic arm is spatially consistent with the gap of the vehicle body.

Images