A visual detection method based on real-time positioning information

Abstract

The application discloses a kind of visual detection methods based on real-time positioning information, detection area is equipped in visual detection station;Visual detection sensor and visual positioning sensor are equipped around the detection area, the positional relationship between the two is calibrated;The workpiece to be measured is placed in detection area, visual positioning sensor collects feature I;The conversion matrix between theoretical numerical model coordinate system and visual positioning sensor coordinate system is solved using feature I;Visual detection sensor collects the two-dimensional image of the workpiece to be measured, and the pixel coordinates A of the measured point or defect point in the image are solved;Based on the point corresponding to the pixel coordinates A of the origin of camera coordinate system, a space straight line is constructed;The space straight line is rotated and translated, the intersection coordinates of the space straight line after rotation and translation and the theoretical numerical model are calculated, the intersection coordinates are marked as measurement coordinates, and the visual detection is completed.The measurement coordinates obtained by the method have higher accuracy, do not need to install expensive positioning tooling, and greatly reduce the detection time and equipment cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual inspection, in particular to a visual inspection method based on real-time positioning information. BACKGROUND

[0002] At present, visual inspection schemes have been widely applied in the manufacturing industry, especially in the high-precision manufacturing fields such as automobiles, ships, and aerospace. In order to ensure the processing quality of products, it is necessary to strictly require the detection accuracy of the visual inspection scheme. For example, in the field of coating defect detection, the visual inspection + automatic polishing method is used to realize automatic detection and repair of paint defects. Since the automatic polishing system completely relies on the defect position provided by the visual inspection system to execute the polishing program, the accuracy of the defect position detection is very important.

[0003] In the application process of the visual inspection scheme, a plurality of workpieces to be detected are placed in the detection station in turn, and the visual inspection sensor is detected respectively. Since different workpieces to be detected will inevitably have position deviation when falling into position, the relative position between the workpiece to be detected and the visual inspection sensor will change. The detection pose of the visual sensor (such as the detection trajectory of the robot, the installation pose of the visual inspection sensor) is obtained based on the position teaching of the first workpiece to be detected. When the relative position between the workpiece to be detected and the visual inspection sensor changes, it will cause the detection result of the visual inspection sensor to have position deviation, affecting the detection accuracy.

[0004] In order to reduce the accuracy loss caused by the falling position deviation of the workpiece to be detected, the existing technical solutions can be divided into the following two kinds:

[0005] Solution one, install a high-precision positioning tool in the detection station to fix the workpiece to be detected, and rely on the machining accuracy of the positioning tool to ensure that the positions of different workpieces to be detected do not deviate after falling into position. For example, a plurality of positioning pins are arranged on the high-precision positioning tool, and the positioning holes of the workpiece to be detected are used as positioning references. When in use, the pins are inserted into the positioning holes of the workpiece to be detected, and the pin-hole cooperation limits the degrees of freedom of the workpiece to be detected, controls the linear motion of the workpiece to be detected in X, Y, and Z three axial directions, and the rotation motion around X, Y, and Z.

[0006] This solution has the following problems: high cost, especially for positioning tools for large workpieces (such as car bodies, aircraft body skeletons, aircraft body skins, etc.), which consume a huge amount of money.

[0007] Solution two, increase auxiliary parts on the positioning tool, see patent document CN 109443273 B for a method for accurately positioning a workpiece to be detected by using a three-dimensional measurement system. By solving the conversion relationship between the coordinate systems before and after the workpiece to be detected is installed on the clamp (positioning tool) and the theoretical coordinate system, accurate positioning of the workpiece to be detected is realized.

[0008] The scheme has the following problems: each time the workpiece to be measured is positioned, a three-dimensional measurement system (laser tracker, three-coordinate machine, three-dimensional scanner) is needed to measure the auxiliary part, which is complex and time-consuming. SUMMARY

[0009] To solve the above problems, the present application provides a visual detection method based on real-time positioning information, which realizes real-time positioning of the workpiece to be measured by real-time acquisition of features I by a visual positioning sensor, compensates the real-time positioning result into the light intersection process, and makes the measurement coordinate calculation more accurate; the entire process does not need to involve standard instruments (laser tracker, photogrammetry system, etc.) and does not need to install expensive positioning tooling, which greatly reduces the detection time and equipment cost compared with the existing method.

[0010] The technical scheme of the present application is as follows:

[0011] A visual detection method based on real-time positioning information, a detection area is provided in a visual detection station, which is used to place a workpiece to be measured; a visual detection sensor and a visual positioning sensor are provided around the detection area, and the positional relationship between the two is calibrated in advance, denoted as RT 外参 ;

[0012] During detection, the workpiece to be measured is placed in the detection area, and the visual positioning sensor acquires features I on the workpiece to be measured, the features I being inherent feature holes, feature balls or feature points on the workpiece to be measured, and there are multiple features I;

[0013] The coordinates of each feature I in the visual positioning sensor coordinate system are calculated, denoted as measured coordinates;

[0014] The coordinates of features I in the theoretical numerical model coordinate system of the workpiece to be measured are denoted as theoretical coordinates;

[0015] The conversion matrix between the theoretical numerical model coordinate system of the workpiece to be measured and the visual positioning sensor coordinate system is calculated using the theoretical coordinates and the measured coordinates, denoted as RT 移动 ;

[0016] A two-dimensional image of the workpiece to be measured is acquired by using the visual detection sensor, and the pixel coordinates A of the measured points or defect points in the image are calculated;

[0017] Based on the point corresponding to the pixel coordinates A in the camera coordinate system in the visual detection sensor, a space straight line is constructed;

[0018] The space straight line is rotated and translated by using RT 外参 and RT 移动 , the intersection coordinates of the rotated and translated space straight line and the theoretical numerical model of the workpiece to be measured are calculated, and the intersection coordinates are denoted as the measured coordinates of the feature points or defect points, and the visual detection is completed.

[0019] Further, the visual positioning sensor is provided with 2-8, which are evenly fixed around the detection area, for collecting features I at different positions of the workpiece to be measured;

[0020] The external parameters of adjacent visual positioning sensors are calibrated in advance.

[0021] Preferably, the features I include at least 4 and are not coplanar.

[0022] Further, the visual detection sensor is provided with one or more;

[0023] When multiple visual detection sensors are provided, different visual detection sensors are used to collect two-dimensional images at different positions on the workpiece to be measured.

[0024] The positional relationship between different visual detection sensors and the visual positioning sensor is obtained respectively, and during detection, the positional relationship between the visual detection sensor corresponding to the two-dimensional image and the visual positioning sensor is combined with RT 移动 The spatial straight line is rotated and translated.

[0025] Further, the visual detection sensor is installed at the end of the robot, and RT 外参 is calibrated in advance.

[0026] One or more marker circles are fixed at the end of the robot, the robot moves multiple poses, the visual positioning sensor collects images of the marker circles at different poses, and the conversion relationship between the visual positioning sensor coordinate system and the robot base coordinate system is solved according to the coordinates of the marker circles in the image and the coordinates in the robot base coordinate system.

[0027] The conversion relationship between the visual detection sensor coordinate system and the robot base coordinate system is obtained by using hand-eye calibration and robot D-H model.

[0028] The conversion relationship between the visual detection sensor coordinate system and the visual positioning sensor coordinate system is obtained by combining the above two conversion relationships, denoted as RT 外参 .

[0029] Further, the visual detection sensor is fixed around the detection area, and the method for calibrating RT 外参 in advance is as follows:

[0030] A calibration board is placed within the common field of view of the visual detection sensor and the visual positioning sensor, the pose of the calibration board is changed, the visual detection sensor and the visual positioning sensor collect calibration board images each time, and the conversion relationship between the visual detection sensor coordinate system and the visual positioning sensor coordinate system is obtained by using the external parameter calibration method, denoted as RT 外参 .

[0031] Further, the workpiece to be measured is a car body, the visual positioning sensor is provided with 4, and the feature I is an RPS hole.

[0032] The technical scheme has the following advantages:

[0033] ①The visual positioning sensor is fixed around the detection area, and the feature I (feature hole, ball, point) is collected. According to the coordinates of the feature I, the conversion relationship is obtained by using rigid body transformation, the real-time positioning of the workpiece to be measured is realized, the real position relationship between the theoretical model of the workpiece to be measured and the visual detection sensor is obtained, the real-time positioning result (position deviation) is compensated into the light intersection process, and the detection result is more accurate by rotating and translating the space straight line, and the deviation is less than 3 mm, which meets the detection accuracy requirement of the precision machining industry.

[0034] ②In the positioning process of the workpiece to be measured, no standard instrument (tracking instrument, three-coordinate machine, photographic measurement system, etc.) is needed, and the positioning and detection time is greatly reduced. In comparison, the existing method needs to collect the features on the workpiece to be measured by using the standard instrument (laser tracking instrument, three-coordinate machine, photographic measurement system, etc.), establish the conversion relationship between the workpiece coordinate system and the standard instrument coordinate system, and then use the standard instrument to calibrate the robot to establish the conversion relationship between the robot base coordinate system and the standard instrument coordinate system. Through the standard instrument, the conversion relationship between the workpiece coordinate system and the robot base coordinate system is established, and the workpiece is positioned. The method is complicated, needs to install features (such as tracking instrument target ball) at the end of the robot, and needs to change the pose of the robot multiple times. The standard instrument collects feature information, and constructs the conversion relationship, which usually takes more than 6 hours, is time-consuming, and occupies the production rhythm.

[0035] ③In the scheme, the workpiece to be measured only needs to be positioned in the detection area, and no complex and expensive mechanical precision positioning device (positioning tooling / support) needs to be installed in the detection area. In comparison, the mechanical precision positioning device is raised-stable-lowered, and the whole process takes about 6s. The positioning time of the method is ≤2s. The method saves the machining cost and installation time. At the same time, the mechanical precision positioning device will have the problem of precision decline after long-term use, and the cost of later maintenance, maintenance and calibration is high. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a schematic view of the visual positioning sensor and the robot installation layout in the specific embodiment.

[0037] Figure 2 It is a schematic view of the light intersection in the specific embodiment. DETAILED DESCRIPTION

[0038] The technical scheme of the present application will be described in detail in combination with the drawings and the specific embodiment.

[0039] A visual detection method based on real-time positioning information, characterized in that: a detection area is arranged in a visual detection station, and is used for placing a workpiece to be detected; a visual detection sensor and a visual positioning sensor are arranged around the detection area, and a positional relationship between the two is calibrated in advance, denoted as RT 外参 ; that is, a rotation and translation relationship between a visual detection sensor coordinate system and a visual positioning sensor coordinate system;

[0040] During detection, the workpiece to be detected is placed in the detection area, and the visual positioning sensor collects features I on the workpiece to be detected, the features I being inherent feature holes, feature balls or feature points on the workpiece to be detected, and the features I being multiple;

[0041] The coordinates of each feature I in the visual positioning sensor coordinate system are calculated, denoted as measured coordinates;

[0042] The coordinates of the features I in a theoretical numerical model coordinate system of the workpiece to be detected are denoted as theoretical coordinates;

[0043] The conversion matrix between the theoretical numerical model coordinate system of the workpiece to be detected and the visual positioning sensor coordinate system is calculated by using the theoretical coordinates and the measured coordinates, denoted as RT 移动 ;

[0044] A two-dimensional image of the workpiece to be detected is collected by using the visual detection sensor, and pixel coordinates A of a detection point or a defect point in the image are calculated;

[0045] A space straight line is constructed based on a point corresponding to the pixel coordinates A in the camera coordinate system of the visual detection sensor;

[0046] The space straight line is rotated and translated by using RT 外参 and RT 移动 , and an intersection coordinate of the rotated and translated space straight line and the theoretical numerical model of the workpiece to be detected is calculated, and the intersection coordinate is denoted as a measurement coordinate of the feature point or the defect point, so that the visual detection is completed.

[0047] In order to obtain more features I, so that RT 移动 is more accurate; the visual positioning sensor is provided with 2-8, which are uniformly fixed around the detection area and are used for collecting features I at different positions of the workpiece to be detected;

[0048] The external parameters of adjacent visual positioning sensors are calibrated in advance, that is, a rotation and translation relationship between coordinate systems of two adjacent visual positioning sensors.

[0049] RT 移动When detecting, the coordinates of the features I in the coordinate system of each vision positioning sensor are unified to the coordinate system of one of the vision positioning sensors, denoted as measured coordinates, by using the external parameters of the adjacent vision positioning sensors.

[0050] The features I include at least four features and are not coplanar.

[0051] Specifically, the vision detection sensor is provided with one or more vision positioning sensors.

[0052] When the vision detection sensor is provided with multiple vision positioning sensors, different vision positioning sensors are used to acquire two-dimensional images at different positions on the workpiece to be detected.

[0053] The position relationship between the different vision detection sensors and the vision positioning sensors is acquired respectively, and when detecting, the position relationship between the vision detection sensor corresponding to the two-dimensional image and the vision positioning sensor is used in combination with the RT 移动 The spatial straight line is rotated and translated.

[0054] In a specific implementation, the vision detection sensor is installed at the end of the robot or fixed around the detection area.

[0055] When the vision detection sensor is installed at the end of the robot, the RT 外参 is pre-calibrated by the following method:

[0056] One or more marker circles are fixed at the end of the robot, the robot moves multiple poses, the vision positioning sensor acquires images of the marker circles at different poses, and the conversion relationship between the vision positioning sensor coordinate system and the robot base coordinate system is solved according to the coordinates of the marker circles in the images and the coordinates of the marker circles in the robot base coordinate system.

[0057] The conversion relationship between the vision detection sensor coordinate system and the robot base coordinate system is obtained by using hand-eye calibration and a robot D-H model.

[0058] The conversion relationship between the vision detection sensor coordinate system and the vision positioning sensor coordinate system is obtained by combining the above two conversion relationships, denoted as RT 外参 .

[0059] When the vision detection sensor is fixed around the detection area, the RT 外参 is pre-calibrated by the following method:

[0060] A calibration board is placed within the common field of view of the vision detection sensor and the vision positioning sensor, the pose of the calibration board is changed, the vision detection sensor and the vision positioning sensor acquire images of the calibration board each time the pose is changed, and the conversion relationship between the vision detection sensor coordinate system and the vision positioning sensor coordinate system is obtained by using an external parameter calibration method, denoted as RT 外参.

[0061] Wherein, the external parameter calibration method is based on PNP principle to calculate RT 外参 ; such as Zhang Zhengyou calibration method.

[0062] In more detail, when multiple visual detection sensors are fixed around the detection area, the external parameters between adjacent visual detection sensors are calibrated in advance; that is, the rotation and translation relationship between the coordinate systems of two adjacent visual detection sensors;

[0063] Take one visual detection sensor, and calculate the positional relationship between it and the visual positioning sensor; other visual detection sensors are combined with the calibrated external parameters between the visual detection sensors to convert the positional relationship between the other visual detection sensors and the visual positioning sensor. That is, RT 外参 There are multiple, the number of which is consistent with the number of installed visual detection sensors. During detection, the corresponding RT 外参 is found according to the installation position of the visual detection sensor. 外参 , and RT 移动 is combined to rotate and translate the space straight line.

[0064] The following takes the automobile body as an example to illustrate the visual detection process:

[0065] A visual detection method based on real-time positioning information, as shown in Figure 1 , a detection area 1 is provided in the visual detection station, which is used to place the workpiece 5 (automobile body) to be detected;

[0066] A visual detection sensor 3 and a visual positioning sensor 2 are provided around the detection area 1, and the positional relationship (the rotation and translation relationship between the visual detection sensor coordinate system and the visual positioning sensor coordinate system) between the two is calibrated in advance, denoted as RT 外参 ;

[0067] In this embodiment, the visual positioning sensor is provided with four, which are distributed in four directions of the detection area, and the feature I is the RPS hole.

[0068] The visual detection sensor is installed at the end of the robot 4, and the method for pre-calibrating RT 外参 is as follows:

[0069] One or more marker circles are fixed at the end of the robot, the robot moves multiple poses, the visual positioning sensor collects images of the marker circles in different poses, and the conversion relationship between the visual positioning sensor coordinate system and the robot base coordinate system is calculated according to the coordinates of the marker circles in the image and the coordinates in the robot base coordinate system.

[0070] The conversion relationship between the vision detection sensor coordinate system and the robot base coordinate is obtained by using the hand-eye calibration and the robot D-H model.

[0071] The conversion relationship between the vision detection sensor coordinate system and the vision positioning sensor coordinate system is obtained by combining the above two conversion relationships, and is denoted as RT 外参 .

[0072] During detection, the workpiece to be detected is placed in the detection area, and the vision positioning sensor collects the features I on the workpiece to be detected. The features I are inherent feature holes, feature balls or feature points on the workpiece to be detected, and there are multiple features I.

[0073] The coordinates of each feature I in the vision positioning sensor coordinate system are calculated and denoted as the measured coordinates.

[0074] The coordinates of the features I in the theoretical numerical model coordinate system of the workpiece to be detected are denoted as the theoretical coordinates.

[0075] The conversion matrix between the theoretical numerical model coordinate system of the workpiece to be detected and the vision positioning sensor coordinate system is calculated by using the theoretical coordinates and the measured coordinates, and is denoted as RT 移动 .

[0076] A two-dimensional image of the workpiece to be detected is collected by using the vision detection sensor, and the pixel coordinates A of the measured points or defect points in the image are calculated.

[0077] As shown in Figure 2 , a space straight line is constructed based on the point corresponding to the pixel coordinates A in the camera coordinate system of the vision detection sensor.

[0078] The space straight line is rotated and translated by using RT 外参 and RT 移动 , the intersection coordinates of the rotated and translated space straight line and the theoretical numerical model of the workpiece to be detected are calculated, and the intersection coordinates are denoted as the measured coordinates of the feature points or defect points, and the vision detection is completed.

[0079] Since the size of the workpiece to be detected is large, the measurement poses of the vision detection sensor are multiple, and the robot needs to adjust different poses to drive the vision detection sensor to move around the workpiece to be detected, and collect images of the automobile body at different measurement poses.

[0080] At this time, the conversion relationship between the vision detection sensor coordinate system and the robot base coordinate at different measurement poses can be obtained by combining the robot D-H model.

[0081] In combination with the known conversion relationship between the vision positioning sensor coordinate system and the robot base coordinate system, the conversion relationship between the vision detection sensor coordinate system and the vision positioning sensor coordinate system at different measurement poses can be obtained. That is, RT外参 There are multiple, the number is consistent with the number of measured poses. When detecting, according to the measured pose of the visual detection sensor, the corresponding RT 外参 is searched 外参 , and the RT 移动 obtained is combined again 移动 , that is, RT 外参 is rotated and translated to the space straight line, that is, RT

[0082] The scheme realizes real-time positioning of the workpiece to be measured by acquiring the feature I through the visual positioning sensor and obtaining the conversion relationship by using the rigid body transformation according to the coordinates of the feature I, and obtains the real position relationship between the theoretical numerical model of the workpiece to be measured and the visual detection sensor, and the detection result is high in accuracy. The workpiece to be measured only needs to be positioned to the detection area, and a complex and expensive mechanical precision positioning device does not need to be installed in the detection area, thereby saving cost and time.

[0083] The foregoing description of specific exemplary embodiments presenting the application is for the purpose of illustration and description. The foregoing description is not intended to be exhaustive or to limit the application to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teachings. The exemplary embodiments are chosen and described so that others skilled in the art can best understand the principles of the application and its practical application. The scope of the application is intended to be defined by the claims appended hereto and their equivalents.

Claims

1. A visual detection method based on real-time positioning information, characterized in that: A detection area is arranged in the visual detection station, and is used for placing a workpiece to be detected; a visual detection sensor and a visual positioning sensor are arranged around the detection area, and a position relationship between the two is calibrated in advance and is denoted as RT 外参 . During testing, the workpiece to be tested is placed in the testing area, and the visual positioning sensor collects feature I on the workpiece to be tested. Feature I is an inherent feature hole, feature ball, or feature point on the workpiece to be tested, and there are multiple features I. Calculate the coordinates of each feature I in the visual positioning sensor coordinate system, and record them as the measured coordinates; Let the coordinates of feature I in the theoretical numerical model coordinate system of the workpiece be called the theoretical coordinates; The conversion matrix between the theoretical coordinate system of the workpiece to be measured and the coordinate system of the visual positioning sensor is calculated using the theoretical coordinates and the measured coordinates, and is denoted as RT 移动 ; A two-dimensional image of the workpiece to be tested is acquired using a visual inspection sensor, and the pixel coordinates A of the points to be tested or the defect points in the image are calculated. Based on the origin of the camera coordinate system in the visual inspection sensor and the point in the camera coordinate system corresponding to pixel coordinate A, a spatial straight line is constructed; Using RT 外参 and RT 移动 The space straight line is rotated and translated, the intersection coordinates of the space straight line after rotation and translation and the theoretical numerical model of the workpiece to be measured are calculated, the intersection coordinates are marked as the measurement coordinates of the feature points or defect points, and the visual detection is completed.

2. The visual detection method based on real-time positioning information as described in claim 1, characterized in that, The visual positioning sensor is provided in 2 to 8 units, which are evenly distributed and fixed around the detection area to collect feature I at different positions of the workpiece to be tested. Pre-calibrate the extrinsic parameters of adjacent visual positioning sensors.

3. The visual detection method based on real-time positioning information as described in claim 1 or 2, characterized in that, Feature I includes at least four features, and they are not coplanar.

4. The visual detection method based on real-time positioning information as described in claim 1, characterized in that, The visual inspection sensor is provided with one or more; When multiple sensors are provided, different visual inspection sensors are used to acquire two-dimensional images of different locations on the workpiece to be tested.

5. The visual detection method based on real-time positioning information as described in claim 1 or 4, characterized in that, The visual inspection sensor is mounted at the end of the robot, the RT is calibrated in advance 外参 The method is as follows: One or more marker circles are fixed at the end of the robot. The robot moves in multiple poses. The visual positioning sensor acquires images of the marker circles in different poses. Based on the coordinates of the marker circles in the images and their coordinates in the robot's base coordinates, the transformation relationship between the visual positioning sensor coordinate system and the robot's base coordinate system is calculated. Using hand-eye calibration and the robot DH model, the transformation relationship between the visual inspection sensor coordinate system and the robot base coordinate system is obtained; Combining the two transformation relationships mentioned above, the transformation relationship between the visual inspection sensor coordinate system and the visual positioning sensor coordinate system is derived, denoted as RT. 外参 .

6. The visual detection method based on real-time positioning information as described in claim 1 or 4, characterized in that, The visual inspection sensor is fixed around the inspection area and pre-calibrated RT. 外参 The method is as follows: Within the common field of view of the visual inspection sensor and the visual positioning sensor, a calibration board is placed. The pose of the calibration board is varied, and both the visual inspection sensor and the visual positioning sensor acquire images of the calibration board at each change. The transformation relationship between the coordinate systems of the visual inspection sensor and the visual positioning sensor is obtained using an extrinsic parameter calibration method, denoted as RT. 外参 .

7. The visual detection method based on real-time positioning information as described in claim 1, characterized in that, The workpiece to be tested is a car body, and the visual positioning sensor has 4 sensors, with feature I being an RPS hole.

Images