Detection method for surface related to object under test

Abstract

An embodiment of the present invention discloses a method for detecting the relevant surface of a measured object. First, the vehicle body is transported to the assembly line flat plate, and the vehicle body is successively transported to the flat plate assembly line with a limiting device through a crane. When the vehicle body runs to the visual inspection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the stop signal is acquired by the vision system, and then the user operation selection is carried out. The vision system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are the distortion calibration process, the N-point calibration process, the magnification acquisition process, and the template production process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template production, the position of the laser sensor is adjusted to approach the position during template production, so as to obtain accurate laser sensor values.

Description

Technical Field

[0001] The present invention belongs to the technical field of the production of autonomous vehicles, and particularly relates to a method for detecting a relevant surface of an object to be measured. Background Art

[0002] An autonomous vehicle is an intelligent vehicle that realizes driverless operation through a computer system. An autonomous vehicle relies on the collaborative cooperation of artificial intelligence, visual computing, radar, monitoring devices, and the global positioning system, enabling the computer to automatically and safely operate a motor vehicle without active human operation.

[0003] With the rise of autonomous vehicle technology, more and more enterprises have invested in the research and development of autonomous vehicles. Currently, technologies such as artificial intelligence, visual monitoring and computing, and radar have become mature. By effectively integrating these technologies, an autonomous driving system can be realized. Installing the autonomous driving system into a manually driven vehicle for vehicle modification can convert a manually driven vehicle into an autonomous vehicle. After long-term testing, the autonomous driving system can achieve safe and reliable autonomous driving on the learned lanes.

[0004] The autonomous driving system includes many hardware components such as radar and vision systems. Currently, after an automobile modification factory manually drills holes at designated installation positions on the vehicle body of a manually driven vehicle, the above-mentioned hardware devices are then installed. During the installation process, if manual drilling is carried out, there are various problems such as low precision, low efficiency, and poor repeatability. Summary of the Invention

[0005] In view of the above technical problems, the present invention provides a method for detecting a relevant surface of an object to be measured, which is used to detect each surface of the vehicle body, so as to achieve high-precision positioning and drilling of the hole positions on the vehicle body of the modified autonomous vehicle.

[0006] To solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A detection method for the relevant surface of the object to be measured. First, the vehicle body is transported to the assembly line flat plate. The vehicle body is successively transported to the flat plate assembly line with a limiting device by a crane. When the vehicle body runs to the visual detection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the stop signal is acquired by the visual system. Then, user operations can be selected. The visual system performs single-station operations or full-station operations according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are the distortion calibration process, the N-point calibration process, the magnification acquisition process, and the template making process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template making, the position of the laser sensor is adjusted to approach the position during template making, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding detection components at each station to acquire and process the images and the measured values of the laser sensor. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top detection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera surface. Then, calculate the deflection angle of the vehicle body on the camera surface by obtaining the features of the image through the camera and the image features during template making. The corresponding deflection angles are used for processing during the distortion correction and image coordinate conversion at each station; through the images acquired by the cameras of the detection components at each station, further calculate the physical positions of the feature points. According to the feature information of the images during template making and real-time positioning and the obtained three-dimensional deflection angle information of the vehicle body, accurately calculate the position information and punching posture of the punching points, and then realize precise positioning and punching.

[0008] Preferably, the single-station manual operation process includes distortion calibration, N-point calibration of the punching robot, magnification acquisition, template making, and real-time positioning operation. The specific implementation process is as follows:

[0009] If the user selects "distortion calibration", the camera distortion calibration process will be executed. By executing the camera distortion calibration process, the distortion calibration files at different angles of the feature surface will be obtained. This file can be used for image distortion correction when the feature surface and the camera surface are at different angles;

[0010] If the user selects "N-point calibration of the punching robot", the N-point calibration process of the punching robot will be executed. By executing the N-point calibration process of the punching robot, the N-point calibration file will be obtained, and the image coordinates will be converted into physical coordinates in the coordinate system of the punching robot;

[0011] If the user selects "magnification acquisition", the magnification acquisition process will be executed. By executing the magnification acquisition process, the first-order polynomial relationship function between the height of the feature surface and the image magnification will be obtained. According to this function, the image coordinate conversion at different heights of the feature surface will be realized, so as to obtain accurate physical coordinates of the feature points;

[0012] If the user selects "template making", the template making process will be executed. By executing the template making process, the position information of the image processing feature points obtained by the camera and the position information of the punching points on the punching surface are obtained, and based on this, the positional relationship between the feature points and the punching points is obtained.

[0013] If the user selects "real-time positioning", the single-station real-time positioning process will be executed. Through the single-station real-time positioning process, the position information of the punching points will be obtained by performing real-time positioning on the currently selected station, and then the punching robot will be controlled to perform the punching operation.

[0014] Preferably, the distortion calibration process includes the following steps:

[0015] Control the checkerboard calibration plate to the initial position. Place the calibration plate near the detection height of the detection feature points on the object to be measured, that is, the vehicle. After placement, control the laser sensor to collect the height information at each position.

[0016] According to the obtained laser sensor height information, calculate the offset angle between the calibration plate and the camera plane.

[0017] If the angle meets the requirements, trigger the camera to take pictures and collect images.

[0018] Perform distortion calibration on the collected images to generate a distortion calibration file.

[0019] After generating the distortion calibration file, load the distortion calibration file for the images collected by the camera to perform distortion correction.

[0020] Preferably, calculating the offset angle between the calibration plate and the camera plane according to the obtained laser sensor height information further includes: Three laser sensors are installed during the distortion calibration process. One laser sensor is installed on each of the left rear and right rear sides of the calibration plate, that is, the top left distance measuring sensor and the top right distance measuring sensor, and one laser sensor is installed on the right front, that is, the top front distance measuring sensor. The distance between the top left distance measuring sensor and the top right distance measuring sensor is denoted as H1, the distance between the top right distance measuring sensor and the top front distance measuring sensor is denoted as H2, the height obtained by the top left distance measuring sensor is denoted as h1, the height obtained by the top right distance measuring sensor is denoted as h2, and the height obtained by the top front distance measuring sensor is denoted as h3. The offset angles that can be obtained according to the three heights include the left-right deviation angle and the front-back deviation angle with respect to the camera plane, where the left-right deviation angle is denoted as θ1 and the front-back deviation angle is denoted as θ2. The calculation formulas are as follows:

[0021]

[0022] Determine whether the obtained θ1 and θ2 are stable within the set threshold range. If so, it is considered that the checkerboard calibration board is in a stable and stationary state; otherwise, it is considered that the checkerboard calibration board is still in a moving state. At this time, determine whether the number of angle measurements exceeds the set value. If so, end the current distortion calibration operation and prompt "The calibration board is not stable. Please re-place the calibration board"; otherwise, continue to measure θ1 and θ2.

[0023] Within the range of ±10°, perform distortion calibration on the left-right deflection angle and the front-back deflection angle at intervals of 1°. When calibrating the left-right deflection angle, keep the front-back deflection angle at 0°. When the left-right deflection angle is between -10° and 10°, each time control the angle of the checkerboard calibration board to deviate by 1° for distortion calibration, and the same is true for the front-back deflection angle calibration process. According to the current calibration process, determine whether the currently measured angle is within the required range. For example, when performing distortion calibration at -10° of the left-right deflection angle, determine whether θ1 is within the threshold range near -10° set by the system, and at the same time ensure that θ2 is 0°. If the conditions are met, it is considered that the angle is within the required range and proceed to the next step; otherwise, control the movement of the checkerboard calibration board to adjust the angle of the checkerboard calibration board so that the calibration board angle meets the above angle requirements.

[0024] Preferably, the magnification acquisition includes the magnification within ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and a fitting function of magnification and distance is obtained by fitting.

[0025] Preferably, the magnification within ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and obtaining a fitting function of magnification and distance by fitting includes: The specific measurement process is from h - 10 cm to h + 10 cm, and the measurement is performed at intervals of 5 mm each time. First, control the calibration board to h - 10 cm, then obtain the distance from the upper laser sensor to the calibration board, and use the angle calculation method in the distortion calibration process to obtain the left-right deflection angle θ1' and the front-back deflection angle θ2' of the calibration board.

[0026] Determine whether θ1′ and θ2′ are stable. If not, it means that the calibration board has not come to a complete stop. Then determine whether the number of measurements exceeds the set value. If so, end the current magnification acquisition and wait for the user to operate; otherwise, continue to obtain the distance from the laser sensor to the calibration board and calculate the angle.

[0027] If θ1′ and θ2′ are stable, then determine whether θ1′ and θ2′ are within the required range. When the camera and the measured surface are completely perpendicular, after the collected image is corrected for distortion to remove the distortion effect of the lens itself, there is no distortion at each position of the image. Therefore, in order to remove the distortion of the angle on the magnification, it is required that θ1′ and θ2' are around 0°. If the conditions are met, trigger the camera to take a picture to obtain the image and save it. If θ1' and θ2′ are not within the required range, adjust the angle of the calibration board and then obtain θ1′ and θ2′ again.

[0028] Judge whether the images at all the above height positions have been collected. If so, process the images corresponding to all heights to obtain the pixel pitch of the checkerboard at the same position on the calibration board. The checkerboard position is selected as the checkerboard in the middle of the image because the distortion at the middle position of the image is the smallest and the error of the obtained pixel pitch is the smallest. Then, according to the actual physical size of the checkerboard, the proportional relationship between the pixel pitch and the physical size is the image magnification of the calculated height. If the images at all height positions have not been collected yet, adjust the height of the calibration board to collect the image at the next height.

[0029] After obtaining the magnifications at all heights, perform a first-order polynomial fitting according to the corresponding heights to obtain the fitting coefficients k and b, that is, m = k * h4 + b, where m is the magnification and h4 is the distance value measured by the laser sensor. Store the obtained fitting coefficients k and b.

[0030] Preferably, the template making includes: the system judges whether the tool center point TCP of the punching robot has been switched, that is, the system switches the TCP position of the punching robot from the marker head TCP to the punching head TCP. After the switching is completed, the punching robot will perform position movement and attitude adjustment in the workpiece coordinate system of the punching robot with the punching head TCP as the standard;

[0031] If not, the system first controls the punching robot to switch the tool TCP. After the switching is completed, control each laser sensor to obtain the distance from the vehicle body. If so, each laser sensor obtains the distance from the vehicle body;

[0032] The system judges whether the distances obtained by each laser sensor are stable, that is, each distance value is less than the set threshold;

[0033] If not, the system counts the number of measurements and judges whether this number exceeds the set value. If it does not exceed, continue to execute the system to judge whether the distances obtained by each laser sensor are stable. If it exceeds, the system prompts that the vehicle body is not placed stably and returns to judge whether the start making template button is clicked;

[0034] If so, calculate the deflection angles of each direction of the vehicle body according to the distances from the vehicle body obtained by each laser sensor; judge whether the obtained deflection angles of each direction are within the set range;

[0035] If not, prompt that the vehicle body position is abnormal, and the user needs to re-place the vehicle body position and return to judge whether the start making template button is clicked and execute;

[0036] If so, execute the vehicle body feature acquisition process;

[0037] The system judges whether the vehicle body feature acquisition is abnormal;

[0038] If not, the system controls the punching robot to move its punching head to the punching position; the system then records the intersection coordinates of the feature points, the line angles, and the punching point position coordinates respectively; saves the corresponding laser sensor position data, and returns to determine whether to execute by clicking the start template making button;

[0039] If so, according to the exception type prompt, ask the user to confirm the vehicle body position and end the template making process.

[0040] Preferably, the vehicle body feature acquisition process in template making includes the following steps:

[0041] The system controls the camera to collect the vehicle body feature surface image;

[0042] According to the vehicle body deflection angle, the system calls the corresponding distortion calibration file to correct the image distortion;

[0043] According to the vehicle body sheet metal features, the system performs feature template matching at the sheet metal indirect seams;

[0044] The system determines whether the feature matching is successful. If not, the system prompts that the vehicle body feature is not found, asks to confirm the vehicle body position, and this process ends; if so, corrects the position of the found feature, and then searches for the two straight lines at the seam respectively;

[0045] The system determines whether the two straight lines are successfully found. If not, it prompts that the seam at the vehicle body feature is not found, asks to confirm the vehicle body position, and this process ends; if so, finds the intersection of the two found straight lines, and performs coordinate transformation on the intersection point to obtain the physical coordinates.

[0046] Preferably, the real-time positioning punching process includes the following steps:

[0047] Perform the vehicle body angle acquisition process;

[0048] The system determines whether the vehicle body angle acquisition is successful; if not, it prompts a positioning exception and returns to continue to determine whether to click the start real-time positioning button. If so, execute the vehicle body feature acquisition process;

[0049] The system determines whether the vehicle body feature acquisition is successful; if not, it prompts that the vehicle body feature is not found or prompts that the seam at the vehicle body feature is not found, asks to confirm the vehicle body position, and returns to continue to determine whether to click the start real-time positioning button; if so, calculate the new punching point position based on the feature points, punching points, deflection angles in all directions of the vehicle body in template making and the feature point coordinates in real-time positioning;

[0050] The system determines whether the vehicle body offset position is within the set range. If not, it prompts that the vehicle body position is abnormal,

[0051] The user needs to reposition the vehicle body and then return to continue determining whether the start real-time positioning button is clicked. If so, after controlling the punching robot to adjust its posture, it runs above the new punching point, and the laser sensor obtains the distance from the new punching point in real time.

[0052] Determine whether the distance reaches the set range. If not, the punching robot continues to approach the new punching point by a certain distance and then determines whether the distance reaches the set range again. If so, the system controls the punching robot to punch. After completion, return to continue determining whether the start real-time positioning button is clicked.

[0053] Preferably, in the full-station automatic operation process, the single-station real-time positioning process is sequentially executed for a total of 6 workstations on the left and right sides of the top, front side, and rear side of the vehicle body. Among them, the left and right side workstations on the roof first execute the single-station real-time positioning process. After completing the single-station real-time positioning process on both sides of the roof, the three angles of the vehicle body relative to the template-made vehicle body in the three-dimensional direction can be obtained. Among them, the left and right deviation angles and the rotation angle within the camera plane can be used as the rotation angles required in the side single-station real-time positioning process.

[0054] The adoption of the present invention has the following beneficial effects:

[0055] (1) By using machine vision technology combined with high-precision sensors, the positional relationship of positioning points on different curved surfaces is determined, thereby realizing high-precision positioning of the vehicle body's featureless curved surface and the punching operation of the punching robot.

[0056] (2) Through the visual positioning of the feature surface and the rotation angles of the vehicle body's feature surface in the three-dimensional direction, combined with the relative positional relationship between the feature points and punching points on the feature surface, the position and punching posture of the punching point are obtained in real-time positioning detection, and finally the punching robot is controlled to perform the punching operation.

[0057] (3) In real-time detection, the approaching method of the laser sensor to the camera plane position during template making.

[0058] Coarse measurement by the laser sensor and coarse visual positioning. In real-time positioning detection, the height of the measured feature surface is obtained through the laser sensor, and the rotation angle and position offset of the feature point relative to the feature point during template making are obtained through the vision system. Based on the coarse visual positioning result, the laser sensor is controlled to adjust the measurement position on the camera plane so that it runs to the measurement position during template making, and then the fine measurement of the laser sensor is performed. Then, an accurate height value is obtained through the fine measurement result of the laser sensor, and the fine positioning of the feature information of the feature surface is performed visually.

[0059] (4) A high-precision positioning method for the vehicle body's curved surface during planar three-dimensional rotation and translation.

[0060] Obtain the accurate height value of the measured feature surface through a laser sensor. Use the corresponding image distortion correction file for image distortion correction at different heights. Fit the magnification at different heights to obtain the fitting function of height and magnification. Calculate the magnification according to the height during real-time detection and positioning. Through the magnification, convert the pixel coordinates of the feature points on the measured surface image in real-time detection into the coordinates under the pixel coordinate standard of the template-making image. Obtain the N-point calibration file through the N-point calibration of the punching robot and the camera. After the coordinate transformation of the collected vehicle body feature image through the N-point calibration, the translation amount of the feature points in the physical position can be obtained. Obtain the rotation angle of the three-dimensional plane, and calculate the position of the punching points on the vehicle body surface according to the angle and the translation amount of the feature points.

[0061] (5) Realize multi-station unmanned intervention positioning and punching on multiple different surfaces of the vehicle body.

[0062] First, perform the feature positioning on the top surface of the vehicle body. Obtain the rotation angle of the vehicle body feature through the top surface feature image of the vehicle body. Obtain the left and right rotation angles of the feature surface in the three-dimensional space through the laser sensors at the left and right stations on the roof. Obtain the front and rear rotation angles of the vehicle body through the laser sensors at the front and rear stations on the roof. Thus, the three rotation angles of the vehicle body in the three-dimensional space can be obtained, and then the positioning of the corresponding featureless punching points on the top surface can be realized. Then, through the three-dimensional rotation angle, perform the side feature surface positioning, and further obtain the position and punching posture of the featureless punching points on the side. Brief Description of the Drawings

[0063] Figure 1 It is the overall operation flowchart of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0064] Figure 2 It is the structural schematic diagram of the top detection component and the limit component of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0065] Figure 3 It is the structural schematic diagram of the right detection component of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0066] Figure 4 It is the structural schematic diagram of the left detection component of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0067] Figure 5 It is the single-station manual operation flowchart of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0068] Figure 6 It is the camera distortion calibration flowchart of the detection method for the relevant surface of the measured object in the embodiment of the present invention;

[0069] Figure 7It is the full-station automatic operation flowchart of the detection method for the relevant surface of the object to be measured in the embodiment of the present invention;

[0070] Figure 8 It is the schematic diagram of the deflection angle of the detection method for the relevant surface of the object to be measured in the embodiment of the present invention;

[0071] Figure 9 It is the schematic diagram of the workpiece surface coordinate system established by the punching robot in the embodiment of the present invention. Specific embodiments

[0072] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0073] A detection method for the relevant surface of the object to be measured in the embodiment of the present invention first transports the vehicle body to the assembly line flat plate. The vehicle body is successively transported to the flat plate assembly line with a limiting device by a crane. When the vehicle body runs to the visual detection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the stop signal is acquired by the visual system, and then user operation selection is performed. The visual system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are the distortion calibration process, the N-point calibration process, the magnification acquisition process, and the template making process. In the single-station real-time positioning process and the full-station real-time positioning process, through the feature point positions in the template making, the position of the laser sensor is adjusted to approach the position during template making, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding detection components at each station to acquire and process the image and laser sensor measurement values. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top detection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera surface. Then, the deflection angle of the vehicle body on the camera surface is calculated by obtaining the features of the image through the camera and the image features during template making. The corresponding deflection angle is used for processing during the distortion correction and image coordinate conversion at each station; through the images acquired by the cameras of the detection components at each station, the physical positions of the feature points are further calculated. According to the feature information of the images during template making and real-time positioning and the acquired three-dimensional deflection angle information of the vehicle body, the position information and punching posture of the punching points are accurately calculated, and then precise positioning and punching are realized.

[0074] Specifically, the positioning function of the relevant surface is realized through a relevant curved surface positioning and punching system based on machine vision. When the object to be detected runs to the positioning system, the positioning and punching at the relevant surface positioning are realized through the laser sensor positioning method, the machine vision positioning method and the positional relationship of the relevant surface. The operation method includes the following steps:

[0075] After the positioning system is powered on and started up, first execute the system startup initialization process, including: detecting the hardware status, including the connection status detection of the punching robots on both sides, the cameras at each station, and the laser sensors; if all the hardware is normally connected, proceed to the next step, otherwise give an alarm for abnormal connection of the relevant hardware and prompt the user to perform maintenance on the relevant hardware; confirm the position of the punching robot, if the position of the punching robot is not at the zero position, perform the zeroing operation on it according to the zeroing trajectory; read the relevant detection parameters during the system operation;

[0076] After the system startup initialization is completed, the positioning system executes different processes according to different operations of the user:

[0077] If the user selects single-station operation, then execute the single-station manual operation process; the single-station manual operation process can complete the preprocessing operation before real-time positioning of the specified station and the positioning and punching of a single station, mainly including: station selection process, camera distortion calibration process, punching robot N-point calibration process, magnification acquisition process, template production process and real-time positioning process.

[0078] If the user selects full-station operation, then the positioning system will execute the full-station automatic operation process; the full-station automatic operation process completes the positioning and punching operations of all stations to be punched in the set order. After completing the positioning and punching operations of all stations, run the punching robot to the zero point according to the zeroing trajectory and wait for the arrival of the next object to be detected. During the execution of the full-station automatic operation process, if an operation that requires an emergency pause occurs, through the emergency pause, stop the current punching operation and zero the punching robot.

[0079] In an embodiment of the present invention, a detection method for the relevant surface of the object to be measured is implemented through the detection of the relevant surface of the object to be measured. The detection of the relevant surface of the object to be measured includes a top detection component, a limiting device, a left detection component and a right detection component. First, transport the vehicle body to the assembly line flat plate, and the vehicle body is transported to the flat plate assembly line with a limiting device through a crane in turn, ensuring that the position deviation of the vehicle body transported to the flat plate each time is within ±5 cm and the angle deviation is within ±5°.

[0080] Further, in an embodiment of the present invention, refer to Figures 2 to 4, the top detection component includes a top right detection camera 1, a first top right light source 2, a second top right light source 3, a top right sensor left - right adjustment device 4, a top right sensor front - back adjustment device 5, a top right ranging sensor 6, a top left detection camera 7, a first top left light source 8, a second top left light source 9, a top left sensor left - right adjustment device 10, a top left sensor front - back adjustment device 11, a top left ranging sensor 12, a top front sensor left - right adjustment device 13, a top front sensor front - back adjustment device 14, and a top front ranging sensor 15. The top left detection camera 7 is arranged above the left roof position of the rear row seats of the vehicle, with the camera facing vertically downwards. The top left detection camera 7 is approximately at a position above the roof, and the effective measurement height is about 100 cm. The first top left light source 8 and the second top left light source 9 are relatively perpendicular. Among them, the direction of the first top left light source 8 is parallel to the front - back direction of the vehicle, and the second top left light source 9 is parallel to the left - right direction of the vehicle. The illumination directions of the first top left light source 8 and the second top left light source 9 point to the vicinity of the center of the camera's field of view, and the height of the light source is approximately 30 cm from the roof. The top left ranging sensor 12 is at a height of 20 cm from the roof. The top left sensor left - right adjustment device 10 controls the top left sensor front - back adjustment device 11 to achieve left - right movement, and the top left sensor front - back adjustment device controls the top left ranging sensor 12 to achieve front - back movement. It is equivalent to that the top left ranging sensor 12 can move left - right and front - back on the roof plane. The overall sensor movement adjustment mechanism is located near the top of the vehicle tail on the left - hand side vision system. The top right detection camera 1 is approximately at a position above the right roof position of the rear row seats of the vehicle, and the effective measurement height is about 100 cm, with the camera facing vertically downwards. The first top right light source 2 and the second top right light source 3 are relatively perpendicular. Among them, the direction of the first top right light source 3 is parallel to the front - back direction of the vehicle, and the second top right light source 2 is parallel to the left - right direction of the vehicle. Both are installed parallel to the roof, and the illumination direction points to the vicinity of the center of the camera's field of view, and the height of the light source is approximately 30 cm from the roof. The top left ranging sensor 12 is at a height of 20 cm from the roof. The top right sensor left - right adjustment device 4 controls the top right sensor front - back adjustment device 5 to achieve left - right movement, and the top right sensor front - back adjustment device controls the top right ranging sensor 6 to achieve front - back movement. It is equivalent to that the top right ranging sensor 6 can move left - right and front - back on the roof plane. The overall sensor movement adjustment mechanism is located near the top of the vehicle tail on the right - hand side vision system. The top front ranging sensor 15 is at a height of 20 cm from the roof. The top front sensor left - right adjustment device 13 controls the top front sensor front - back adjustment device 14 to achieve left - right movement, and the top front sensor front - back adjustment device 14 controls the top front ranging sensor 15 to achieve front - back movement. It is equivalent to that the top front ranging sensor 15 can move left - right and front - back on the roof plane. The overall sensor movement adjustment mechanism is located near the right roof of the front row of the vehicle.

[0081] The limiting device includes front - rear limiters 16 for the right front wheel, left - right limiters 17 for the right front wheel, front - rear limiters 18 for the left front wheel, left - right limiters 19 for the left front wheel, left - right limiters 20 for the right rear wheel, front - rear limiters 21 for the right rear wheel, front - rear limiters 22 for the left rear wheel, and left - right limiters 23 for the left rear wheel. Among them, the front - rear limiters 16 for the right front wheel and the front - rear limiters 18 for the left front wheel are respectively located at the front parts of the left and right tires at the front of the vehicle, restricting the vehicle from moving forward. The front - rear limiters 21 for the right rear wheel and the front - rear limiters 22 for the left rear wheel are respectively located at the rear parts of the rear tires of the vehicle, restricting the vehicle from moving backward. The left - right limiters 17 for the right front wheel and the left - right limiters 20 for the right rear wheel are respectively located on the side surfaces of the front and rear tires on the right side of the vehicle, restricting the vehicle from moving to the right. The left - right limiters 19 for the left front wheel and the left - right limiters 23 for the left rear wheel are respectively located on the side surfaces of the front and rear tires on the left side of the vehicle, restricting the vehicle from moving to the left.

[0082] The right detection component includes a right front ranging sensor 24, a right front sensor front-back adjustment device 25, a right front sensor up-down adjustment device 26, a right front detection camera 27, a first right front light source 28, a second right front light source 29, a right rear ranging sensor 30, a right rear sensor front-back adjustment device 31, a right rear sensor up-down adjustment device 32, a right rear detection camera 33, a first right rear light source 34, and a second right rear light source 35. The right front detection camera 27 is roughly located on the right side of the connection position between the right front door of the vehicle and the vehicle body, with an effective measurement distance of approximately 100 cm, a height from the ground greater than 80 cm, and the camera surface parallel to the front-back direction of the vehicle and facing left. The first right front light source 28 and the second right front light source 29 are relatively perpendicular. Among them, the direction of the first right front light source 28 is parallel to the front-back direction of the vehicle, and the second right front light source 29 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the lighting direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the right side of the vehicle. The right front ranging sensor 24 is 20 cm away from the right front sheet metal of the vehicle. The right front sensor up-down adjustment device 26 controls the right front sensor front-back adjustment device 25 to achieve up-down movement, and the right front sensor front-back adjustment device 25 controls the right front ranging sensor 24 to achieve front-back movement. It is equivalent to that the right front ranging sensor 24 can perform front-back and up-down movements on the right side plane of the vehicle. The overall sensor movement adjustment mechanism is located near the front of the vehicle in the right front vision system. The right rear detection camera 33 is roughly located on the right side of the connection position between the right rear fender and the right rear bumper of the vehicle, with an effective measurement distance of approximately 100 cm, a height from the ground greater than 80 cm, and the camera surface parallel to the front-back direction of the vehicle and facing left. The first right rear light source 34 and the second right rear light source 35 are relatively perpendicular. Among them, the direction of the first right rear light source 34 is parallel to the front-back direction of the vehicle, and the second right rear light source 35 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the lighting direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the right side of the vehicle. The right rear ranging sensor 30 is 20 cm away from the right front sheet metal of the vehicle. The right rear sensor up-down adjustment device 32 controls the right rear sensor front-back adjustment device 31 to achieve up-down movement, and the right rear sensor front-back adjustment device 31 controls the right rear ranging sensor 30 to achieve front-back movement. It is equivalent to that the right rear ranging sensor 30 can perform front-back and up-down movements on the right side plane of the vehicle. The overall sensor movement adjustment mechanism is located away from the rear of the vehicle in the right rear vision system.

[0083] The left detection component includes a left front distance measurement sensor 36, a left front sensor vertical adjustment device 37, a left front sensor horizontal adjustment device 38, a left front detection camera 39, a first left front light source 40, a second left front light source 41, a left rear distance measurement sensor 42, a left rear sensor vertical adjustment device 43, a left rear sensor horizontal adjustment device 45, a left rear detection camera 44, a first left rear light source 46, and a second left rear light source 47. The left front detection camera 39 is approximately located on the left side of the connection position between the left front door of the vehicle and the vehicle body. The effective measurement distance is approximately 100 cm, the height from the ground is greater than 80 cm, and the camera plane is parallel to the front-rear direction of the vehicle and faces right. The first left front light source 40 and the second left front light source 41 are relatively perpendicular. Among them, the direction of the first left front light source 40 is parallel to the front-rear direction of the vehicle, and the second left front light source 41 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the light illumination direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the left side of the vehicle. The left front distance measurement sensor 36 is 20 cm away from the right front sheet metal of the vehicle. The left front sensor vertical adjustment device 38 controls the left front sensor horizontal adjustment device 37 to achieve vertical movement, and the left front sensor horizontal adjustment device 37 controls the left front distance measurement sensor 36 to achieve horizontal movement. It is equivalent to that the left front distance measurement sensor 36 can move horizontally and vertically on the left side plane of the vehicle. The overall sensor movement adjustment mechanism is located near the front of the vehicle in the left front vision system. The left rear detection camera 44 is approximately located on the left side of the connection position between the left rear fender and the left rear bumper of the vehicle. The effective measurement distance is approximately 100 cm, the height from the ground is greater than 80 cm, and the camera plane is parallel to the front-rear direction of the vehicle and faces right. The first left rear light source 46 and the second left rear light source 47 are relatively perpendicular. Among them, the direction of the first left rear light source 46 is parallel to the front-rear direction of the vehicle, and the second left rear light source 47 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the light illumination direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the right side of the vehicle. The left rear distance measurement sensor 42 is 20 cm away from the right front sheet metal of the vehicle. The left rear sensor vertical adjustment device 43 controls the left rear sensor horizontal adjustment device 45 to achieve vertical movement, and the left rear sensor horizontal adjustment device 45 controls the left rear distance measurement sensor 42 to achieve horizontal movement. It is equivalent to that the left rear distance measurement sensor 42 can move horizontally and vertically on the left side plane of the vehicle. The overall sensor movement adjustment mechanism is located away from the rear of the vehicle in the left rear vision system.

[0084] In an embodiment of the present invention, before the system performs full-station automatic operation, single-station manual operation needs to be carried out. Through the single-station manual operation, the image distortion correction file, the fitting function of the magnification and height, the coordinate conversion file between the image and the punching robot, and the positional relationship between the image feature plane and the punching points on the positioned punching surface of each station are obtained. Through the obtained files and data, automatic positioning punching operations for each station are realized. SeeFigure 5 Shown is the flowchart of single-station manual operation. The single-station manual operation interface provides operation buttons for distortion calibration, N-point calibration of the punching robot, magnification acquisition, template making, and real-time positioning. Corresponding processes can be executed according to the relevant operations of the user. The specific execution process is as follows:

[0085] The single-station operation process first executes the station selection process. Before entering the single-station manual operation interface, the user needs to first select the station to be operated. After completion of the selection, enter the manual operation interface.

[0086] If the user clicks the "Distortion Calibration" button, the camera distortion calibration process will be executed. By executing the camera distortion calibration process, distortion calibration files at different angles of the feature surface will be obtained, and this file can be used for image distortion correction when the feature surface and the camera surface are at different angles.

[0087] If the user clicks the "N-point Calibration of Punching Robot" button, the N-point calibration process of the punching robot will be executed. By executing the N-point calibration process of the punching robot, an N-point calibration file will be obtained to realize the conversion of image coordinates to physical coordinates in the coordinate system of the punching robot.

[0088] If the user clicks the "Magnification Acquisition" button, the magnification acquisition process will be executed. By executing the magnification acquisition process, a first-order polynomial relationship function between the height of the feature surface and the image magnification will be obtained. According to this function, the conversion of image coordinates at different heights of the feature surface can be realized, thereby obtaining accurate physical coordinates of feature points.

[0089] If the user clicks the "Template Making" button, the template making process will be executed. By executing the template making process, the position information of the image processing feature points obtained by the camera and the position information of the punching points on the punching surface can be obtained, and based on this, the position relationship between the feature points and the punching points can be obtained.

[0090] If the user clicks the "Real-time Positioning" button, the single-station real-time positioning process will be executed. Through the single-station real-time positioning process, real-time positioning of the currently selected station will be performed to obtain the position information of the punching points, and then the punching robot will be controlled to perform punching operations. During the real-time positioning process, an emergency pause operation can be performed. After the user presses the "Emergency Pause" button, the emergency pause processing process will be executed.

[0091] When the system is in the single-station operation standby state, after the user presses the "Exit" button, the single-station operation can be exited.

[0092] In an embodiment of the present invention, cameras need to be installed at each station of the object to be detected for visual positioning. Since the feature surface to be detected and the camera surface will present different angles due to the different placement positions of the object to be detected, the camera distortion calibration process of this system will perform distortion calibration for various different angles of the feature surface and the camera surface. In the template production process, the corresponding distortion calibration file is called according to the measured angles of the feature surface and the camera surface for image distortion correction. Through the camera distortion calibration process, distortion calibration files with different deflection angles between the calibration plate and the camera surface can be generated. Each distortion calibration file consists of the perspective distortion of the image and the radial distortion parameters of the image corresponding to the deflection angle. These distortion files are used for image distortion correction. Refer to Figure 6 , the camera distortion calibration process includes the following steps:

[0093] Determine whether the user clicks the "Distortion Calibration" button. If so, perform distortion calibration; otherwise, wait for the user to operate.

[0094] Control the checkerboard calibration plate to the initial position. Place the calibration plate near the detection height of the feature points to be measured on the object to be detected, that is, the vehicle. After placement, control the laser sensor to collect height information at each position.

[0095] According to the height information obtained by the laser sensor, calculate the offset angle between the calibration plate and the camera surface. During the distortion calibration process, 3 laser sensors are installed. One laser sensor is installed on each of the left rear and right rear sides of the calibration plate, that is, the top left distance measuring sensor and the top right distance measuring sensor, and one laser sensor is installed on the front right, that is, the top front distance measuring sensor. The distance between the top left distance measuring sensor and the top right distance measuring sensor is denoted as H1, the distance between the top right distance measuring sensor and the top front distance measuring sensor is denoted as H2, the height obtained by the top left distance measuring sensor is denoted as h1, the height obtained by the top right distance measuring sensor is denoted as h2, and the height obtained by the top front distance measuring sensor is denoted as h3. The offset angles that can be obtained according to the three heights include the left-right deflection angle and the front-back deflection angle with respect to the camera surface, where the left-right deflection angle is denoted as θ1 and the front-back deflection angle is denoted as θ2. The calculation formulas are as follows:

[0096]

[0097] Determine whether the obtained θ1 and θ2 are stable within the set threshold range. If so, it is considered that the checkerboard calibration plate is in a stable and stationary state; otherwise, it is considered that the checkerboard calibration plate is still in a moving state. At this time, determine whether the number of angle measurements exceeds the set value. If so, end the current distortion calibration operation and prompt "The calibration plate is not stable. Please re-place the calibration plate"; otherwise, continue to measure θ1 and θ2.

[0098] Within the range of ±10°, the distortion calibration is performed on the left-right deviation angle and the front-back deviation angle at intervals of 1°. When calibrating the left-right deviation angle, the front-back deviation angle is kept at 0°. Between -10° and 10° of the left-right deviation angle, the angle of the checkerboard calibration plate is offset by 1° each time to perform the distortion calibration. The same process is applied to the calibration of the front-back deviation angle. According to the current calibration process, it is judged whether the currently measured angle is within the required range. For example, when performing the distortion calibration at -10° of the left-right deviation angle, it is judged whether θ1 is within the threshold range near -10° set by the system, and at the same time, it is necessary to ensure that θ2 is 0°. If the conditions are met, the angle is considered to be within the required range and the next step is processed. Otherwise, it is necessary to control the movement of the checkerboard calibration plate to adjust the angle of the checkerboard calibration plate so that the angle of the calibration plate meets the above angle requirements.

[0099] If the angle meets the requirements, the camera is triggered to take pictures and collect images.

[0100] Perform distortion calibration on the collected images to generate a distortion calibration file.

[0101] After generating the distortion calibration file, load the distortion calibration file for the images collected by the camera to perform distortion correction.

[0102] Obtain the horizontal pixel pitch and vertical pixel pitch of the checkerboards at the upper, lower, left, right, and middle positions of the checkerboard calibration plate respectively.

[0103] Judge whether the horizontal pixel pitch and vertical pixel pitch in the 5 directions are all within the threshold range near the standard value. If so, it means that the distortion calibration is successful, and the images at the current angle can be correctly distortion-corrected, and the generated distortion calibration file and the corresponding left-right deviation angle and front-back deviation angle of the calibration plate are stored; otherwise, it means that the distortion correction fails, and the distance from the laser sensor to the calibration plate is re-obtained and the calibration plate angle is calculated, and then the distortion calibration at the current angle is performed again.

[0104] If the distortion calibration at the current angle is successful, it is judged whether all the above angles have completed the distortion calibration. If so, the distortion calibration is completed and waiting for the user's operation. Otherwise, the angle of the calibration plate is adjusted to continue the distortion calibration of the next angle.

[0105] In an embodiment of the present invention, the vision system performs feature localization to obtain the pixel coordinates of the feature points. The punching robot runs according to the coordinate system defined by the punching robot. It is necessary to let the punching robot locate the punching points according to the position of the feature points detected by the vision. Therefore, it is necessary to convert the vision coordinates into the punching robot coordinates. After unifying the two coordinate systems, the punching robot can be controlled through the relative position offset detected by the vision. This system generates an N-point calibration file through the N-point calibration process of the punching robot, and realizes the conversion between the vision coordinates and the punching robot coordinates through the N-point calibration file in real-time positioning. The N-point calibration process of the punching robot includes:

[0106] After entering the N - point calibration interface, wait for the user to perform the N - point calibration operation. If the user clicks the "N - point calibration" button, then perform the N - point calibration; otherwise, wait for the user's operation.

[0107] The punching robot creates the world coordinate system according to the characteristic surface of the vehicle body. The world coordinate system is used during the operation of the punching robot, and this coordinate system can be remade according to the actual working conditions. In this system, the roof camera surface of the vehicle body is used as the XY coordinate system, and the vertical direction is the Z - direction of the coordinate system. Remaking the world coordinate system in this way can make the visual coordinate system consistent with the XY coordinate system direction of the punching robot.

[0108] The punching robot sets the tool TCP (Tool Center Point). Set the tool TCP position of the punching robot to the marking head at the N - point calibration, that is, set the pose calculation reference point of the punching robot to the marking head. Convert the pose calculation reference point of the punching robot to the marking head. In this way, any change in the pose of the punching robot will not affect the coordinates of the marking head, and the coordinates of the marking head are also the actual position of the punching robot during the N - point calibration process.

[0109] Obtain the distance from the upper laser sensor to the marking plate, and calculate the front - rear deflection angle and left - right deflection angle of the marking plate.

[0110] Judge whether the angle is within the angle range of distortion correction. If so, control the punching robot to make N - point circular dot marks within the camera's field of view; otherwise, prompt the user to adjust the calibration plate angle and wait for the user's operation.

[0111] After the punching robot finishes making N - point circular dot marks, control the punching robot to exit the marking area and leave the field of view.

[0112] Control the camera to collect images, and respectively call the distortion calibration files at the corresponding angles according to the measured left - right deflection angle and front - rear deflection angle between the marking plate and the camera surface for distortion correction.

[0113] Find the center - pixel coordinates of N marking points through the center - finding algorithm.

[0114] According to the center - pixel coordinates of N marking points and the corresponding physical coordinates of the punching robot, perform N - point calibration to obtain the N - point calibration file. This N - point calibration file is the transformation matrix between the camera vision coordinate system and the punching robot coordinate system in the XY two - dimensional plane. Through this transformation matrix, the pixel coordinate values of the camera can be converted into the physical coordinate values of the punching robot.

[0115] After obtaining the N - point calibration file, perform N - point coordinate conversion for the current image to verify whether the generated calibration file is correct.

[0116] Perform circular search on the punctuation points in the currently completed distortion-corrected image to find the respective center pixel coordinates of N points. Call the above-generated N-point calibration file to convert the center pixel coordinates of the N points into the physical coordinates of the punching robot.

[0117] Compare the obtained physical coordinates of the N points with the physical coordinates of the N points recorded during the punching process of the punching robot to obtain statistical data. Determine whether the difference is within the set threshold range. If so, store the N-point calibration file and the corresponding height value measured by the laser sensor. Otherwise, prompt "Abnormal coordinate conversion after N-point calibration. Please re-perform N-point calibration" and wait for the user to operate.

[0118] In an embodiment of the present invention, due to the physical characteristics of machine vision imaging, when the measured surface is at different heights, the measured object appears larger near and smaller far in the image. When performing the conversion of the physical coordinates of the feature points, in the template making process and the real-time positioning punching process, there will be a difference in the distance of the feature surface relative to the laser sensor. Therefore, the coordinate standards of the same image collected in the two processes are different, and there will be a position deviation when directly performing coordinate conversion. This system converts the feature point coordinates of the images collected in the two processes into the image coordinates during N-point calibration, and then converts the image coordinates into physical coordinates. Since the image magnification factor can quantitatively represent the size of the measured object in the image at different heights, this process is converted through the magnification factors at different heights. The image magnification factor acquisition process includes the following steps:

[0119] After entering the magnification factor acquisition interface, wait for the user to perform the magnification factor acquisition operation. If the user clicks the "Magnification Factor Acquisition" button, then perform the magnification factor acquisition; otherwise, wait for the user to operate.

[0120] Combined with the tooling accuracy at the vehicle body positioning punching station and the situation of the vehicle body itself, this system will obtain the magnification factors of ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and fit to obtain the fitting function of the magnification factor and the distance. The specific measurement process is from (standard distance h - 10 cm) to (standard distance h + 10 cm), with each interval of 5 mm for measurement. Therefore, first control the calibration plate to the position of (standard distance h - 10 cm), then obtain the distance from the upper laser sensor to the calibration plate, and use the angle calculation method in the distortion calibration process to obtain the left-right deflection angle θ1' and the front-back deflection angle θ2′ of the calibration plate.

[0121] Determine whether θ1′ and θ2′ are stable. If not, it means that the calibration plate has not come to a complete stop yet. Then determine whether the number of measurements exceeds the set value. If so, end the current magnification factor acquisition and wait for the user to operate. Otherwise, continue to obtain the distance from the laser sensor to the calibration plate and calculate the angle.

[0122] If θ1′ and θ2′ are stable, then it is judged whether θ1′ and θ2′ are within the required range. When the camera and the measured surface are completely perpendicular, after the acquired image is corrected for distortion to remove the influence of the lens itself, there is no distortion at each position of the image. Therefore, in order to remove the influence of the distortion introduced by the angle on the magnification, it is required that θ1′ and θ2′ are around 0° and between ±0.1°. If the condition is satisfied, the camera is triggered to take a picture to obtain the image and save it. If θ1′ and θ2′ are not within the required range, the angle of the calibration plate is adjusted and then θ1′ and θ2′ are obtained again.

[0123] It is judged whether the images at all the above-mentioned height positions have been collected. If so, for the images corresponding to all the heights, the pixel pitch of the checkerboard at the same position on the calibration plate is obtained through image processing. The checkerboard position is selected as the checkerboard in the middle of the image because the distortion at the middle position of the image is the smallest and the error of the obtained pixel pitch is the smallest. Then, according to the actual physical size of the checkerboard, the proportional relationship between the pixel pitch and the physical size is the image magnification of the calculated height. If the images at all the height positions have not been collected yet, the height of the calibration plate is adjusted to collect the image at the next height.

[0124] After obtaining the magnifications at all the heights, a first-order polynomial fitting is performed according to the corresponding heights to obtain the fitting coefficients k and b, that is, m = k*h4 + b, where m is the magnification and h4 is the distance value measured by the laser sensor. The fitting coefficients k and b are obtained and stored.

[0125] In an embodiment of the present invention, the template making process includes the following steps:

[0126] Tap the template making button to enter the template making process interface;

[0127] Judge whether the start making template button is tapped.

[0128] If not, judge whether the exit template making button is tapped. If so, exit the template making interface. If not, the system does not act and waits for the user to tap the start making template button.

[0129] If so, the system judges whether the punching robot has switched the tool TCP (Tool Center Point), that is, the system switches the TCP position of the punching robot from the marker head TCP to the punching head TCP. After the switching is completed, the punching robot will perform position movement and attitude adjustment in the workpiece coordinate system of the punching machine with the punching head TCP as the standard.

[0130] If not, the system first controls the punching robot to switch the tool TCP. After the switching is completed, it controls each laser sensor (including 3 on the top and 4 on the side) to obtain the distance from the vehicle body. If so, each laser sensor (including 3 on the top and 4 on the side) obtains the distance from the vehicle body.

[0131] The system determines whether the distances obtained by each laser sensor are stable, that is, whether each distance value is less than the set threshold.

[0132] If not, the system counts the number of measurements and determines whether this number exceeds the set value. If it does not exceed, the system continues to determine whether the distances obtained by each laser sensor are stable, that is, whether each distance value is less than the set threshold. If it exceeds, the system prompts that the vehicle body is not placed stably and returns to determine whether the start template-making button is clicked.

[0133] If so, calculate the deflection angles of each direction of the vehicle body (the rotation angle α of the vehicle body around the Y-axis direction, the rotation angle β of the vehicle body around the X-axis direction, and the rotation angle γ of the vehicle body around the Z-axis direction) according to the distances from the vehicle body obtained by each laser sensor (referring to the three sensors on the top and the two sensors on the right or left).

[0134] Judge whether the obtained deflection angles of each direction are within the set range.

[0135] If not, prompt that the position of the vehicle body is abnormal, and the user needs to re-place the position of the vehicle body, and return to determine whether the start template-making button is clicked to execute.

[0136] If so, execute the vehicle body feature acquisition process.

[0137] The system determines whether there is an abnormality in the vehicle body feature acquisition.

[0138] If not, the system controls the punching robot to make the punching head run to the punching position; the system then records the intersection coordinates of the feature points, the straight line angle, and the punching point position coordinates respectively; saves the corresponding laser sensor position data, and returns to determine whether the start template-making button is clicked to execute.

[0139] If so, according to the abnormality type (including not finding the vehicle body feature and not finding the seam at the vehicle body feature), prompt the user to confirm the position of the vehicle body and end the template-making process.

[0140] Further, in an embodiment of the present invention, the vehicle body feature acquisition process in template making is as follows:

[0141] The system controls the camera to collect the image of the vehicle body feature surface.

[0142] According to the vehicle body deflection angle, the system calls the corresponding distortion calibration file to perform image distortion correction.

[0143] According to the vehicle body sheet metal feature, the system performs feature template matching at the sheet metal indirect seam.

[0144] The system determines whether the feature is found successfully, that is, whether the matching is successful. If not, the system prompts that the vehicle body feature is not found and asks the user to confirm the position of the vehicle body, and this process ends; if so, correct the position of the found feature, and then perform the search for two straight lines at the seam respectively.

[0145] The system determines whether the search for two straight lines is successful. If not, it prompts that the seam at the vehicle body feature is not found, asks to confirm the vehicle body position, and this process ends; if so, it finds the intersection point of the two found straight lines and performs coordinate transformation on the intersection point to obtain the physical coordinates.

[0146] In one embodiment of the present invention, the real-time positioning and punching process includes the following steps:

[0147] Tap the real-time positioning and punching button to enter the real-time positioning and punching process interface;

[0148] Determine whether the start real-time positioning button is tapped.

[0149] If not, determine whether the exit single-station operation button is tapped. If so, exit the real-time positioning and punching interface; if not, the system does not act and waits for the user to tap the start real-time positioning button.

[0150] If so, perform the vehicle body angle acquisition process.

[0151] The system determines whether the vehicle body angle acquisition is successful; if not, it prompts abnormal positioning and returns to continue determining whether the start real-time positioning button is tapped. If so, execute the vehicle body feature acquisition process.

[0152] The system determines whether the vehicle body feature acquisition is successful; if not, it prompts that the vehicle body feature is not found or the seam at the vehicle body feature is not found, asks to confirm the vehicle body position, and returns to continue determining whether the start real-time positioning button is tapped; if so, based on the feature points, punching points, deflection angles in each direction of the vehicle body in the template making and the coordinate of the feature point in the real-time positioning, calculate the position of the new punching point.

[0153] The system determines whether the vehicle body offset position is within the set range. If not, it prompts abnormal vehicle body position,

[0154] The user needs to re-place the vehicle body position and return to continue determining whether the start real-time positioning button is tapped; if so, after controlling the punching robot to adjust its posture, it runs above the new punching point, and the laser sensor obtains the distance from the new punching point in real time.

[0155] Determine whether the distance reaches the set range. If not, the punching robot continues to approach the new punching point by a certain distance and then determines whether the distance reaches the set range; if so, the system controls the punching robot to punch. After completion, return to continue determining whether the start real-time positioning button is tapped.

[0156] Furthermore, the vehicle body angle acquisition process in the real-time positioning and punching process is as follows:

[0157] Execute the sensor approaching the template point process.

[0158] The system determines whether the approximation is completed; if not, continue to execute the sensor approximation template point process. If so, the system calculates and obtains the non-camera plane deflection angle based on the distance measured by the laser sensor.

[0159] Execute the vehicle body feature acquisition process to obtain the camera plane deflection angle. Thus, the rotation angles of the vehicle body in three directions are obtained.

[0160] The above vehicle body angle acquisition process is different for two cases: real-time positioning and punching on the top of the vehicle body and real-time positioning and punching on the side of the vehicle body:

[0161] Taking the real-time positioning and punching on the top of the vehicle body as an example, the workpiece surface coordinate system established by the punching robot is as Figure 9 shown, where the positive direction of the Y-axis is the vehicle head direction.

[0162] The non-camera plane deflection angle obtained by real-time positioning and punching on the top of the vehicle body includes the rotation angle α of the vehicle body around the Y-axis (the left-right tilt angle of the vehicle body), the rotation angle β of the vehicle body around the X-axis (the front-back tilt angle of the vehicle body), and the top camera plane deflection angle refers to the rotation angle γ of the vehicle body around the Z-axis (the rotation angle in the roof plane parallel to the top camera plane).

[0163] The camera plane deflection angle obtained by real-time positioning and punching on the side of the vehicle body refers to the rotation angle β of the vehicle body around the X-axis obtained by the side camera, and the non-camera plane deflection angle includes the rotation angle α around the Y-axis and the rotation angle γ of the vehicle body around the Z-axis obtained by the top camera.

[0164] In an embodiment of the present invention, due to the angle change at different positions in the curved surface space, there may be a position deviation when the point laser sensor collects the distance, resulting in inaccurate obtained distance. This system adopts the sensor approximation process to accurately obtain the position of the point laser sensor on the feature surface during template making. The sensor approximation template point process in the vehicle body angle acquisition process is as follows:

[0165] Each laser sensor obtains the distance from the vehicle body for rough positioning.

[0166] The system determines whether the distances obtained by each laser sensor are stable, that is, each distance value is less than the set threshold. If not, the system counts the number of measurements and determines whether this number exceeds the set value. If it does not exceed, return to continue the rough positioning. If it exceeds, the system prompts that the vehicle body is not placed stably and ends this process.

[0167] If so, according to the distances from the vehicle body obtained by each laser sensor, calculate the deflection angles of each direction of the vehicle body.

[0168] The system determines whether the deflection angles are all within the set range; if not, prompt that the vehicle body position is abnormal and the user needs to re-place the vehicle body to end this process; if so, execute the vehicle body feature acquisition process.

[0169] Find the position difference between the target point and the template point, calculate the deviation value for the sensor to move to the position of the template point, and control the movement of the sensor.

[0170] After moving, the laser sensor measures the distance from the vehicle body. Based on the distances from the vehicle body obtained by them, calculate the deflection angles in all directions of the vehicle body.

[0171] The above sensor approaching template point process is different for two cases: real-time positioning and punching on the top of the vehicle body and real-time positioning and punching on the side of the vehicle body:

[0172] In real-time positioning and punching on the top of the vehicle body, each laser sensor refers to the top sensors (6, 12, 15). The calculated deflection angles in all directions of the vehicle body include the rotation angle α of the vehicle body around the Y-axis direction and the rotation angle β of the vehicle body around the X-axis direction.

[0173] In real-time positioning and punching on the side of the vehicle body, each laser sensor includes the top sensors (6, 12) and the sensors for measuring the working distance of the camera at the corresponding positioning and punching positions.

[0174] The vehicle body feature acquisition process in real-time positioning and punching is the same as that in template making, except that "find the intersection point of the two found straight lines and perform coordinate transformation on the intersection point to obtain the physical coordinates" is changed to "find the intersection point of the two found straight lines, and according to the height of the real-time feature point and the height of the template feature point, convert the intersection point of the two straight lines into the pixel coordinates of the template image. And perform coordinate transformation on the intersection point to obtain the physical coordinates".

[0175] The formula for converting the intersection point (i.e., the feature point) of the two straight lines into the pixel coordinates of the template image in the vehicle body feature acquisition process during real-time positioning and punching is as follows:

[0176]

[0177] where m 0 represents the magnification of the feature point in template making, which is calculated by substituting the distance value measured by the corresponding station sensor during template making into the first-order fitting function in the magnification acquisition process; m 1 represents the magnification of the feature point in real-time positioning, which is calculated by substituting the distance value measured by the corresponding station sensor after position correction during real-time positioning into the first-order fitting function in the magnification acquisition process; the point (X, Y) represents the pixel coordinates of the feature point in real-time positioning, and the point (X center , Y center ) represents the pixel coordinates of the image center point in real-time positioning, and the point (X', Y') represents the pixel coordinates of the feature point in real-time positioning after magnification compensation.

[0178] The point (X', Y') is converted into the physical coordinate point (X 1 , Y 1 ) through coordinate transformation.

[0179] Calculating the position of the new punching point in the real-time positioning punching process The specific calculation process is as follows:

[0180] Case 1: The deflection angles of the vehicle body measured during real-time positioning are the same as those during template making in all three directions;

[0181]

[0182] Among them, the point (X 0 , Y 0 ) represents the physical coordinates obtained by coordinate conversion of the pixel coordinates of the feature points in template making, and the point represents the physical coordinates of the punching point in template making.

[0183] Case 2: The deflection angles (α and β) of the vehicle body measured during real-time positioning are the same as those during template making;

[0184]

[0185] Among them, the angle γ Δ is equal to the vehicle body rotation angle γ 1 during real-time positioning minus the vehicle body rotation angle γ 0 during template making, indicating that the vehicle body rotates by an angle γ Δ relative to that during template making during real-time positioning, causing the offset in the X-axis direction of the punching point, indicating that the vehicle body rotates by an angle γ Δ relative to that during template making during real-time positioning, causing the offset in the Y-axis direction of the punching point.

[0186] Case 3: The deflection angle β of the vehicle body measured during real-time positioning is the same as that during template making;

[0187] (1) If the line segment formed by the punching point and the feature point during template making has no angle with the template plane;

[0188]

[0189] Among them, the angle α Δ is equal to the vehicle body rotation angle α 1 during real-time positioning minus the vehicle body rotation angle α 0 , indicating that the offset in the X-axis direction in Case 2 is multiplied by cos(α Δ ), which is due to the change in the offset in the X-axis direction in Case 2 caused by the left-right tilt of the vehicle body, that is, indicating that the offset becomes its projection on the template plane.

[0190] (2) If the line segment formed by the punching point and the feature point during template making has an angle η with the template plane, the punching point is lower than the feature point and the punching point slopes downward relative to the feature point (or the punching point is higher than the feature point and the punching point slopes upward relative to the feature point);

[0191]

[0192] Among them, the angle η X is equal to the projection angle of the angle formed by the line segment formed by the punching point and the feature point during template making and the template plane on the plane formed by the X-axis and the Z-axis, indicating the X-axis offset that still needs to be compensated when the real-time positioning of the vehicle body has a different left-right tilt angle compared to template making and there is an angle η.

[0193] (3) If the line segment formed by the punching point and the feature point during template making has an angle η with the template plane, the punching point is higher than the feature point and the punching point slopes downward relative to the feature point (or the punching point is lower than the feature point and the punching point slopes upward relative to the feature point);

[0194]

[0195] Case 4: The vehicle body deflection angle α measured during real-time positioning is the same as that during template making;

[0196] (1) If the line segment formed by the punching point and the feature point during template making has no angle with the template plane;

[0197]

[0198] Among them, the angle β Δ is equal to the vehicle body rotation angle β during real-time positioning 1 minus the vehicle body rotation angle β during template making 0 .

[0199] represents the Y-axis direction offset in Case 2 multiplied by cos(β Δ ), which is the change in the Y-axis direction offset in Case 2 caused by the front-back tilt of the vehicle body, that is, it represents that the offset becomes its projection on the template plane.

[0200] (2) If the line segment formed by the punching point and the feature point during template making has an angle η with the template plane, the punching point is lower than the feature point and the punching point slopes downward relative to the feature point (or the punching point is higher than the feature point and the punching point slopes upward relative to the feature point);

[0201]

[0202] Among them, the angle η YIt is equal to the projection angle on the plane formed by the Y-axis and the Z-axis of the angle formed by the line segment composed of the punching point and the feature point during template production and the template plane. It represents the Y-axis offset that still needs to be compensated when the front and rear tilt angles of the vehicle body during real-time positioning are different from those during template production and there is an included angle η.

[0203] (3) If the line segment formed by the punching point and the feature point during template production has an included angle η with the template plane, the punching point is higher than the feature point and the punching point is inclined downward relative to the feature point (or the punching point is lower than the feature point and the punching point is inclined upward relative to the feature point);

[0204]

[0205] Case 5: The tilt angles α and β of the vehicle body measured during real-time positioning are different from those during template production;

[0206] (1) If the line segment formed by the punching point and the feature point during template production has no included angle with the template plane;

[0207]

[0208] (2) If the line segment formed by the punching point and the feature point during template production has an included angle η with the template plane, the punching point is lower than the feature point and the punching point is inclined downward relative to the feature point (or the punching point is higher than the feature point and the punching point is inclined upward relative to the feature point);

[0209]

[0210] (3) If the line segment formed by the punching point and the feature point during template production has an included angle η with the template plane, the punching point is higher than the feature point and the punching point is inclined downward relative to the feature point (or the punching point is lower than the feature point and the punching point is inclined upward relative to the feature point);

[0211]

[0212] In an embodiment of the present invention, the full-station automatic operation process completes the positioning and punching operations for all stations to be punched in accordance with the set sequence. After completing the positioning and punching operations for all stations, the punching robot is run to the zero return point according to the zero return trajectory and waits for the arrival of the next object to be detected. As Figure 7 shown, the full-station automatic operation process includes the following steps:

[0213] After entering the full-station operation interface, wait for the user to click the "Full-station operation" button. If the user clicks "Full-station operation", then execute the vehicle body in-place judgment process; otherwise, wait for the user's operation.

[0214] The vehicle body operation assembly line consists of multiple large flat plates. The front and rear tires of the vehicle body and the side limit devices of the tires are installed on the large flat plates. After the vehicle body is placed in the limit position in sequence, the large flat plates move forward in sequence. When it runs to the visual positioning station, the bottom limit device of the large flat plate triggers the limit signal. Therefore, the vehicle body in-place judgment process first judges whether the limit signal is received. If the limit signal is received, it controls the laser sensor at the visual positioning station to continuously obtain the height value from the vehicle roof. If the height value remains stable within the set time, it is considered that the vehicle body has arrived and is stable, and then the single-station real-time positioning process is executed. Otherwise, it continues to obtain the height value of the laser sensor and judges whether the height value remains stable within the set time.

[0215] The system sequentially executes the single-station real-time positioning process for a total of 6 stations on the left and right sides of the top, the front side, and the rear side of the vehicle body. Among them, the left and right stations on the vehicle roof execute the single-station real-time positioning process first because the front-back deviation angle and the left-right deviation angle of the top relative to the camera plane are the smallest, and the rotation angle of the top in the camera plane can be accurately measured visually. The three-dimensional angle offset of the vehicle body relative to the template during production is the most accurate in this way. After completing the single-station real-time positioning process for both sides of the vehicle roof, the three angles of the vehicle body relative to the template vehicle body in the three-dimensional direction can be obtained. Among them, the left-right deviation angle and the rotation angle in the camera plane can be used as the rotation angles required in the single-station real-time positioning process for the side. A total of 7 laser sensors are installed in this system. Among them, 3 laser sensors are installed on the vehicle roof to obtain the front-back deviation angle and the left-right deviation angle of the vehicle body camera plane. At the same time, the left and right laser sensors under the vehicle roof camera are also used to obtain the actual height of the vehicle body. One laser sensor is installed at the front and rear stations on the left and right sides of the vehicle body to obtain the distance of the vehicle body. Therefore, a total of 7 laser sensors are installed in this system.

[0216] Judge whether all stations have completed punching. If so, control the punching robot to return to the standby point and wait for the arrival of the next vehicle body. Otherwise, continue the real-time positioning of the vehicle body at the next station.

[0217] The surface of the detected vehicle is not a completely flat plane, and there is an arc in the sheet metal of the vehicle surface. If the position of the sensor remains unchanged, when making the template, there will be a deviation between the distance information collected by the laser sensor and the actual measurement by the laser sensor. The vehicle angle offset obtained through the distance is not the correct angle. Therefore, it is set that the camera is fixed and the laser sensor is movable.

[0218] After the execution of the laser sensor approaching process is completed, the front-back deviation angle and the left-right deviation angle of the vehicle roof can be calculated according to the laser sensor. The solution method is illustrated as follows: See Figure 8As shown in the figure, if circle 1 is the initial position, the distances measured at the left rear and right rear are HG and AC respectively, and the distance between the two sensors is AH. Circle 2 is the position for re-parking. If the sensors do not move along, the measured positions are HD and AB. In actual use, the sensors will approach the position according to the visual positioning situation. Therefore, only the distances after the sensors move along need to be calculated. The new positions of the sensors after moving along are I and J, and the measured positions are IM and JK. As shown in the figure, the angle of the vehicle does not change significantly. When making the template, the sensors do not move. At this time, the deflection angle of the vehicle body in a certain direction can be calculated through the distance relationship between the sensors; while in real-time detection, after the sensors move through the approximation method, the deflection angle of the vehicle body in a certain direction can also be calculated through the distance relationship between the sensors. The calculation formula is as follows:

[0219] θ is the included angle between AH and GC, which is the vehicle body angle when making the template

[0220] θ′ is the included angle between AH and MK, which is the vehicle body angle during real-time positioning

[0221] Through the above formula, the inclination angles of the vehicle body in the specified direction during template making and real-time positioning can be calculated. According to the above definition, the α angle of the vehicle body is the left-right inclination angle of the vehicle body, and the β angle is the front-back inclination angle of the vehicle body. Then, when calculating the left-right inclination angle of the vehicle body, θ in this formula corresponds to the α angle of the vehicle body during template making, and θ′ corresponds to the α angle of the vehicle body during real-time positioning. Through (θ′ - θ), the deflection angle Δ of the vehicle body during real-time positioning can be obtained, specifically α Δ , β Δ and γ Δ .

[0222] Example: Assume that the station is the camera station on the right side of the vehicle roof. According to the above picture display, through the camera, the change in the characteristic position of the vehicle roof in the direction around the Y-axis (the left-right direction of the vehicle body) is K, and the control center controls the left-right adjustment device 4 of the top right sensor to move a displacement of K value.

[0223] Assume that the station is the camera station on the right side of the vehicle roof. According to the above picture display, through the camera, the change in the characteristic position of the vehicle roof in the direction around the X-axis (the front-back direction of the vehicle body) is L, and the control center controls the left-right adjustment device 4 of the top right sensor to move a displacement of L value. The same applies to other stations.

[0224] Six fixed cameras respectively collect the feature point information, and seven distance measurement sensors measure the distances.

[0225] It is known that the top sensors are at the same height of 12, and the distance between 6 and 6 is H1

[0226] The top sensors are at the same height of 6, and the distance between 6 and 15 is H2

[0227] The right sensor is at the same height of 30, and the distance between 24 is H3

[0228] The left sensor is at the same height of 36, and the distance between 42 is H4

[0229] Set the measured value of the sensor template to: H 12 ,H 6 ,H 15 ,H 30 ,H 24 ,H 36 ,H 42

[0230] Set the measured value of the sensor during template making to: h 12 ,h 6 ,h 15 ,h 30 ,h 24 ,h 36 ,h 42

[0231] The measured value of the sensor after correcting the position during real-time measurement is: h 12-1 ,h 6-1 ,h 15-1 ,h 30-1 ,h 24-1 ,h 36-1 ,h 42-1 When the vehicle body is in place, start making the template. The distance values measured by 7 sensors are fed back to the control center to calculate the magnification ratio.

[0232] The camera takes pictures to record the feature point information.

[0233] The vehicle body is in place again and starts real-time measurement;

[0234] The top sensor measures the distance value h 12-1 ,h 6-1 , calculate the magnification ratio and send it to the control center.

[0235] The two top cameras calculate the offset angle in the Z-axis direction and the offset in the XY coordinate system plane and send the data to the control center.

[0236] The control center adjusts the sensor position according to the camera parameters.

[0237] Rely on the three top sensors to calculate the offset angles of the vehicle body in the X-axis and Y-axis directions. Thus, the rotation angles of the vehicle body in the XYZ three coordinate axes directions are obtained.

[0238] Among them, the angle calculation formula during template making is as follows:

[0239] Rotation angle around the X-axis: α 0 =tan -1 (h 15 -h6 ) / H 2

[0240] Rotation angle in the Y-axis direction: β 0 = tan -1 (h 12 -h 6 ) / H 1

[0241] The angle calculation formula during real-time positioning is as follows:

[0242] Rotation angle in the X-axis direction: α 1 = tan -1 (h 15-1 -h 6-1 ) / H 2

[0243] Rotation angle in the Y-axis direction: β 1 = tan -1 (h 12-1 -h 6-1 ) / H 1

[0244] By calculating (β 1 -β 0 ), β can be obtained. Δ , and by calculating (α 1 -α 0 ), α can be obtained. Δ .

[0245] The corrected value of the sensor position is transmitted to the control center and used as the magnification correction value for the camera.

[0246] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, those of ordinary skill in the art should understand that various forms and details can be changed without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A detection method for the relevant surface of the object to be measured, characterized in that, first, the vehicle body is transported to the assembly line flat plate, and the vehicle body is successively transported to the flat plate assembly line with a limiting device by a crane. When the vehicle body runs to the visual detection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the stop signal is obtained by the vision system, and then the user operation selection is carried out. The vision system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are respectively the distortion calibration process, the N-point calibration process, the magnification acquisition process and the template production process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template production, the position of the laser sensor is adjusted to make it approach the position during template production, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding detection components at each station to obtain and process the image and the laser sensor measurement values. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top detection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera surface. Then, the deflection angle of the vehicle body on the camera surface is calculated by obtaining the features of the image by the camera and the image features during template production. The corresponding deflection angle is used for processing during the distortion correction and image coordinate conversion at each station; through the images obtained by the cameras of the detection components at each station, the physical position of the feature points is further calculated. According to the feature information of the images during template production and real-time positioning and the obtained three-dimensional deflection angle information of the vehicle body, the position information and punching posture of the punching points are accurately calculated, and then accurate positioning and punching are realized.

2. The detection method for the relevant surface of the object to be measured according to claim 1, characterized in that, The single-station manual operation process includes distortion calibration, N-point calibration of the punching robot, magnification acquisition, template production and real-time positioning operation. The specific execution process is as follows: If the user selects "distortion calibration", the camera distortion calibration process will be executed. By executing the camera distortion calibration process, the distortion calibration file at different angles of the feature surface will be obtained. This file can be used for image distortion correction when the feature surface and the camera surface are at different angles; If the user selects "N-point calibration of the punching robot", the N-point calibration process of the punching robot will be executed. By executing the N-point calibration process of the punching robot, the N-point calibration file will be obtained, and the image coordinates will be converted into physical coordinates in the coordinate system of the punching robot; If the user selects "magnification acquisition", the magnification acquisition process will be executed. By executing the magnification acquisition process, the first-order polynomial relationship function between the height of the feature surface and the image magnification will be obtained. According to this function, the image coordinate conversion at different heights of the feature surface will be realized, so as to obtain accurate physical coordinates of the feature points; If the user selects "template making", the template making process will be executed. By executing the template making process, the position information of the image processing feature points obtained by the camera and the position information of the punching points on the punching surface are obtained, and based on this, the position relationship between the feature points and the punching points is obtained. If the user selects "real-time positioning", the single-station real-time positioning process will be executed. Through the single-station real-time positioning process, the position information of the punching points will be obtained by performing real-time positioning on the currently selected station, and then the punching robot will be controlled to perform the punching operation.

3. The method for detecting the relevant surface of the object to be measured according to claim 2, characterized in that, The distortion calibration process includes the following steps: Control the checkerboard calibration board to the initial position, place the calibration board near the detection height of the detection feature points on the object to be measured, that is, on the vehicle. After placement, control the laser sensor to collect height information at each position; Calculate the offset angle between the calibration board and the camera plane according to the obtained height information of the laser sensor; If the angle meets the requirements, trigger the camera to take pictures and collect images; Perform distortion calibration on the collected images to generate a distortion calibration file; After generating the distortion calibration file, load the distortion calibration file for the images collected by the camera to perform distortion correction.

4. The method for detecting the relevant surface of the object to be measured according to claim 3, characterized in that, Calculating the offset angle between the calibration board and the camera plane according to the obtained height information of the laser sensor further includes: 3 laser sensors installed during the distortion calibration process, with one laser sensor installed on each of the left rear and right rear sides of the calibration board, that is, the top left distance measurement sensor and the top right distance measurement sensor, and one laser sensor installed on the right front, that is, the top front distance measurement sensor. The distance between the top left distance measurement sensor and the top right distance measurement sensor is denoted as H1, the distance between the top right distance measurement sensor and the top front distance measurement sensor is denoted as H2, the height obtained by the top left distance measurement sensor is denoted as h1, the height obtained by the top right distance measurement sensor is denoted as h2, and the height obtained by the top front distance measurement sensor is denoted as h3. The offset angles that can be obtained from the three heights include the left-right deviation angle and the front-back deviation angle with respect to the camera plane, where the left-right deviation angle is denoted as θ1 and the front-back deviation angle is denoted as θ2. The calculation formulas are as follows: Judge whether the obtained θ1 and θ2 are stable within the set threshold range. If so, it is considered that the checkerboard calibration board is in a stable and stationary state; otherwise, it is considered that the checkerboard calibration board is still in a moving state. At this time, judge whether the number of angle measurements exceeds the set value. If so, end this distortion calibration operation and prompt "The calibration board is not stable. Please re-place the calibration board"; otherwise, continue to measure θ1 and θ2. Within the range of ±10°, the distortion calibration is performed on the left-right deflection angle and the front-back deflection angle every 1°. When calibrating the left-right deflection angle, keep the front-back deflection angle at 0°. When the left-right deflection angle is between -10° and 10°, control the angle of the checkerboard calibration board to shift 1° each time for distortion calibration. The same is true for the front-back deflection angle calibration process. According to the current calibration process, judge whether the currently measured angle is within the required range. For example, when performing distortion calibration at the left-right deflection angle of -10°, judge whether θ1 is within the threshold range near -10° set by the system, and at the same time ensure that θ2 is 0°. If the conditions are met, it is considered that the angle is within the required range and proceed to the next step; otherwise, control the movement of the checkerboard calibration board to adjust the angle of the checkerboard calibration board so that the calibration board angle meets the above angle requirements.

5. The detection method of the relevant surface of the object to be measured as described in claim 2, characterized in that, the magnification acquisition includes the magnification of ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and a fitting function of magnification and distance is obtained by fitting.

6. The detection method of the relevant surface of the object to be measured as described in claim 5, characterized in that, the magnification of ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and a fitting function of magnification and distance is obtained by fitting, including: the specific measurement process is from h - 10 cm to h + 10 cm, and the measurement is performed every 5 mm. First, control the calibration board to the position of h - 10 cm, then obtain the distance from the upper laser sensor to the calibration board, and use the angle calculation method in the distortion calibration process to obtain the left-right deflection angle θ1' and the front-back deflection angle θ2' of the calibration board; judge whether θ1' and θ2' are stable. If not, it means that the calibration board has not come to a complete stop, then judge whether the number of measurements exceeds the set value. If so, end the current magnification acquisition and wait for the user to operate; otherwise, continue to obtain the distance from the laser sensor to the calibration board and calculate the angle; if θ1' and θ2' are stable, then judge whether θ1' and θ2' are within the required range. When the camera and the measured surface are completely perpendicular, after the collected image is corrected for distortion to remove the distortion effect of the lens itself, there is no distortion at each position of the image. Therefore, in order to remove the influence of the distortion introduced by the angle on the magnification, it is required that θ1' and θ2' be around 0°. If the conditions are met, trigger the camera to take a picture and save the image; if θ1' and θ2' are not within the required range, adjust the angle of the calibration board and then obtain θ1' and θ2' again; judge whether the images at each of the above height positions have been collected. If so, process the images corresponding to all heights to obtain the pixel pitch of the checkerboard at the same position of the calibration board. The checkerboard position is selected as the checkerboard in the middle of the image because the distortion at the middle position of the image is the smallest and the pixel pitch error obtained is the smallest. Then, according to the actual physical size of the checkerboard, the proportional relationship between the pixel pitch and the physical size is the image magnification of the calculated height. If the images at each height position have not been collected yet, adjust the height of the calibration board to collect the image at the next height; After obtaining the magnification ratios at various heights, a first-order polynomial fitting is performed according to the corresponding heights to obtain the fitting coefficients k and b, that is, m = k * h4 + b, where m is the magnification ratio and h4 is the distance value measured by the laser sensor. The fitting coefficients k and b are obtained and stored.

7. The method for detecting the related surface of the object to be measured according to claim 2, characterized in that the template making includes: the system determines whether the tool center point TCP of the punching robot has been switched, that is, the system switches the TCP position of the punching robot from the marking head TCP to the punching head TCP. After the switching is completed, the punching robot will perform position movement and attitude adjustment in the workpiece coordinate system of the punching robot with the punching head TCP as the standard; if not, the system first controls the punching robot to switch the tool TCP. After the switching is completed, it controls each laser sensor to obtain the distance from the vehicle body; if so, each laser sensor obtains the distance from the vehicle body; the system determines whether the distances obtained by each laser sensor are stable, that is, each distance value is less than the set threshold; if not, the system counts the number of measurements and determines whether this number exceeds the set value. If it does not exceed, the system continues to determine whether the distances obtained by each laser sensor are stable. If it exceeds, the system prompts that the vehicle body is not placed stably, and returns to determine whether the start button for making the template is clicked; if so, the deviation angles in all directions of the vehicle body are calculated according to the distances from the vehicle body obtained by each laser sensor; it is determined whether the obtained deviation angles in all directions are within the set range; if not, it is prompted that the position of the vehicle body is abnormal, and the user is required to re-place the position of the vehicle body, and return to determine whether the start button for making the template is clicked and execute; if so, the vehicle body feature obtaining process is executed; the system determines whether there is an abnormality in obtaining the vehicle body features; if not, the system controls the punching robot to make its punching head run to the punching position; the system then respectively records the intersection coordinates of the feature points, the straight line angles, and the position coordinates of the punching points; saves the corresponding laser sensor position data, and returns to determine whether the start button for making the template is clicked and execute; if so, according to the prompt of the abnormality type, the user is asked to confirm the position of the vehicle body, and the template making process ends.

8. The method for detecting the related surface of the object to be measured according to claim 7, characterized in that the vehicle body feature obtaining process in the template making includes the following steps: the system controls the camera to collect the images of the vehicle body feature surface; according to the vehicle body deviation angle, the system calls the corresponding distortion calibration file to correct the image distortion; according to the vehicle body sheet metal feature, the system performs feature template matching at the sheet metal indirect seam; the system determines whether the feature matching is successful. If not, the system prompts that the vehicle body feature is not found, and asks to confirm the position of the vehicle body, and this process ends; if so, the position of the found feature is corrected, and then the two straight lines at the seam are searched respectively; the system determines whether the two straight lines are successfully searched. If not, it is prompted that the seam of the vehicle body feature is not found, and asks to confirm the position of the vehicle body, and this process ends; if so, the intersection of the two found straight lines is obtained, and the coordinates of the intersection are converted to physical coordinates.

9. The method for detecting the related surface of the object to be measured according to claim 2, characterized in that the real-time positioning punching process includes the following steps: perform the vehicle body angle obtaining process; The system determines whether the acquisition of the vehicle body angle is successful; if not, it prompts a positioning abnormality and returns to continue determining whether the start real-time positioning button is clicked. If so, it executes the vehicle body feature acquisition process; The system determines whether the acquisition of vehicle body features is successful; if not, it prompts that no vehicle body features are found or prompts that no seams of the vehicle body feature location are found, and asks the user to confirm the vehicle body position, then returns to continue determining whether the start real-time positioning button is clicked; if so, based on the feature points, punching points, deflection angles in all directions of the vehicle body in the template making, and the coordinate of the feature points obtained by real-time positioning, the position of the new punching point is calculated through computation; The system determines whether the offset position of the vehicle body is within the set range. If not, it prompts that the vehicle body position is abnormal, and requires the user to re-place the vehicle body position, then returns to continue determining whether the start real-time positioning button is clicked; if so, after controlling the punching robot to adjust its posture, it runs above the new punching point, and the laser sensor acquires the distance from the new punching point in real time; It determines whether the distance reaches the set range; if not, the punching robot approaches the new punching point by a certain distance and then determines whether the distance reaches the set range again; if so, the system controls the punching robot to punch holes. After that, it returns to continue determining whether the start real-time positioning button is clicked.

10. The method for detecting the relevant surface of the object to be measured as described in claim 9, characterized in that, in the full-station automatic operation process, the single-station real-time positioning process is sequentially executed for a total of 6 stations on the left and right sides of the top, front side and rear side of the vehicle body. Among them, the single-station real-time positioning process is first executed for the left and right stations on the roof. After completing the single-station real-time positioning process for both sides of the roof, three angles of the vehicle body relative to the vehicle body in the template making in the three-dimensional direction can be obtained. Among them, the left and right deflection angles and the rotation angle in the camera plane can be used as the rotation angles required in the single-station real-time positioning process for the side surface.

Images