Automobile Assembly Detection and Positioning System

Abstract

An embodiment of the present invention discloses a detection and positioning system for automobile assembly, including a top detection component, a limiting device, a left detection component, and a right detection component. First, the vehicle body is transported to the assembly line flat plate, and the vehicle body is successively transported to the flat plate assembly line with a limiting device through a crane. When the vehicle body runs to the visual detection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and this signal is acquired by the visual system, and then user operation selection is carried out. The visual system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are the distortion calibration process, the N-point calibration process, the magnification acquisition process, and the template making process. According to the characteristic information of the images during template making and real-time positioning and the three-dimensional deviation angle information of the vehicle body obtained, the position information and punching posture of the punching points are accurately calculated.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile production, and particularly relates to a detection and positioning system for automobile assembly. Background Art

[0002] An autonomous vehicle is an intelligent vehicle that realizes driverless operation through a computer system. Autonomous vehicles rely on the collaborative cooperation of artificial intelligence, visual computing, radar, monitoring devices, and the global positioning system, enabling the computer to automatically and safely operate a motor vehicle without the active operation of a human.

[0003] With the rise of autonomous vehicle technology, more and more enterprises have invested in the research and development of autonomous vehicles. Currently, technologies such as artificial intelligence, visual monitoring and computing, and radar have matured. By effectively integrating these technologies, an autonomous driving system can be realized. Installing the autonomous driving system into a manually driven vehicle for vehicle modification can convert a manual vehicle into an autonomous vehicle. After long-term testing, the autonomous driving system can achieve safe and reliable autonomous driving on the learned lanes.

[0004] The autonomous driving system includes many hardware components such as radar and visual systems. Currently, after manual punching at the designated installation positions on the vehicle body of a manually driven vehicle by an automobile modification factory, the above-mentioned hardware devices are then installed. As can be seen from the above, there are various problems such as low precision, low efficiency, and poor repeatability in the positioning and manual punching of modified or newly built autonomous vehicles. Summary of the Invention

[0005] In view of the above existing technical problems, the present invention provides a detection and positioning system for automobile assembly, which is used to achieve high-precision positioning and punching of the punching positions on the vehicle body of a modified autonomous vehicle, and to realize the automated and intelligent production of the entire modification line.

[0006] To solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An inspection and positioning system for vehicle assembly, characterized in that it includes a top inspection component, a limiting device, a left inspection component and a right inspection component. First, the vehicle body is transported to the assembly line flat plate, and the vehicle body is sequentially transported to the flat plate assembly line with a limiting device by a crane. When the vehicle body runs to the visual inspection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and this signal is acquired by the vision system, and then user operation selection is carried out. The vision system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are respectively the distortion calibration process, the N-point calibration process, the magnification acquisition process and the template making process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template making, the position of the laser sensor is adjusted to approach the position during template making, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding inspection components at each station to acquire and process the image and the measurement value of the laser sensor. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top inspection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera plane. Then, the deflection angle of the vehicle body on the camera plane is calculated by obtaining the features of the image through the camera and the image features during template making. The corresponding deflection angles are used for processing during the distortion correction and image coordinate conversion at each station; through the images acquired by the cameras of the inspection components at each station, the physical positions of the feature points are further calculated. According to the feature information of the images during template making and real-time positioning and the acquired three-dimensional deflection angle information of the vehicle body, the position information and punching posture of the punching points are accurately calculated, and then accurate positioning and punching are realized.

[0008] Preferably, the top inspection component includes a top right inspection camera, a first top right light source, a second top right light source, a top right sensor left-right adjustment device, a top right sensor front-back adjustment device, a top right distance measurement sensor, a top left inspection camera, a first top left light source, a second top left light source, a top left sensor left-right adjustment device, a top left sensor front-back adjustment device, a top left distance measurement sensor, a top front sensor left-right adjustment device, a top front sensor front-back adjustment device and a top front distance measurement sensor.

[0009] Preferably, the limiting device includes front-back limit for the right front wheel, left-right limit for the right front wheel, front-back limit for the left front wheel, left-right limit for the left front wheel, left-right limit for the right rear wheel, front-back limit for the right rear wheel, front-back limit for the left rear wheel and left-right limit for the left rear wheel.

[0010] Preferably, the right - hand detection component includes a right - front ranging sensor, a right - front sensor front - rear adjustment device, a right - front sensor up - down adjustment device, a right - front detection camera, a first right - front light source, a second right - front light source, a right - rear ranging sensor, a right - rear sensor front - rear adjustment device, a right - rear sensor up - down adjustment device, a right - rear detection camera, a first right - rear light source, and a second right - rear light source.

[0011] Preferably, the left - hand detection component includes a left - front ranging sensor, a left - front sensor front - rear adjustment device, a left - front sensor up - down adjustment device, a left - front detection camera, a first left - front light source, a second left - front light source, a left - rear ranging sensor, a left - rear sensor up - down adjustment device, a left - rear sensor front - rear adjustment device, a left - rear detection camera, a first left - rear light source, and a second left - rear light source.

[0012] Preferably, the camera distortion calibration process is used to obtain distortion calibration files at different angles of the feature plane, and the distortion calibration files are used for image distortion correction when the feature plane and the camera plane are at different angles.

[0013] Preferably, the N - point calibration process of the punching robot is used to obtain an N - point calibration file to realize the conversion of image coordinates to physical coordinates in the coordinate system of the punching robot.

[0014] Preferably, the magnification acquisition process is used to obtain a first - order polynomial relationship function between the height of the feature plane and the magnification. According to the relationship function, the conversion of image coordinates at different heights of the feature plane is realized, so as to obtain accurate physical coordinates of feature points.

[0015] Preferably, the template making process is used to obtain the position information of the image - processing feature points obtained by the camera and the position information of the punching points on the punching surface, and based on this, the position relationship between the feature points and the punching points is obtained.

[0016] Preferably, in the full - station automatic operation process, the single - station real - time positioning process is sequentially executed for a total of 6 stations on the left and right sides of the top, the front - side and the rear - side of the vehicle body. Among them, the single - station real - time positioning process is first executed for the left and right stations on the top of the vehicle body. After completing the single - station real - time positioning process for both sides of the top of the vehicle body, three angles of the vehicle body relative to the template - making vehicle body in the three - dimensional direction can be obtained. Among them, the left - right deviation angle and the rotation angle within the camera plane can be used as the rotation angles required in the side - single - station real - time positioning process.

[0017] Adopting the present invention has the following beneficial effects:

[0018] (1) In the system solution, machine vision technology is combined with high - precision sensors to determine the position relationship of positioning points on different curved surfaces, so as to realize high - precision positioning of the vehicle body's featureless curved surface and the punching operation of the punching robot.

[0019] Locate image features through machine vision technology to obtain the positions of feature points and the rotation angle of the feature plane. Obtain the front-back and left-right deflection angles of the vehicle body feature plane through multiple high-precision sensors, so as to obtain the rotation angle of the vehicle body feature plane in three-dimensional directions.

[0020] (2) Positioning of the featureless curved surface and punching by the punching robot: Through the visual positioning of the feature plane and the rotation angle of the vehicle body feature plane in three-dimensional directions, combined with the relative position relationship between the feature points and the punching points in the feature plane, obtain the position and punching posture of the punching points in real-time positioning detection, and finally control the punching robot to perform the punching operation.

[0021] (3) In real-time detection, the approximation method of the laser sensor to the camera plane position during template making.

[0022] Coarse measurement by the laser sensor and rough visual positioning. In real-time positioning detection, obtain the height of the measured feature plane through the laser sensor, and obtain the rotation angle and position offset of the feature points relative to the feature points during template making through the vision system. According to the rough visual positioning result, control the laser sensor to adjust the measurement position on the camera plane so that it runs to the measurement position during template making, and then perform fine measurement of the laser sensor. Through the fine measurement result of the laser sensor (obtain the accurate height value), perform fine positioning of the feature information of the feature plane by vision.

[0023] (4) High-precision positioning method for the vehicle body curved surface during three-dimensional rotation and translation of the plane.

[0024] Obtain the accurate height value of the measured feature plane through the laser sensor. Use the corresponding image distortion correction file for different heights to correct the image distortion. Obtain the fitting function of height and magnification through magnification fitting at different heights, and calculate the magnification according to the height during real-time detection and positioning. Through the magnification, convert the pixel coordinates of the image feature points on the measured surface during real-time detection into the coordinates under the standard of the pixel coordinates of the template-making image. Obtain the N-point calibration file through the N-point calibration of the punching robot and the camera. After coordinate conversion of the collected vehicle body feature images through N-point calibration, the translation amount of the feature points in the physical position can be obtained. Obtain the rotation angle of the three-dimensional plane, and calculate the position of the punching points on the vehicle body curved surface according to the angle and the translation amount of the feature points.

[0025] (5) Realize multi-station non-manual intervention positioning and punching for multiple different surfaces of the vehicle body.

[0026] First, perform the positioning of the characteristics on the top surface of the vehicle body. Obtain the rotation angle of the vehicle body characteristics through the image of the top surface of the vehicle body, obtain the left - right rotation angle of the characteristic surface in the three - dimensional space through the laser sensors at the left and right stations on the roof, and obtain the front - rear rotation angle of the vehicle body through the laser sensors at the front and rear stations on the roof. Thus, the rotation angles of the vehicle body in three directions in the three - dimensional space can be obtained, and further the positioning of the featureless punching points corresponding to the top surface can be realized. Through the three - dimensional rotation angles, perform the positioning of the side characteristic surface, and then obtain the position and punching posture of the featureless punching points on the side. Brief Description of the Drawings

[0027] Figure 1 Schematic diagram of the structures of the top - detection component and the limit component of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0028] Figure 2 Schematic diagram of the structure of the right - hand detection component of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0029] Figure 3 Schematic diagram of the structure of the left - hand detection component of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0030] Figure 4 General flowchart of the execution process of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0031] Figure 5 Single - station manual operation flowchart of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0032] Figure 6 Full - station automatic operation flowchart of the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0033] Figure 7 Schematic diagram of the deflection angle in the detection and positioning system for vehicle assembly according to an embodiment of the present invention;

[0034] Figure 8 Schematic diagram of the workpiece - surface coordinate system established by the punching robot according to an embodiment of the present invention. Detailed Embodiment

[0035] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0036] A detection and positioning system for automobile assembly according to an embodiment of the present invention includes a top detection component, a limiting device, a left detection component, and a right detection component. First, the vehicle body is transported to the assembly line flat plate, and the vehicle body is successively transported to the flat plate assembly line with a limiting device by a crane, ensuring that the position deviation of the vehicle body transported to the flat plate each time is within ±5 cm, and the angle deviation is within ±5°. When the vehicle body runs to the visual detection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the stop signal is acquired by the visual system, and then the user operation selection is carried out. The visual system performs single-station operation or full-station operation according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are respectively the distortion calibration process, the N-point calibration process, the magnification acquisition process, and the template production process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template production, the position of the laser sensor is adjusted to approach the position during template production, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding detection components at each station to acquire and process the image and the measurement value of the laser sensor. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top detection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera plane. Then, calculate the deflection angle of the vehicle body on the camera plane by obtaining the features of the image by the camera and the image features during template production. The corresponding deflection angle is used for processing during the distortion correction and image coordinate conversion at each station; through the images acquired by the cameras of the detection components at each station, further calculate the physical position of the feature points. According to the feature information of the images during template production and real-time positioning and the acquired three-dimensional deflection angle information of the vehicle body, accurately calculate the position information and punching posture of the punching points, and then realize accurate positioning and punching.

[0037] Further, in an embodiment of the present invention, refer to Figures 1 to 3, the top detection component includes a top right detection camera 1, a first top right light source 2, a second top right light source 3, a top right sensor left - right adjustment device 4, a top right sensor front - back adjustment device 5, a top right distance measurement sensor 6, a top left detection camera 7, a first top left light source 8, a second top left light source 9, a top left sensor left - right adjustment device 10, a top left sensor front - back adjustment device 11, a top left distance measurement sensor 12, a top front sensor left - right adjustment device 13, a top front sensor front - back adjustment device 14, and a top front distance measurement sensor 15. The top left detection camera 7 is arranged above the left roof position of the vehicle's rear row seat, with the camera facing vertically downwards. The top left detection camera 7 is approximately located above the roof position, and the effective measurement height is about 100 cm; the first top left light source 8 and the second top left light source 9 are relatively perpendicular. Among them, the direction of the first top left light source 8 is parallel to the vehicle's front - back direction, and the second top left light source 9 is parallel to the vehicle's left - right direction. The illumination directions of the first top left light source 8 and the second top left light source 9 point to the vicinity of the camera's field - of - view center, and the height of the light source is approximately 30 cm from the roof; the top left distance measurement sensor 12 is at a height of 20 cm from the roof. The top left sensor left - right adjustment device 10 controls the top left sensor front - back adjustment device 11 to achieve left - right movement, and the top left sensor front - back adjustment device controls the top left distance measurement sensor 12 to achieve front - back movement. It is equivalent that the top left distance measurement sensor 12 can perform front - back and left - right movements on the roof plane. The overall sensor movement adjustment mechanism is located near the top of the vehicle's rear end on the left - hand side visual system. The top right detection camera 1 is approximately located above the right roof position of the vehicle's rear row seat, with the effective measurement height being about 100 cm, and the camera faces vertically downwards. The first top right light source 2 and the second top right light source 3 are relatively perpendicular. Among them, the direction of the first top right light source 3 is parallel to the vehicle's front - back direction, and the second top right light source 2 is parallel to the vehicle's left - right direction. Both are installed parallel to the roof, and the illumination direction points to the vicinity of the camera's field - of - view center, and the height of the light source is approximately 30 cm from the roof. The top left distance measurement sensor 12 is at a height of 20 cm from the roof. The top right sensor left - right adjustment device 4 controls the top right sensor front - back adjustment device 5 to achieve left - right movement, and the top right sensor front - back adjustment device controls the top right distance measurement sensor 6 to achieve front - back movement. It is equivalent that the top right distance measurement sensor 6 can perform front - back and left - right movements on the roof plane. The overall sensor movement adjustment mechanism is located near the top of the vehicle's rear end on the right - hand side visual system. The top front distance measurement sensor 15 is at a height of 20 cm from the roof. The top front sensor left - right adjustment device 13 controls the top front sensor front - back adjustment device 14 to achieve left - right movement, and the top front sensor front - back adjustment device 14 controls the top front distance measurement sensor 15 to achieve front - back movement. It is equivalent that the top front distance measurement sensor 15 can perform front - back and left - right movements on the roof plane. The overall sensor movement adjustment mechanism is located near the right roof of the vehicle's front row.

[0038] The limiting device includes front - rear limit 16 for the right front wheel, left - right limit 17 for the right front wheel, front - rear limit 18 for the left front wheel, left - right limit 19 for the left front wheel, left - right limit 20 for the right rear wheel, front - rear limit 21 for the right rear wheel, front - rear limit 22 for the left rear wheel, and left - right limit 23 for the left rear wheel. Among them, the front - rear limit 16 for the right front wheel and the front - rear limit 18 for the left front wheel are respectively located at the front parts of the left and right tires at the front of the vehicle, restricting the vehicle from moving forward. The front - rear limit 21 for the right rear wheel and the front - rear limit 22 for the left rear wheel are respectively located at the rear parts of the rear tires of the vehicle, restricting the vehicle from moving backward. The left - right limit 17 for the right front wheel and the left - right limit 20 for the right rear wheel are respectively located on the side surfaces of the front and rear tires on the right side of the vehicle, restricting the vehicle from moving to the right. The left - right limit 19 for the left front wheel and the left - right limit 23 for the left rear wheel are respectively located on the side surfaces of the front and rear tires on the left side of the vehicle, restricting the vehicle from moving to the left.

[0039] The right detection component includes a right front ranging sensor 24, a right front sensor front-back adjustment device 25, a right front sensor up-down adjustment device 26, a right front detection camera 27, a first right front light source 28, a second right front light source 29, a right rear ranging sensor 30, a right rear sensor front-back adjustment device 31, a right rear sensor up-down adjustment device 32, a right rear detection camera 33, a first right rear light source 34, and a second right rear light source 35. The right front detection camera 27 is approximately located on the right side of the connection position between the right front door of the vehicle and the vehicle body. The effective measurement distance is about 100 cm, the height from the ground is greater than 80 cm, and the camera surface is parallel to the front-back direction of the vehicle and faces left. The first right front light source 28 and the second right front light source 29 are relatively perpendicular. Among them, the direction of the first right front light source 28 is parallel to the front-back direction of the vehicle, and the second right front light source 29 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the lighting direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the right side of the vehicle. The right front ranging sensor 24 is 20 cm away from the right front sheet metal of the vehicle. The right front sensor up-down adjustment device 26 controls the right front sensor front-back adjustment device 25 to achieve up-down movement, and the right front sensor front-back adjustment device 25 controls the right front ranging sensor 24 to achieve front-back movement. It is equivalent to that the right front ranging sensor 24 can perform front-back and up-down movements on the right side plane of the vehicle. The overall sensor movement adjustment mechanism is located near the front of the vehicle in the right front vision system. The right rear detection camera 33 is approximately located on the right side of the connection position between the right rear fender and the right rear bumper of the vehicle. The effective measurement distance is about 100 cm, the height from the ground is greater than 80 cm, and the camera surface is parallel to the front-back direction of the vehicle and faces left. The first right rear light source 34 and the second right rear light source 35 are relatively perpendicular. Among them, the direction of the first right rear light source 34 is parallel to the front-back direction of the vehicle, and the second right rear light source 35 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the lighting direction points to the vicinity of the center of the camera's field of view. The height of the light source is approximately 30 cm from the right side of the vehicle. The right rear ranging sensor 30 is 20 cm away from the right front sheet metal of the vehicle. The right rear sensor up-down adjustment device 32 controls the right rear sensor front-back adjustment device 31 to achieve up-down movement, and the right rear sensor front-back adjustment device 31 controls the right rear ranging sensor 30 to achieve front-back movement. It is equivalent to that the right rear ranging sensor 30 can perform front-back and up-down movements on the right side plane of the vehicle. The overall sensor movement adjustment mechanism is located away from the rear of the vehicle in the right rear vision system.

[0040] The left detection component includes a left front distance measuring sensor 36, a left front sensor vertical adjustment device 37, a left front sensor horizontal adjustment device 38, a left front detection camera 39, a first left front light source 40, a second left front light source 41, a left rear distance measuring sensor 42, a left rear sensor vertical adjustment device 43, a left rear sensor horizontal adjustment device 45, a left rear detection camera 44, a first left rear light source 46, and a second left rear light source 47. The left front detection camera 39 is roughly located on the left side of the connection position between the left front door of the vehicle and the vehicle body. The effective measurement distance is about 100 cm, the height from the ground is greater than 80 cm, and the camera surface is parallel to the front-rear direction of the vehicle and faces right. The first left front light source 40 and the second left front light source 41 are perpendicular to each other. Among them, the direction of the first left front light source 40 is parallel to the front-rear direction of the vehicle, and the second left front light source 41 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the illumination direction points to the vicinity of the center of the camera's field of view. The height of the light source is about 30 cm from the left side of the vehicle. The left front distance measuring sensor 36 is 20 cm away from the right front sheet metal of the vehicle. The left front sensor vertical adjustment device 38 controls the left front sensor horizontal adjustment device 37 to achieve vertical movement, and the left front sensor horizontal adjustment device 37 controls the left front distance measuring sensor 36 to achieve horizontal movement. It is equivalent that the left front distance measuring sensor 36 can move horizontally and vertically on the left side plane of the vehicle. The overall sensor movement adjustment mechanism is located near the front of the vehicle in the left front vision system. The left rear detection camera 44 is roughly located on the left side of the connection position between the left rear fender and the left rear bumper of the vehicle. The effective measurement distance is about 100 cm, the height from the ground is greater than 80 cm, and the camera surface is parallel to the front-rear direction of the vehicle and faces right. The first left rear light source 46 and the second left rear light source 47 are perpendicular to each other. Among them, the direction of the first left rear light source 46 is parallel to the front-rear direction of the vehicle, and the second left rear light source 47 is parallel to the up-down direction of the vehicle. Both are installed parallel to the side of the vehicle body, and the illumination direction points to the vicinity of the center of the camera's field of view. The height of the light source is about 30 cm from the right side of the vehicle. The left rear distance measuring sensor 42 is 20 cm away from the right front sheet metal of the vehicle. The left rear sensor vertical adjustment device 43 controls the left rear sensor horizontal adjustment device 45 to achieve vertical movement, and the left rear sensor horizontal adjustment device 45 controls the left rear distance measuring sensor 42 to achieve horizontal movement. It is equivalent that the left rear distance measuring sensor 42 can move horizontally and vertically on the left side plane of the vehicle. The overall sensor movement adjustment mechanism is located away from the rear of the vehicle in the left rear vision system.

[0041] The above-mentioned detection and positioning system for vehicle assembly operates using the following method. The positioning function of the relevant surface is achieved through a relevant surface positioning and punching system based on machine vision. When the object to be detected runs to the positioning system, the positioning and punching at the relevant surface positioning are achieved through the position relationship between the laser sensor positioning method, the machine vision positioning method, and the relevant surface. Refer to Figure 4 , including the following steps:

[0042] After the positioning system is powered on and starts up, it first executes the system startup initialization process, including: detecting the hardware status, including the connection status detection of the punching robots on both sides, cameras at each station, and laser sensors; if all hardware is normally connected, proceed to the next step, otherwise issue an alarm for abnormal connection of relevant hardware and prompt the user to repair the relevant hardware; confirm the position of the punching robot, if the position of the punching robot is not at the zero position, perform the zeroing operation on it according to the zeroing trajectory; read the relevant detection parameters during the system operation;

[0043] After the system startup initialization is completed, the positioning system executes different processes according to different operations of the user:

[0044] If the user selects single-station operation, the single-station manual operation process is executed; the single-station manual operation process can complete the preprocessing operation before real-time positioning of the specified station and the positioning punching of a single station, mainly including: station selection process, camera distortion calibration process, punching robot N-point calibration process, magnification acquisition process, template production process, and real-time positioning process.

[0045] If the user selects full-station operation, the positioning system will execute the full-station automatic operation process; the full-station automatic operation process completes the positioning punching operations of all stations to be punched in the set order. After completing the positioning punching operations of all stations, the punching robot is run to the zero point according to the zeroing trajectory and waits for the arrival of the next object to be detected. During the execution of the full-station automatic operation process, if an operation that requires an emergency pause occurs, through the emergency pause, the current punching operation is stopped and the punching robot is zeroed.

[0046] In an embodiment of the present invention, before the system performs the full-station automatic operation, single-station manual operation needs to be performed. Through the single-station manual operation, the image distortion correction file, the fitting function of magnification and height, the coordinate conversion file between the image and the punching robot, and the position relationship between the image feature plane and the punching point on the surface to be positioned and punched of each station are obtained. Through the obtained files and data, the automatic positioning punching operations of each station are realized. Refer to Figure 5 , which shows the single-station manual operation flow chart. The single-station manual operation interface provides operation buttons for distortion calibration, punching robot N-point calibration, magnification acquisition, template production, and real-time positioning. Corresponding processes can be executed according to the relevant operations of the user. The specific execution process is as follows:

[0047] The single-station operation process first executes the station selection process. Before entering the single-station manual operation interface, the user needs to first select the station to be operated. After completion, enter the manual operation interface.

[0048] If the user clicks the "Distortion Calibration" button, the camera distortion calibration process will be executed. By executing the camera distortion calibration process, distortion calibration files at different angles of the feature plane will be obtained, and these files can be used for image distortion correction when the feature plane and the camera plane are at different angles.

[0049] If the user clicks the "Punching Robot N-Point Calibration" button, the punching robot N-point calibration process will be executed. By executing the punching robot N-point calibration process, an N-point calibration file is obtained, and the conversion of image coordinates to physical coordinates in the punching robot coordinate system is realized.

[0050] If the user clicks the "Magnification Acquisition" button, the magnification acquisition process will be executed. By executing the magnification acquisition process, a first-order polynomial relationship function between the height of the feature plane and the image magnification is obtained. According to this function, the conversion of image coordinates at different heights of the feature plane can be realized, so as to obtain accurate physical coordinates of the feature points.

[0051] If the user clicks the "Template Making" button, the template making process will be executed. By executing the template making process, the position information of the image processing feature points obtained by the camera and the position information of the punching points on the punching surface can be obtained, and the position relationship between the feature points and the punching points can be obtained accordingly.

[0052] If the user clicks the "Real-time Positioning" button, the single-station real-time positioning process will be executed. Through the single-station real-time positioning process, the position information of the punching points will be obtained by real-time positioning of the currently selected station, and then the punching robot will be controlled to perform punching operations. In the real-time positioning process, an emergency pause operation can be performed. After the user presses the "Emergency Pause" button, the emergency pause processing process will be executed.

[0053] When the system is in the single-station operation standby state, after the user presses the "Exit" button, the single-station operation can be exited.

[0054] In an embodiment of the present invention, cameras need to be installed at each station of the object to be detected for visual positioning. Since the feature plane to be detected and the camera plane will present different angles due to the different placement positions of the object to be detected, the camera distortion calibration process of this system will perform distortion calibration on each different angle of the feature plane and the camera plane. In the template making process, the corresponding distortion calibration file is called according to the measured angles of the feature plane and the camera plane for image distortion correction. Through the camera distortion calibration process, distortion calibration files with different deflection angles between the calibration plate and the camera plane can be generated. Each distortion calibration file consists of the perspective distortion and the radial distortion parameters of the image with the corresponding deflection angle, and these distortion files are used for image distortion correction. The camera distortion calibration process includes the following steps:

[0055] Judge whether the user clicks the "Distortion Calibration" button. If so, perform distortion calibration; otherwise, wait for the user to operate.

[0056] Control the checkerboard calibration target to the initial position. Place the calibration target near the detection feature point on the object to be measured, i.e., the vehicle, and after placement, control the laser sensor to collect height information at each position.

[0057] According to the height information obtained by the laser sensor, calculate the offset angle between the calibration target and the camera plane. During the distortion calibration process, 3 laser sensors are installed. One laser sensor is installed on each of the left rear and right rear sides of the calibration target, i.e., the top left distance sensor and the top right distance sensor, and one laser sensor is installed on the right front, i.e., the top front distance sensor. The distance between the top left distance sensor and the top right distance sensor is denoted as H1, the distance between the top right distance sensor and the top front distance sensor is denoted as H2, the height obtained by the top left distance sensor is denoted as h1, the height obtained by the top right distance sensor is denoted as h2, and the height obtained by the top front distance sensor is denoted as h3. The offset angles that can be obtained based on the three heights include the left-right deviation angle and the front-back deviation angle with respect to the camera plane, where the left-right deviation angle is denoted as θ1 and the front-back deviation angle is denoted as θ2. The calculation formulas are as follows:

[0058]

[0059] Judge whether the obtained θ1 and θ2 are stable within the set threshold range. If so, it is considered that the checkerboard calibration target is in a stable and stationary state; otherwise, it is considered that the checkerboard calibration target is still in a moving state. At this time, judge whether the number of angle measurements exceeds the set value. If so, end the current distortion calibration operation and prompt "The calibration target is not stable. Please re-place the calibration target"; otherwise, continue to measure θ1 and θ2.

[0060] Within the range of ±10°, perform distortion calibration on the left-right deviation angle and the front-back deviation angle at intervals of 1°. When calibrating the left-right deviation angle, keep the front-back deviation angle at 0°. Between the left-right deviation angle from -10° to 10°, each time control the checkerboard calibration target to deviate by 1° for distortion calibration. The same is true for the front-back deviation angle calibration process. According to the current calibration process, judge whether the currently measured angle is within the required range. For example, when performing distortion calibration at the left-right deviation angle of -10°, judge whether θ1 is within the threshold range near -10° set by the system, and at the same time, it is necessary to ensure that θ2 is 0°. If the conditions are met, it is considered that the angle is within the required range and proceed to the next step; otherwise, it is necessary to control the movement of the checkerboard calibration target to adjust the angle of the checkerboard calibration target so that the calibration target angle meets the above angle requirements.

[0061] If the angle meets the requirements, trigger the camera to take pictures and collect images.

[0062] Perform distortion calibration on the collected images to generate a distortion calibration file.

[0063] After generating the distortion calibration file, load the distortion calibration file for the images captured by the camera to perform distortion correction.

[0064] Obtain the horizontal pixel pitch and vertical pixel pitch of the checkerboards at the upper, lower, left, right, and middle positions of the checkerboard calibration plate respectively.

[0065] Judge whether the horizontal pixel pitch and vertical pixel pitch in the 5 directions are all within the threshold range near the standard value. If so, it means that the distortion calibration is successful, and the images at the current angle can be correctly distortion-corrected, and the generated distortion calibration file and the corresponding left-right deflection angle and front-back deflection angle of the calibration plate are stored; otherwise, it indicates that the distortion correction fails, and the distance from the laser sensor to the calibration plate needs to be re-obtained and the calibration plate angle is calculated, and then the distortion calibration at the current angle is performed again.

[0066] If the distortion calibration at the current angle is successful, judge whether the distortion calibration has been completed for all the above angles. If it has been completed, the distortion calibration is completed, and wait for the user to operate; otherwise, adjust the calibration plate angle and continue the distortion calibration for the next angle.

[0067] In an embodiment of the present invention, the vision system performs feature positioning to obtain the pixel coordinates of the feature points. The punching robot operates according to the coordinate system defined by the punching robot. It is necessary to let the punching robot locate the punching points according to the position of the feature points detected by vision. Therefore, it is necessary to convert the vision coordinates into the punching robot coordinates. After unifying the two coordinate systems, the punching robot can be controlled through the relative position offset detected by vision. This system generates an N-point calibration file through the N-point calibration process of the punching robot, and realizes the conversion between the vision coordinates and the punching robot coordinates through the N-point calibration file during real-time positioning. The N-point calibration process of the punching robot includes:

[0068] After entering the N-point calibration interface, wait for the user to perform the N-point calibration operation. If the user clicks the "N-point calibration" button, the N-point calibration is performed; otherwise, wait for the user to operate.

[0069] The punching robot makes a world coordinate system according to the body feature surface. The world coordinate system is used during the operation of the punching robot, and this coordinate system can be remade according to the actual working conditions. This system uses the roof camera surface of the vehicle body as the XY coordinate system, and the vertical direction is the Z direction of the coordinate system. Remaking the world coordinate system in this way can make the directions of the vision coordinate system and the punching robot XY coordinate system consistent.

[0070] The punching robot performs tool TCP (Tool Center Point) setting, sets the tool TCP position of the punching robot to the calibration marking head at point N, that is, sets the attitude calculation reference point of the punching robot to the marking head. The attitude calculation reference point of the punching robot is converted to the marking head, so that any change in the attitude of the punching robot will not affect the coordinates of the marking head, and the coordinates of the marking head are also the actual position of the punching robot during the calibration process of point N.

[0071] Obtain the distance from the upper laser sensor to the marking plate, and calculate the front-back deflection angle and left-right deflection angle of the marking plate.

[0072] Judge whether the angle is within the angle range of distortion correction. If so, control the punching robot to perform dot marking at point N within the camera's field of view; otherwise, prompt the user to adjust the calibration plate angle and wait for the user to operate.

[0073] After the punching robot completes the dot marking at point N, control the punching robot to exit the marking area and leave the field of view.

[0074] Control the camera to collect images, and respectively call the distortion calibration files at the corresponding angles according to the measured left-right deflection angle and front-back deflection angle between the marking plate and the camera plane to perform distortion correction.

[0075] Find the center pixel coordinates of the N marking points through the circle center finding algorithm.

[0076] Based on the center pixel coordinates of the N marking points and the corresponding physical coordinates of the punching robot, perform N-point calibration to obtain the N-point calibration file. This N-point calibration file is the transformation matrix between the camera vision coordinate system and the punching robot coordinate system in the XY two-dimensional plane. Through this transformation matrix, the pixel coordinate values of the camera can be converted into the physical coordinate values of the punching robot.

[0077] After obtaining the N-point calibration file, perform N-point coordinate transformation on the current image to verify whether the generated calibration file is correct.

[0078] Perform circle search on the marking points in the current image after distortion correction to find the center pixel coordinates of each of the N points. Call the above-generated N-point calibration file to convert the center pixel coordinates of the N points into the physical coordinates of the punching robot.

[0079] Compare the obtained physical coordinates of the N points with the physical coordinates of the N points recorded during the punching process of the punching robot to obtain statistical data. Judge whether the difference is within the set threshold range. If so, store the N-point calibration file and the corresponding height value measured by the laser sensor. Otherwise, prompt "Abnormal coordinate transformation after N-point calibration, please re-perform N-point calibration" and wait for the user to operate.

[0080] In an embodiment of the present invention, due to the physical characteristics of machine vision imaging, when the measured surface is at different heights, the measured object appears larger near and smaller far in the image. When performing the conversion of the physical coordinates of the feature points, there will be a difference in the distance of the feature surface relative to the laser sensor during the template making process and the real-time positioning and punching process. Therefore, the coordinate standards of the same image collected in the two processes are different, and there will be a position deviation if the coordinate conversion is directly performed. In this system, the coordinate of the feature points of the images collected in the two processes are both converted into the image coordinates during the N-point calibration, and then the image coordinates are converted into physical coordinates. Since the image magnification can quantitatively represent the size of the measured object in the images at different heights, this process is converted through the magnification at different heights. The image magnification acquisition process includes the following steps:

[0081] After entering the magnification acquisition interface, wait for the user to perform the magnification acquisition operation. If the user clicks the "Magnification Acquisition" button, the magnification is acquired; otherwise, wait for the user operation.

[0082] Combined with the tooling accuracy at the vehicle body positioning and punching station and the situation of the vehicle body itself, this system will acquire the magnification of ±10 cm of the standard distance h between the measured surface of the vehicle body and the laser sensor, and fit to obtain the fitting function of the magnification and the distance. The specific measurement process is from (standard distance h - 10 cm) to (standard distance h + 10 cm), and the measurement is performed every 5 mm. Therefore, first control the calibration plate to the position of (standard distance h - 10 cm), then obtain the distance from the upper laser sensor to the calibration plate, and use the angle calculation method in the distortion calibration process to obtain the left-right deflection angle θ1' and the front-back deflection angle θ2' of the calibration plate.

[0083] Judge whether θ1' and θ2' are stable. If they are not stable, it means that the calibration plate has not come to a complete stop yet. Then judge whether the number of measurements exceeds the set value. If so, end the current magnification acquisition and wait for the user operation; otherwise, continue to obtain the distance from the laser sensor to the calibration plate and calculate the angle.

[0084] If θ1' and θ2' are stable, then judge whether θ1' and θ2' are within the required range. When the camera and the measured surface are completely perpendicular, after the collected image is corrected for distortion to remove the distortion effect of the lens itself, there is no distortion at each position of the image. Therefore, in order to remove the influence of the distortion introduced by the angle on the magnification, it is required that θ1' and θ2' are around 0° and between ±0.1°. If the condition is satisfied, trigger the camera to take a picture to obtain the image and save it. If θ1' and θ2' are not within the required range, adjust the angle of the calibration plate and then obtain θ1' and θ2' again.

[0085] Determine whether the images at all the above-mentioned height positions have been collected. If so, process the images corresponding to all heights to obtain the pixel pitch of the checkerboard at the same position on the calibration board. The checkerboard position is selected as the checkerboard in the middle of the image because the distortion at the middle position of the image is the smallest and the pixel pitch error obtained is the smallest. Then, based on the actual physical size of the checkerboard, obtain the proportional relationship between the pixel pitch and the physical size, which is the image magnification of the calculated height. If the images at all height positions have not been collected yet, adjust the height of the calibration board to collect the image at the next height.

[0086] After obtaining the magnifications at each height, perform a first-order polynomial fitting according to the corresponding height to obtain the fitting coefficients k and b, that is, m = k * h4 + b, where m is the magnification and h4 is the distance value measured by the laser sensor. Store the obtained fitting coefficients k and b.

[0087] In an embodiment of the present invention, the template making process includes the following steps:

[0088] Click the template making button to enter the template making process interface;

[0089] Determine whether the start making template button is clicked.

[0090] If not, determine whether the exit template making button is clicked. If so, exit the template making interface. If not, the system does not act and waits for the user to click the start making template button.

[0091] If so, the system determines whether the punching robot has switched the tool TCP (Tool Center Point), that is, the system switches the TCP position of the punching robot from the marker head TCP to the punching head TCP. After the switch is completed, the punching robot will perform position movement and attitude adjustment in the workpiece coordinate system of the punching machine with the punching head TCP as the standard.

[0092] If not, the system first controls the punching robot to switch the tool TCP. After the switch is completed, control each laser sensor (including 3 on the top and 4 on the side) to obtain the distance from the vehicle body. If so, each laser sensor (including 3 on the top and 4 on the side) obtains the distance from the vehicle body.

[0093] The system determines whether the distances obtained by each laser sensor are stable, that is, each distance value is less than the set threshold.

[0094] If not, the system counts the number of measurements and determines whether this number exceeds the set value. If it does not exceed, continue to execute the system to determine whether the distances obtained by each laser sensor are stable, that is, each distance value is less than the set threshold. If it exceeds, the system prompts that the vehicle body is not placed stably and returns to determine whether the start making template button is clicked.

[0095] If so, calculate the angular deviations of each direction of the vehicle body (the rotation angle α of the vehicle body around the Y-axis, the rotation angle β of the vehicle body around the X-axis, and the rotation angle γ of the vehicle body around the Z-axis) based on the distances from the vehicle body obtained by each laser sensor (referring to the three sensors on the top and the two sensors on the right or left).

[0096] Judge whether the obtained angular deviations of each direction are within the set range.

[0097] If not, prompt that the vehicle body position is abnormal, require the user to re-place the vehicle body position, and return to judge whether to click the start button to execute the template making.

[0098] If so, execute the vehicle body feature acquisition process.

[0099] The system judges whether the vehicle body feature acquisition is abnormal.

[0100] If not, the system controls the punching robot to make its punching head run to the punching position; the system then records the intersection coordinates of the feature points, the straight line angle, and the punching point position coordinates respectively; saves the corresponding laser sensor position data, and returns to judge whether to click the start button to execute the template making.

[0101] If so, according to the abnormal type (including not finding the vehicle body feature and not finding the seam at the vehicle body feature), prompt, ask the user to confirm the vehicle body position, and end the template making process.

[0102] Furthermore, in an embodiment of the present invention, the vehicle body feature acquisition process in template making is as follows:

[0103] The system controls the camera to collect the vehicle body feature surface image.

[0104] According to the vehicle body angular deviation, the system calls the corresponding distortion calibration file to perform image distortion correction.

[0105] According to the vehicle body sheet metal feature, the system performs feature template matching at the sheet metal indirect seam.

[0106] The system judges whether the feature is found successfully, that is, whether the matching is successful. If not, the system prompts that the vehicle body feature is not found, please confirm the vehicle body position, and this process ends; if so, correct the position of the found feature, and then perform the search for two straight lines at the seam respectively.

[0107] The system judges whether the two straight lines are found successfully. If not, prompt that the seam at the vehicle body feature is not found, please confirm the vehicle body position, and this process ends; if so, find the intersection of the two found straight lines, and perform coordinate conversion on the intersection to obtain the physical coordinates.

[0108] In an embodiment of the present invention, the real-time positioning punching process includes the following steps:

[0109] Tap the real-time positioning and punching button to enter the real-time positioning and punching process interface;

[0110] Determine whether the start real-time positioning button is tapped.

[0111] If not, determine whether the exit single-station operation button is tapped. If so, exit the real-time positioning and punching interface. If not, the system does not act and waits for the user to tap the start real-time positioning button.

[0112] If so, perform the vehicle body angle acquisition process.

[0113] The system determines whether the vehicle body angle acquisition is successful; if not, prompt positioning abnormality and return to continue determining whether the start real-time positioning button is tapped. If so, execute the vehicle body feature acquisition process.

[0114] The system determines whether the vehicle body feature acquisition is successful; if not, prompt that the vehicle body feature is not found or prompt that the joint where the vehicle body feature is not found, please confirm the vehicle body position, return to continue determining whether the start real-time positioning button is tapped; if so, according to the feature points, punching points in the template making, the deflection angles in each direction of the vehicle body and the coordinates of the feature points of the real-time positioning, calculate the position of the new punching point through calculation.

[0115] The system determines whether the vehicle body offset position is within the set range. If not, prompt vehicle body position abnormality,

[0116] The user needs to re-place the vehicle body position and return to continue determining whether the start real-time positioning button is tapped; if so, after controlling the punching robot to adjust its posture, run above the new punching point, and the laser sensor obtains the distance from the new punching point in real time.

[0117] Determine whether the distance reaches the set range. If not, the punching robot continues to approach the new punching point by a certain distance and then determines whether the distance reaches the set range; if so, the system controls the punching robot to punch. After completion, return to continue determining whether the start real-time positioning button is tapped.

[0118] Furthermore, the vehicle body angle acquisition process in the real-time positioning and punching process is as follows:

[0119] Execute the sensor approaching template point process.

[0120] The system determines whether the approach is completed; if not, continue to execute the sensor approaching template point process. If so, the system calculates and obtains the non-camera surface deflection angle according to the distance measured by the laser sensor.

[0121] Execute the vehicle body feature acquisition process to obtain the camera surface deflection angle. Thus, the rotation angles of the vehicle body in three directions are obtained.

[0122] The above vehicle body angle acquisition process is different for the two cases of real-time positioning and punching on the top of the vehicle body and real-time positioning and punching on the side of the vehicle body:

[0123] Taking the positioning and drilling on the top of the vehicle body as an example, the workpiece surface coordinate system established by the drilling robot is as Figure 8 shown, where the positive direction of the Y-axis is the vehicle head direction.

[0124] The non-camera surface deflection angles obtained by real-time positioning and drilling on the top of the vehicle body include the rotation angle α of the vehicle body around the Y-axis (the left-right inclination angle of the vehicle body) and the rotation angle β of the vehicle body around the X-axis (the front-back inclination angle of the vehicle body). The top camera surface deflection angle refers to the rotation angle γ of the vehicle body around the Z-axis (the rotation angle in the roof plane parallel to the top camera surface).

[0125] The camera surface deflection angle obtained by real-time positioning and drilling on the side of the vehicle body refers to the rotation angle β of the vehicle body around the X-axis obtained by the side camera. The non-camera surface deflection angles include the rotation angle α around the Y-axis and the rotation angle γ of the vehicle body around the Z-axis obtained by the top camera.

[0126] In an embodiment of the present invention, due to the angle changes at different positions in the curved surface space, there may be position deviations when the point laser sensor performs distance acquisition, resulting in inaccurate acquired distances. This system adopts a sensor approximation process to accurately obtain the position of the point laser sensor on the feature surface during template making. The sensor approximation template point process in the vehicle body angle acquisition process is as follows:

[0127] Each laser sensor obtains the distance from the vehicle body for rough positioning.

[0128] The system determines whether the distances obtained by each laser sensor are stable, that is, whether each distance value is less than the set threshold. If not, the system counts the number of measurements and determines whether this number exceeds the set value. If it does not exceed, it returns to continue with rough positioning. If it exceeds, the system prompts that the vehicle body is not placed stably and ends this process.

[0129] If so, according to the distances from the vehicle body obtained by each laser sensor, the deflection angles of each direction of the vehicle body are calculated.

[0130] The system determines whether the deflection angles are all within the set range; if not, it prompts that the vehicle body position is abnormal and requires the user to re-place the vehicle body, and ends this process; if so, the vehicle body feature acquisition process is executed.

[0131] Find the position difference between the target point and the template point, calculate the deviation value for the sensor to move to the template point position, and control the movement of the sensor.

[0132] After moving, the laser sensor measures the distance from the vehicle body, and according to the distances from the vehicle body obtained by them, the deflection angles of each direction of the vehicle body are calculated.

[0133] The above sensor approximation template point process has differences for the two cases of real-time positioning and drilling on the top of the vehicle body and real-time positioning and drilling on the side of the vehicle body:

[0134] In the real-time positioning and drilling of the vehicle body top, each laser sensor refers to the top sensors (6, 12, 15), and the calculated deflection angles of each direction of the vehicle body include the rotation angle α of the vehicle body around the Y-axis direction and the rotation angle β of the vehicle body around the X-axis direction.

[0135] In the real-time positioning and drilling of the vehicle body side, each laser sensor includes the top sensors (6, 12) and the sensors for measuring the working distance of the camera corresponding to the positioning and drilling positions.

[0136] The process of obtaining vehicle body features in real-time positioning and drilling is the same as that in template making, except that "finding the intersection of two found straight lines and performing coordinate transformation on the intersection point to obtain the physical coordinates" is changed to "finding the intersection of two found straight lines, and according to the height of the real-time feature point and the height of the template feature point, converting the intersection point of the two straight lines into the pixel coordinates of the template image. And performing coordinate transformation on the intersection point to obtain the physical coordinates".

[0137] The formula for converting the intersection point (i.e., the feature point) of two straight lines into the pixel coordinates of the template image in the process of obtaining vehicle body features during real-time positioning and drilling is as follows:

[0138]

[0139] Where m 0 represents the magnification of the feature point in template making, which is calculated by substituting the distance value measured by the corresponding station sensor during template making into the first-order fitting function in the magnification acquisition process; m 1 represents the magnification of the feature point in real-time positioning, which is calculated by substituting the distance value measured by the corresponding station sensor after position correction during real-time positioning into the first-order fitting function in the magnification acquisition process; the point (X, Y) represents the pixel coordinates of the feature point in real-time positioning, and the point (X center , Y center ) represents the pixel coordinates of the center point of the image in real-time positioning, and the point (X', Y') represents the pixel coordinates of the feature point in real-time positioning after magnification compensation.

[0140] The point (X', Y') is transformed through coordinate transformation to obtain the physical coordinate point (X 1 , Y 1 ).

[0141] In the real-time positioning and drilling process, calculating the position of the new drilling point The specific calculation process is as follows:

[0142] Case 1: The deflection angles of the three directions of the vehicle body measured during real-time positioning are the same as those during template making;

[0143]

[0144] Among them, the point (X 0 , Y 0 ) represents the physical coordinates obtained by coordinate transformation of the pixel coordinates of the feature points in template making, and the point represents the physical coordinates of the punching point in template making.

[0145] Case 2: The vehicle body deflection angles (α and β) measured during real-time positioning are the same as those during template making;

[0146]

[0147] Among them, the angle γ Δ is equal to the vehicle body rotation angle γ during real-time positioning 1 minus the vehicle body rotation angle γ during template making 0 , represents the offset in the X-axis direction of the punching point caused by the vehicle body rotating by an angle γ Δ during real-time positioning relative to that during template making, represents the offset in the Y-axis direction of the punching point caused by the vehicle body rotating by an angle γ Δ during real-time positioning relative to that during template making.

[0148] Case 3: The vehicle body deflection angle β measured during real-time positioning is the same as that during template making;

[0149] (1) If the line segment formed by the punching point and the feature point during template making has no angle with the template plane;

[0150]

[0151] Among them, the angle α Δ is equal to the vehicle body rotation angle α during real-time positioning 1 minus the vehicle body rotation angle α during template making 0 , represents the X-axis direction offset in Case 2 multiplied by cos(α Δ ), which is the change in the X-axis direction offset in Case 2 caused by the left-right tilt of the vehicle body, that is, it represents that the offset becomes its projection on the template plane.

[0152] (2) If the line segment formed by the punching point and the feature point during template making has an angle η with the template plane, the punching point is lower than the feature point and the punching point slopes downward relative to the feature point (or the punching point is higher than the feature point and the punching point slopes upward relative to the feature point);

[0153]

[0154] Among them, the angle η X is equal to the projection angle of the angle formed by the line segment formed by the punching point and the feature point during template making and the template plane on the plane formed by the X-axis and the Z-axis, It represents the X-axis offset that needs to be compensated when there is a difference in the left-right tilt angle during real-time vehicle body positioning compared to when the template was made and there is an included angle η.

[0155] (3) If the line segment formed by the punching point and the feature point during template making has an included angle η with the template plane, the punching point is higher than the feature point and the punching point slopes downward relative to the feature point (or the punching point is lower than the feature point and the punching point slopes upward relative to the feature point);

[0156]

[0157] Case 4: The vehicle body deflection angle α measured during real-time positioning is the same as that during template making;

[0158] (1) If the line segment formed by the punching point and the feature point during template making has no included angle with the template plane;

[0159]

[0160] Among them, the angle β Δ is equal to the vehicle body rotation angle β during real-time positioning 1 minus the vehicle body rotation angle β during template making 0 .

[0161] It represents the Y-axis offset in Case 2 multiplied by cos(β Δ ), which is the change in the Y-axis offset in Case 2 caused by the front-back tilt of the vehicle body, that is, it represents that this offset becomes its projection on the template plane.

[0162] (2) If the line segment formed by the punching point and the feature point during template making has an included angle η with the template plane, the punching point is lower than the feature point and the punching point slopes downward relative to the feature point (or the punching point is higher than the feature point and the punching point slopes upward relative to the feature point);

[0163]

[0164] Among them, the angle η Y is equal to the projection angle of the included angle formed by the line segment formed by the punching point and the feature point during template making and the template plane on the plane formed by the Y-axis and the Z-axis. It represents the Y-axis offset that needs to be compensated when there is a difference in the front-back tilt angle during real-time vehicle body positioning compared to when the template was made and there is an included angle η.

[0165] (3) If the line segment formed by the punching point and the feature point during template making has an included angle η with the template plane, the punching point is higher than the feature point and the punching point slopes downward relative to the feature point (or the punching point is lower than the feature point and the punching point slopes upward relative to the feature point);

[0166]

[0167] Case 5: The body tilt angles α and β measured during real-time positioning are different from those during template production;

[0168] (1) If the line segment formed by the punching point and the feature point during template production has no angle with the template plane;

[0169]

[0170] (2) If the line segment formed by the punching point and the feature point during template production has an angle η with the template plane, the punching point is lower than the feature point and the punching point slopes downward relative to the feature point (or the punching point is higher than the feature point and the punching point slopes upward relative to the feature point);

[0171]

[0172] (3) If the line segment formed by the punching point and the feature point during template production has an angle η with the template plane, the punching point is higher than the feature point and the punching point slopes downward relative to the feature point (or the punching point is lower than the feature point and the punching point slopes upward relative to the feature point);

[0173]

[0174] In an embodiment of the present invention, the full-station automatic operation process completes the positioning and punching operations for all stations to be punched in accordance with the set sequence. After completing the positioning and punching operations for all stations, the punching robot is run to the return zero point according to the return zero trajectory and waits for the arrival of the next object to be detected. As Figure 6 shown, the full-station automatic operation process includes the following steps:

[0175] After entering the full-station operation interface, wait for the user to click the "Full-station operation" button. If the user clicks the "Full-station operation", the vehicle body in-place judgment process is executed; otherwise, wait for the user's operation.

[0176] The vehicle body operation assembly line consists of multiple large flat plates. The front and rear tires of the vehicle body and the tire side limiting devices are installed on the large flat plates. After the vehicle body is placed in the limiting positions in sequence, the large flat plates move forward in sequence. When it reaches the visual positioning station, the bottom limiting device of the large flat plate triggers the limiting signal. Therefore, the vehicle body in-place judgment process first judges whether the limiting signal is received. If the limiting signal is received, the laser sensor at the visual positioning station is controlled to obtain the height value from the vehicle roof in real time. If the height value remains stable within the set time, it is considered that the vehicle body has reached the position and is stable, and the single-station real-time positioning process is executed; otherwise, continue to obtain the height value of the laser sensor and judge whether the height value remains stable within the set time.

[0177] The system sequentially executes the single-station real-time positioning process for a total of 6 workstations on the left and right sides of the top, front side, and rear side of the vehicle body. Among them, the left and right workstations on the top of the vehicle body execute the single-station real-time positioning process first because the front-back deviation angle and left-right deviation angle of the top relative to the camera plane are the smallest, and the rotation angle of the top within the camera plane can be accurately measured visually. The three-dimensional angle offset of the vehicle body relative to the template during production obtained in this way is the most accurate. After completing the single-station real-time positioning process for both sides of the vehicle top, the three angles of the vehicle body relative to the template vehicle body in the three-dimensional direction can be obtained. Among them, the left-right deviation angle and the rotation angle within the camera plane can be used as the rotation angles required in the side single-station real-time positioning process. A total of 7 laser sensors are installed in this system, with 3 laser sensors installed on the top of the vehicle body to obtain the front-back deviation angle and left-right deviation angle of the vehicle body camera plane. At the same time, the left and right laser sensors below the camera on the top of the vehicle body are also used to obtain the actual height of the vehicle body; 1 laser sensor is installed at each of the front and rear workstations on the left and right sides of the vehicle body to obtain the distance of the vehicle body. Therefore, a total of 7 laser sensors are installed in this system.

[0178] Judge whether all workstations have completed punching. If so, control the punching robot to return to the standby point and wait for the arrival of the next vehicle body. Otherwise, continue with the real-time positioning of the vehicle body at the next workstation.

[0179] The surface of the detected vehicle is not a completely flat plane, and there is an arc in the sheet metal of the vehicle surface. If the position of the sensor remains unchanged, when making the template, there will be a deviation between the distance information collected by the laser sensor and the actual measurement by the laser sensor. The vehicle angle offset obtained through the distance is not the correct angle. Therefore, it is set that the camera is fixed and the laser sensor is movable.

[0180] After the execution of the laser sensor approaching process is completed, the front-back deviation angle and left-right deviation angle of the vehicle top can be calculated according to the laser sensor. An example of the solution method is as follows: See Figure 7 As shown, if circle 1 is the initial position, the distances measured by the left rear and right rear are HG and AC respectively, and the distance between the two sensors is AH. Circle 2 is the position of re-parking. If the sensor does not move with it, the measured positions are HD and AB. In actual use, the sensor will approach the position according to the visual positioning situation, so only the distances after the sensor follows the movement need to be calculated. The new positions of the sensors I and J after following the movement are the positions where the sensor follows the movement, and the measured positions are IM and JK. As shown in the figure, the vehicle angle does not change much. When making the template, the sensor does not move. At this time, the deviation angle of the vehicle body in a certain direction can be calculated through the distance relationship of the sensor; while in real-time detection, after the sensor moves through the approaching method, the deviation angle of the vehicle body in a certain direction can be calculated again through the distance relationship of the sensor. The calculation formula is as follows:

[0181] θ is the included angle between AH and GC, which is the vehicle body angle during template production

[0182] θ' is the included angle between AH and MK, which is the angle of the vehicle body during real-time positioning.

[0183] Through the above formula, the inclination angles of the vehicle body in the specified direction during template making and real-time positioning can be calculated. According to the above definition, the α angle of the vehicle body is the left-right inclination angle of the vehicle body, and the β angle is the front-back inclination angle of the vehicle body. Then, when calculating the left-right inclination angle of the vehicle body, θ in this formula corresponds to the α angle of the vehicle body during template making, and θ′ corresponds to the α angle of the vehicle body during real-time positioning. The deviation angle Δ of the vehicle body during real-time positioning can be obtained through (θ′ - θ), specifically α Δ , β Δ and γ Δ .

[0184] Example: Assume the work station is at the camera work station on the right side of the roof. According to the above picture display, through the camera, the change in the characteristic position of the roof in the direction around the Y-axis (left-right direction of the vehicle body) is K, and the control center controls the left-right adjustment device 4 of the top right sensor to move a displacement of K value.

[0185] Assume the work station is at the camera work station on the right side of the roof. According to the above picture display, through the camera, the change in the characteristic position of the roof in the direction around the X-axis (front-back direction of the vehicle body) is L, and the control center controls the left-right adjustment device 4 of the top right sensor to move a displacement of L value. The same applies to other work stations.

[0186] 6 fixed cameras respectively collect the information of the feature points, and 7 ranging sensors measure the distances.

[0187] It is known that the top sensors are at the same height of 12, and the distance between 6 and 6 is H1

[0188] The top sensors are at the same height of 6, and the distance between 6 and 15 is H2

[0189] The right sensors are at the same height of 30, and the distance between 30 and 24 is H3

[0190] The left sensors are at the same height of 36, and the distance between 36 and 42 is H4

[0191] Set the measured value of the sensor template as: H 12 , H 6 , H 15 , H 30 , H 24 , H 36 , H 42

[0192] Set the measured value of the sensor during template making as: h 12 , h 6 , h 15 , h 30 , h 24 , h 36 , h42

[0193] When measuring in real time, the measured value after the sensor corrects its position is: h 12-1 , h 6-1 , h 15-1 , h 30-1 , h 24-1 , h 36-1 , h 42-1 When the vehicle body is in place, start making the template. The distance values measured by 7 sensors are fed back to the control center to calculate the magnification ratio.

[0194] The camera takes pictures to record the feature point information.

[0195] The vehicle body is in place again and real-time measurement starts;

[0196] The top sensor measures the distance value h 12-1 , h 6-1 , calculate the magnification ratio and send it to the control center.

[0197] The two top cameras calculate the offset angle in the Z-axis direction and the offset in the XY coordinate system plane and send the data to the control center.

[0198] The control center adjusts the sensor position according to the camera parameters.

[0199] Rely on the three top sensors to calculate the offset angles of the vehicle body in the X-axis and Y-axis directions. Thus, the rotation angles of the vehicle body in the XYZ three coordinate axis directions are obtained.

[0200] The angle calculation formula during template making is as follows:

[0201] Rotation angle around the X-axis: α 0 = tan -1 (h 15 -h 6 ) / H 2

[0202] Rotation angle around the Y-axis: β 0 = tan -1 (h 12 -h 6 ) / H 1

[0203] The angle calculation formula during real-time positioning is as follows:

[0204] Rotation angle around the X-axis: α 1 = tan -1 (h 15-1 -h 6-1 ) / H 2

[0205] Rotation angle around the Y-axis: β1 = tan -1 (h 12-1 - h 6-1 ) / H 1

[0206] By calculating (β 1 - β 0 ), β Δ can be obtained. By calculating (α 1 - α 0 ), α Δ can be obtained.

[0207] The corrected value of the sensor position is transmitted to the control center and used as the magnification correction value for the camera.

[0208] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, those of ordinary skill in the art should understand that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. An inspection and positioning system for automobile assembly, characterized in that, it includes a top inspection component, a limiting device, a left-side inspection component and a right-side inspection component. First, the vehicle body is transported to the assembly line flat plate, and the vehicle body is successively transported to the flat plate assembly line with a limiting device through a crane. When the vehicle body runs to the visual inspection position through the assembly line, the flat plate is controlled by the limiting device installed below to stop, and the signal is acquired by the vision system, and then user operations are selected. The vision system performs single-station operations or full-station operations according to the user's selection. Before the single-station and full-station real-time positioning operations, the preparation processes for positioning and punching at each station must be carried out first, which are the distortion calibration process, the N-point calibration process, the magnification acquisition process and the template production process. In the single-station real-time positioning process and the full-station real-time positioning process, through the position of the feature points in the template production, the position of the laser sensor is adjusted to approach the position during template production, so as to obtain accurate laser sensor values. The preparation processes for positioning and punching at each station respectively use the corresponding inspection components at each station to acquire and process the images and laser sensor measurement values. In the single-station real-time positioning process and the full-station real-time positioning process, first control the laser sensor in the top inspection component to approach the position to obtain accurate laser sensor values, and then calculate the deflection angle of the vehicle body on the non-camera plane. Then, the deflection angle of the vehicle body on the camera plane is calculated by obtaining the features of the image through the camera and the image features during template production. The corresponding deflection angles are used for processing during the distortion correction and image coordinate conversion at each station; through the images acquired by the cameras of the inspection components at each station, the physical positions of the feature points are further calculated. According to the feature information of the images during template production and real-time positioning and the acquired three-dimensional deflection angle information of the vehicle body, the position information and punching posture of the punching points are accurately calculated, and then precise positioning and punching are realized.

2. The inspection and positioning system for automobile assembly according to claim 1, characterized in that, the top inspection component includes a top right-side inspection camera, a first top right-side light source, a second top right-side light source, a top right-side sensor left-right adjustment device, a top right-side sensor front-back adjustment device, a top right-side distance measurement sensor, a top left-side inspection camera, a first top left-side light source, a second top left-side light source, a top left-side sensor left-right adjustment device, a top left-side sensor front-back adjustment device, a top left-side distance measurement sensor, a top front-side sensor left-right adjustment device, a top front-side sensor front-back adjustment device and a top front-side distance measurement sensor.

3. The inspection and positioning system for automobile assembly according to claim 1, characterized in that, the limiting device includes front-back limit for the right front wheel, left-right limit for the right front wheel, front-back limit for the left front wheel, left-right limit for the left front wheel, left-right limit for the right rear wheel, front-back limit for the right rear wheel, front-back limit for the left rear wheel and left-right limit for the left rear wheel.

4. The inspection and positioning system for automobile assembly according to claim 1, characterized in that, The right detection component includes a right front distance measurement sensor, a front-back adjustment device for the right front sensor, an up-down adjustment device for the right front sensor, a right front detection camera, a first right front light source, a second right front light source, a right rear distance measurement sensor, a front-back adjustment device for the right rear sensor, an up-down adjustment device for the right rear sensor, a right rear detection camera, a first right rear light source, and a second right rear light source.

5. The detection and positioning system for vehicle assembly according to claim 1, wherein, the left detection component includes a left front distance measurement sensor, a front-back adjustment device for the left front sensor, an up-down adjustment device for the left front sensor, a left front detection camera, a first left front light source, a second left front light source, a left rear distance measurement sensor, an up-down adjustment device for the left rear sensor, a front-back adjustment device for the left rear sensor, a left rear detection camera, a first left rear light source, and a second left rear light source.

6. The detection and positioning system for vehicle assembly according to claim 1, wherein, the camera distortion calibration process is used to obtain distortion calibration files at different angles of the feature plane, and the distortion calibration files are used for image distortion correction when the feature plane and the camera plane are at different angles.

7. The detection and positioning system for vehicle assembly according to claim 1, wherein, the N-point calibration process is used to obtain an N-point calibration file to realize the conversion of image coordinates to physical coordinates in the punching robot coordinate system.

8. The detection and positioning system for vehicle assembly according to claim 1, wherein, the magnification acquisition process is used to obtain a first-order polynomial relationship function between the height of the feature plane and the magnification. According to the relationship function, the conversion of image coordinates at different heights of the feature plane is realized, so as to obtain accurate physical coordinates of the feature points.

9. The detection and positioning system for vehicle assembly according to claim 1, wherein, the template making process is used to obtain the position information of the image processing feature points obtained by the camera and the position information of the punching points on the punching surface, and based on this, the position relationship between the feature points and the punching points is obtained.

10. The detection and positioning system for vehicle assembly according to claim 1, wherein, in the full-station automatic operation process, the single-station real-time positioning process is sequentially executed for a total of 6 stations on the left and right sides of the top, front side, and rear side of the vehicle body. Among them, the left and right stations on the roof first execute the single-station real-time positioning process. After completing the single-station real-time positioning process on both sides of the roof, three angles of the vehicle body relative to the template-making vehicle body in the three-dimensional direction can be obtained. Among them, the left-right deviation angle and the rotation angle in the camera plane can be used as the rotation angles required in the side single-station real-time positioning process.

Images