A calibration method for multiple suction nozzles

Abstract

The present invention discloses and provides a method for calibrating multiple suction nozzles capable of discharge. This calibration method is applicable to fly-by-flight photography. It primarily relies on the corresponding coordinates between the Mark points of an overhead camera during static shooting and the Mark points during fly-by-flight photography, thereby calibrating the coordinates of the Mark points during actual fly-by-flight photography. This method can effectively reduce camera calibration errors and improve camera accuracy. The present invention includes a. overhead camera calibration; b. overhead camera calibration; c. nozzle rotation center calibration; d. calibration of the relationship between the overhead camera optical center and the nozzle rotation center; and e. fly-by-flight offset calibration. The present invention is applicable to the technical field of fly-by-flight calibration.

Description

Technical Field

[0001] The present invention relates to a calibration method, and in particular to a calibration method for multiple suction nozzles capable of discharge and suction. Background Art

[0002] With the development of the 3C industry and advancements in automation equipment, large numbers of products awaiting inspection are now placed on pallets in current electronic product manufacturing. These pallets are then loaded and handled using vision positioning systems. Prior to vision positioning, camera calibration is required. Traditional camera calibration is only semi-automatic and requires operator assistance. This process is characterized by a low degree of automation, high operational complexity, and a long process time, resulting in low accuracy. Furthermore, existing calibration techniques are cumbersome and complex, requiring the operator to simultaneously control both the motion axes and the vision system. Specifically, existing manual or semi-automatic calibration methods involve manually moving a robotic arm so that a material point appears in the upper left, upper center, upper right, center, center right, lower left, lower center, or lower right field of view. The user then takes a photo and records the pixel coordinates. The corresponding robotic arm position and pixel coordinates are then stored sequentially. However, this manual calibration method results in inconsistent positioning due to manual movement, preventing stable measurement. Manually calibrating and saving data point by point increases the complexity and accuracy of the process, resulting in low accuracy due to human variability, and significantly increases operator workload and fatigue.

[0003] Furthermore, as production speeds increase, camera positioning speeds are also increasing. Therefore, flying camera photography is often used. However, maintaining a constant speed during flying camera photography can be difficult, leading to unstable positioning of the products under inspection and making it difficult to meet the demands of high-volume, high-quality production. This is primarily due to the short flying distance of the socket, making it difficult for the PLC to maintain a constant flying speed. This leads to deviations in the camera position signals provided by the PLC, necessitating socket slot calibration. However, existing calibration methods often overlook this error, ultimately resulting in unsatisfactory calibration results.

[0004] Therefore, if an automatic calibration method that is simple to use, highly automated, and adaptable to flying photography can be designed, the above problems can be solved well. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a multi-suction nozzle calibration method. This calibration method can be applied to flying photography. It mainly relies on the corresponding relationship between the coordinates of the Mark point when the overhead camera is shooting at a static point and the coordinates of the Mark point during the flying photography process, so as to calibrate the coordinates of the Mark point during actual flying photography, which can effectively reduce the error of camera calibration and improve the accuracy of the camera.

[0006] The technical solution adopted by the present invention is: a method for calibrating multiple suction nozzles, which is applied to a machine vision system. The machine vision system includes a mechanical axis motion module, a vision module Vision, a controller PLC, a suction nozzle, a top-down camera, and an overhead camera. The vision module Vision is connected to the controller PLC signal, the top-down camera and the overhead camera are connected to the vision module Vision signal, and the suction nozzle and the overhead camera are arranged on the mechanical axis motion module. The method is characterized in that: the method comprises the following steps:

[0007] a. Calibration of overhead camera: the vision module Vision notifies the controller PLC to get the calibration plate; the calibration plate is placed on the calibration plate fixing bracket above the overhead camera; the vision module Vision then notifies the controller PLC to move the overhead camera to the nine-point calibration reference point; the overhead camera moves to the next position of the nine-point calibration and returns the corresponding mechanical coordinates. The vision module Vision collects images and calculates the center pixel coordinates of the calibration plate Mark point. The overhead camera then moves to other positions of the nine-point calibration in turn. The overhead camera moves a total of nine times, and each movement returns the corresponding mechanical coordinates. Each time it moves, the vision module Vision collects images and calculates the center pixel coordinates of the calibration plate Mark point. Finally, the affine transformation matrix of the overhead camera image coordinate system and the mechanical coordinate system is calculated based on the nine sets of data. ;

[0008] b. Calibration of the overhead camera: The vision module Vision notifies the controller PLC to move the nozzle to the reference point of the nine-point calibration of the overhead camera; then the nozzle Mark point moves to the next position of the nine-point calibration and returns the corresponding mechanical coordinates. The vision module Vision collects images through the upward camera and calculates the pixel coordinates of the center of the nozzle Mark point. Then the nozzle moves to other positions of the nine-point calibration in turn. The overhead camera moves a total of nine times. Each time it moves, the vision module Vision collects images through the upward camera and calculates the pixel coordinates of the center of the nozzle Mark point. Finally, the affine transformation matrix of the upward camera image coordinate system and the mechanical coordinate system is calculated based on the nine sets of data. ;

[0009] c. Calibration of the nozzle rotation center: The vision module Vision notifies the controller PLC to move the nozzle to the reference point of the five-point calibration of the overhead camera; then the nozzle mark point moves to the next position of the five-point calibration and returns the corresponding mechanical coordinates. The vision module Vision captures the image and calculates the pixel coordinates of the center of the nozzle mark point. The nozzle then moves to other positions of the five-point calibration in sequence. The nozzle moves a total of five times. Each time it moves, the vision module Vision captures the image and calculates the pixel coordinates of the center of the nozzle mark point. Finally, the image coordinate system and mechanical coordinate system of the nozzle rotation center are calculated based on the five sets of data.

[0010] d. Calibration of the relationship between the optical center of the upward camera and the rotation center of the nozzle: Place the circular hole calibration plate in the field of view of the upward camera and the downward camera and photograph the circular hole mark points respectively. Then remove the calibration plate, move the nozzle into the field of view of the upward camera and photograph the nozzle mark points in turn. During this period, it is necessary to ensure that the calibration plate plane, the tray plane, and the plane when the nozzle is extended to take pictures are all in the same plane. That is, first use the downward camera and the upward camera to photograph the circular hole calibration plate mark points respectively, calculate the pixel coordinates of the center of the calibration plate mark points, and then calculate the pixel coordinates of the center of the calibration plate mark points according to the calibration plate. and The pixel coordinates of the center of the Mark point on the calibration plate can be mapped to the gantry axis coordinate system as follows: and ,according to Map the optical center coordinates of the top-view camera to the gantry axis coordinate system: , then move the gantry axis distance , so that the nozzle mark point is in the field of view of the upward-looking camera and extends it out to take a photo. Then, the nozzle is moved according to the sequence of the five-point calibration method. The camera takes a photo every time it moves to a certain position. The coordinates of the center of the nozzle mark point are calculated. The rotation center B(a, b) can be calculated by fitting a circle with the five points.

[0011] Calculate the relationship between the optical center C of the top-view camera and the rotation center B of the nozzle:

[0012]

[0013] ;

[0014] Then the offset between the optical center C of the top view camera and the rotation center B of the nozzle is: .

[0015] Furthermore, the method for calibrating multiple suction nozzles further includes the following steps:

[0016] e. Calibration of the flying camera offset: The position where the socket stops each time is fixed and used as the standard coordinate reference. Move the gantry axis to the position where the flying camera triggers the photo shooting, and then take a photo of the slot at rest to obtain the coordinates of the slot center in the gantry axis in the static state. , and obtain the center coordinates of other slots in turn; after performing linear operation on the obtained slot center coordinates and the reference coordinates, the actual flying shot offset value can be obtained.

[0017] The beneficial effects of the present invention are as follows: the existing manual calibration or semi-automatic calibration, by manually moving the manipulator to make the material point appear in the upper left, upper middle, upper right, middle left, middle, middle right, lower left, lower middle, and lower right of the field of view and taking pictures to record the pixel coordinates, and the corresponding manipulator points and pixel coordinates are saved in sequence, and this manual calibration method will lose consistency due to the different points brought about by manual movement and fail to achieve a stable given amount. Manual calibration point by point to save data increases the complexity of the process, and due to the problem of manual use differences, the accuracy is low, and the workload and fatigue of the users are greatly increased. At the same time, the existing manual calibration or semi-automatic calibration, especially when it comes to flying correction calibration, is often corrected by manual compensation. Such calibration often causes large errors due to changes in external factors, and the manual compensation method is inefficient and unreliable. In addition, the existing calibration technology is complicated to operate, and the user needs to control the motion axis and the visual system at the same time to achieve calibration. The present invention can solve the above problems well and achieve the following effects:

[0018] 1. Adapting to the flying shooting mode, it can accurately calibrate the offset caused by flying shooting, eliminate the problem that the PLC cannot ensure uniform flying speed, resulting in deviation in the shooting position signal given by the PLC, and improve calibration accuracy;

[0019] 2. The calibration method is simple and convenient to use, with one-click calibration, which reduces the operating requirements for users;

[0020] 3. The fully automatic calibration process automatically gives the required movement amount and rotation angle, automatically controls the arrival at the required point to take pictures, transmit data, and perform calculations, ensuring a stable given amount and solving the tedious manual steps of manual calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the robot's moving path;

[0022] Figure 2 It is a schematic diagram of the moving path of the image marker;

[0023] Figure 3 It is a schematic diagram of the calibration of the top-down camera;

[0024] Figure 4 This is a schematic diagram of the calibration of an upward-shooting camera;

[0025] Figure 5 This is the overall diagram of the calibration of the upward-shooting camera;

[0026] Figure 6 This is a schematic diagram of the rotation center calibration of the nozzle;

[0027] Figure 7 This is a schematic diagram of the relationship between the optical center of the upward-shooting camera and the center of the nozzle mark point;

[0028] Figure 8 This is a schematic diagram of the calculation of the optical center of the overhead camera and the center of the nozzle mark point;

[0029] Figure 9 This is a schematic diagram of the socket slot taken with a static camera looking down;

[0030] Figure 10 This is the flowchart of the overhead camera calibration;

[0031] Figure 11 This is the nozzle calibration flow chart. DETAILED DESCRIPTION

[0032] The primary goal of this automated unloading fly-photography calibration system is to acquire images from industrial cameras and calculate the object's position in the surrounding space. The camera maps image coordinates to mechanical axis coordinates. This mapping is determined by the relative position of the camera and mechanical axis, i.e., the conversion between the camera coordinate system and the manipulator coordinate system. Figure 1 is the moving path of the robot using the nine-point calibration method, Figure 2 The pixel coordinates of the circle center of the corresponding image mark point are given. Figure 1 The path shown starts from the reference point and moves sequentially. The camera takes a picture at each position, and then calculates the pixel coordinates of the center of the circle of the marked points. When all 9 points are completed, the affine transformation matrix between the pixel coordinate system and the mechanical coordinate system is calculated.

[0033] 1. Overhead Camera Calibration

[0034] like Figure 3 and Figure 9 As shown in the figure, the overhead camera is moved above the circular hole calibration plate and moved sequentially within the camera's field of view according to the nine-point calibration method. The camera takes a picture at each position and calculates the pixel coordinates corresponding to the center of the circular hole in the calibration plate, ultimately obtaining 9 pixel coordinates.

[0035] Finally, the affine transformation matrix between the image coordinate system of the overhead camera and the mechanical coordinate system of the gantry axis is calculated using the 9 pixel coordinates and their corresponding mechanical coordinates.

[0036] 2. Calibration of the upward-facing camera

[0037] like Figure 4 and Figure 5 As shown, move the nozzle to the field of view above the upward camera, and move the nozzle mark points to the specified positions in the order of the nine-point calibration method.

[0038] The camera takes a picture at each position and calculates the pixel coordinates of the center of the corresponding nozzle marker. Finally, the affine transformation matrix between the image coordinate system of the overhead camera and the mechanical coordinate system of the gantry axis is calculated using the nine pixel coordinates and the corresponding mechanical coordinates.

[0039] 3. Calibration of nozzle rotation center

[0040] like Figure 6 and Figure 10 As shown, since the nozzle can rotate, after the upward camera is calibrated at nine points, the nozzle's rotation center needs to be calibrated to obtain the nozzle's rotation center pixel coordinates B1_(x_pix, y_pix). Take nozzle 1 as an example: Figure 6 The moving path of the gantry axis is based on the five-point calibration method. Figure 6 The path shown starts from reference point 1 and moves sequentially, taking photos and calculating the pixel coordinates of the nozzle's rotation center, B1_(x_pix, y_pix). The angles B2-B8 of the other seven nozzles can be obtained in turn.

[0041] 4. Calibration of the relationship between the optical center of the overhead camera and the rotation center of the nozzle

[0042] Place the circular hole calibration plate in the field of view of the upward camera and the downward camera and take photos of the circular hole marking points respectively. Then remove the calibration plate and move the suction nozzle into the field of view of the upward camera in turn and take photos of the suction nozzle marking points. During this process, it is necessary to ensure that the calibration plate, the tray, and the suction nozzle are all in the same plane when extending to take photos.

[0043] like Figure 7 As shown, it is assumed that the solid line nozzle and the downward-looking camera represent the nozzle and the downward-looking camera before the gantry axis moves, and the dotted line nozzle and the downward-looking camera represent the nozzle and the downward-looking camera after the gantry axis moves.

[0044] First, use the downward camera and upward camera to shoot the Mark point of the circular hole calibration plate respectively, calculate the center pixel coordinates of the Mark point of the calibration plate, and then use the and The pixel coordinates of the center of the Mark point on the calibration plate can be mapped to the gantry axis coordinate system as follows: and .according to Map the optical center coordinates of the top-view camera to the gantry axis coordinate system: .

[0045] Then move the gantry axis distance , so that the nozzle mark point is in the field of view of the upward-looking camera and is extended to take a photo. The nozzle is moved according to the sequence of the five-point calibration method. The camera takes a photo every time it moves to a certain position. The coordinates of the center of the nozzle mark point are calculated. The rotation center B(a, b) can be calculated by fitting a circle with the five points.

[0046] Now we need to calculate the relationship between the optical center C of the top view camera and the rotation center B of the nozzle. ,Depend on Figure 7 It can be seen that:

[0047]

[0048] ;

[0049] Therefore, the offset between the optical center C of the top view camera and the rotation center B of the nozzle is: .

[0050] 5. Flying shot offset calibration

[0051] like Figure 8 As shown, due to the short flying distance of the Socket, the PLC cannot guarantee a uniform flying speed, resulting in deviations in the shooting position signal given by the PLC. The Socket Slot needs to be calibrated. Taking Slot 1 as an example, the position where the Socket stops each time is fixed and used as the standard coordinate reference. Move the gantry axis to the position where the flying camera triggers the shooting, and then take a picture of the Slot at rest to obtain the coordinates of the Slot center in the gantry axis in the stationary state. , and obtain the other seven slot center coordinates in turn. After performing a linear operation on the obtained slot center coordinates and the reference coordinates, the actual flight offset value can be obtained.

[0052] Although the embodiments of the present invention are described with practical solutions, they do not limit the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.

Claims

1. A method for calibrating multiple suction nozzles capable of discharge and suction, which is applied to a machine vision system. The machine vision system includes a mechanical axis motion module, a vision module, a controller, a suction nozzle, a top-down camera, and an overhead camera. The vision module is connected to the controller by signal, and the top-down camera and the overhead camera are connected to the vision module by signal. The suction nozzle and the overhead camera are arranged on the mechanical axis motion module. The method is characterized in that: The method for calibrating multiple suction nozzles includes the following steps: a. Calibration of the overhead camera: the vision module notifies the controller to get the calibration plate; the calibration plate is placed on the calibration plate fixing bracket above the overhead camera; the vision module then notifies the controller to move the overhead camera to the nine-point calibration reference point; the overhead camera moves to the next position of the nine-point calibration and returns the corresponding mechanical coordinates. The vision module collects images and calculates the center pixel coordinates of the calibration plate Mark point. The overhead camera then moves to other positions of the nine-point calibration in turn. The overhead camera moves a total of nine times, and each movement returns the corresponding mechanical coordinates. Each time it moves, the vision module collects images and calculates the center pixel coordinates of the calibration plate Mark point. Finally, the affine transformation matrix of the overhead camera image coordinate system and the mechanical coordinate system is calculated based on the nine sets of data. ; b. Calibration of the upward camera: The vision module notifies the controller to move the nozzle to the reference point of the nine-point calibration of the upward camera; then the nozzle Mark point moves to the next position of the nine-point calibration and returns the corresponding mechanical coordinates. The vision module collects images through the upward camera and calculates the pixel coordinates of the center of the nozzle Mark point. Then the nozzle moves to other positions of the nine-point calibration in turn. The downward camera moves a total of nine times. Each time it moves, the vision module collects images through the upward camera and calculates the pixel coordinates of the center of the nozzle Mark point. Finally, the affine transformation matrix of the upward camera image coordinate system and the mechanical coordinate system is calculated based on the nine sets of data. ; c. Calibration of the nozzle rotation center: The vision module notifies the controller to move the nozzle to the reference point of the five-point calibration of the overhead camera; then the nozzle mark point moves to the next position of the five-point calibration and returns the corresponding mechanical coordinates. The vision module captures the image and calculates the pixel coordinates of the center of the nozzle mark point. The nozzle then moves to other positions of the five-point calibration in sequence. The nozzle moves a total of five times. Each time it moves, the vision module captures the image and calculates the pixel coordinates of the center of the nozzle mark point. Finally, the image coordinate system and mechanical coordinate system of the nozzle rotation center are calculated based on the five sets of data. d. Calibration of the relationship between the optical center of the upward camera and the rotation center of the nozzle: Place the circular hole calibration plate in the field of view of the upward camera and the downward camera and photograph the circular hole mark points respectively. Then remove the calibration plate, move the nozzle into the field of view of the upward camera and photograph the nozzle mark points in turn. During this period, it is necessary to ensure that the calibration plate plane, the tray plane, and the plane when the nozzle is extended to take pictures are all in the same plane. That is, first use the downward camera and the upward camera to photograph the circular hole calibration plate mark points respectively, calculate the pixel coordinates of the center of the calibration plate mark points, and then calculate the pixel coordinates of the center of the calibration plate mark points according to the calibration plate. and The pixel coordinates of the center of the Mark point on the calibration plate can be mapped to the gantry axis coordinate system as follows: and ,according to Map the optical center coordinates of the top-view camera to the gantry axis coordinate system: , then move the gantry axis distance , so that the nozzle mark point is in the field of view of the upward-looking camera and extends it out to take a photo. Then, the nozzle is moved according to the sequence of the five-point calibration method. The camera takes a photo every time it moves to a certain position. The coordinates of the center of the nozzle mark point are calculated. The rotation center B(a, b) can be calculated by fitting a circle with the five points. Calculate the relationship between the optical center C of the top-view camera and the rotation center B of the nozzle: ； Then the offset between the optical center C of the top view camera and the rotation center B of the nozzle is: 。 2. The method for calibrating multiple suction nozzles according to claim 1, characterized in that: The method for calibrating multiple suction nozzles further comprises the following steps: e. Calibration of the flying camera offset: The position where the socket stops each time is fixed and used as the standard coordinate reference. Move the gantry axis to the position where the flying camera triggers the photo shooting, and then take a photo of the slot at rest to obtain the coordinates of the slot center in the gantry axis in the static state. , and obtain the center coordinates of other slots in turn; after performing linear operation on the obtained slot center coordinates and the reference coordinates, the actual flying shot offset value can be obtained.

Images