A visual calibration and alignment method for laminating special-shaped products

Abstract

The present invention discloses a visual calibration and alignment bonding method for special-shaped product bonding. The method comprises the following steps: transporting an assembly object through an xyθ material adsorption platform, which can move up and down and left and right; fixing the assembly object through a loading platform; moving the calibration material adsorption platform through a control mechanism, obtaining a motion coordinate system for the operation of the control mechanism, and then using a dot matrix calibration plate containing X and Y coordinate information. The calibration plate is transferred to the immovable loading platform through the material adsorption platform, and calibration information on the calibration plate is collected by at least one camera installed on the loading platform. The position of each camera is obtained by calculation, and a calibration conversion matrix between the camera position and the loading platform is calculated, thereby obtaining a central reference position and a reference angle of the installation workpiece. At this time, the workpiece can be installed according to the specified position by controlling the corresponding angle and translation amount of the material adsorption platform movement.

Description

Technical Field

[0001] The present invention relates to a visual calibration and alignment lamination method for laminating special-shaped products, belonging to the technical field of camera calibration. Background Art

[0002] During the workpiece assembly process, it's difficult to maintain consistent material placement. High-precision transfer often requires appropriate visual calibration, position correction, and alignment. A visual calibration system calculates the offset of the assembled workpiece, and compensation mechanisms correct the position for precise assembly. Conventional methods are unable to reliably convert coordinates for irregularly shaped parts. Therefore, a low-cost, high-precision visual calibration method is particularly needed to meet the requirements of large-scale workpieces and high-precision inspection of specific parts.

[0003] Existing methods for special-shaped bonding mostly use single cameras with a large field of view. However, this brings problems such as insufficient accuracy and difficulty in imaging when the depth of field is inconsistent. At the same time, the cost of large-size cameras is high.

[0004] High-precision calibration of camera systems is a relatively complex task in the field of computer vision. It often requires the design of high-precision, high-stability hardware to ensure the long-term stability of the entire measurement system. For complex multi-camera systems, the design cost of this hardware will be very expensive. Summary of the Invention

[0005] Purpose of the Invention: To address the problems and shortcomings of the prior art, the present invention provides a visual calibration and alignment method for the assembly of irregularly shaped products. This coordinate fusion calibration method allows for the flexible use of one or more cameras in the upper and lower units, improving assembly accuracy while reducing costs. Furthermore, the present invention provides a visual calibration method that can be applied to various complex scenarios, offering simplified operation and significantly reduced hardware costs.

[0006] A method for visual calibration and alignment of irregular-shaped product assembly involves transporting assembly objects via an xyθ material adsorption platform, which can move up and down (along the θ direction) and left and right (along the x and y directions). A loading platform secures the assembly objects. A control mechanism moves the calibration material adsorption platform, acquiring the motion coordinate system of the control mechanism. A dot matrix calibration plate containing X and Y coordinate information is then transferred to the immobile loading platform via the material adsorption platform. Calibration information on the calibration plate is collected by at least one camera mounted on the loading platform. The positions of each camera are calculated, and a calibration mapping matrix is ​​calculated between the camera positions and the loading platform, thereby determining the central reference position and reference angle of the workpiece to be installed. The workpiece can then be installed in the specified position by controlling the corresponding angle and translation of the material adsorption platform.

[0007] The specific steps include:

[0008] S1, move the material adsorption platform to the location of the calibration plate to collect the calibration plate image; the calibration plate coordinate system direction is consistent with the material adsorption platform movement direction, and it is necessary to ensure that the camera field of view contains at least one complete dot matrix set area with digital coordinate information when in the photo position; the calibration plate is a dot matrix calibration plate with digital coordinate information.

[0009] S2, by randomly selecting 9 points (coordinate points) in the collected calibration plate image and setting the calibration plate coordinate data (just record the digital information corresponding to these nine points), complete the conversion relationship between the material adsorption platform coordinate system and the coordinate system of the camera field of view associated with the immovable loading platform. This conversion relationship can convert the pixel coordinates within the camera field of view to the physical coordinates on the loading platform, thereby being able to describe the posture of the workpiece on the loading platform.

[0010] S3, controlling the material adsorption platform to transfer the calibration plate to the loading platform, so that the camera corresponding to the transfer platform obtains the image of the calibration plate.

[0011] S4, randomly select 9 points on the image within the camera field of view, set the calibration coordinate data, and complete the correlation calculation matrix conversion relationship between the loading platform and the material adsorption platform through the nine-point calibration algorithm, (X t ,Y t ) represents the coordinates of the output loading platform, (X i ,Y i ) represents the image coordinates in the camera field of view of the loading platform, (X o ,Y o ) represents the position of the output loading platform origin in the image coordinate system, α is the angle between the output coordinate system and the image coordinate system, and the conversion formula is:

[0012] S5, by fixing the calibration plate on the material adsorption platform and rotating it at five different angles within the corresponding camera field of view, then recording the image features and finally calculating the rotation center of the material adsorption platform. If there are two or more cameras on the material adsorption platform, the rotation center of the material adsorption platform (x2, y2) can be obtained by calculating the average value in sequence.

[0013] x=(x1-x2)cosθ-(y1-y2)sinθ+x2

[0014] y=(y1-y2)cosθ+(x1-x2)sinθ+y2

[0015] x2, y2 are the rotation centers, x1, y1 are the physical coordinates calculated by calibration, θ is the rotation angle, and x, y are the coordinates of the selected calibration point after rotating by θ around the rotation center (x2, y2).

[0016] S6, translate the calculated offsets xp and yp corresponding to the rotation center into the nine-point calibration matrix corresponding to each camera to obtain the final accurate physical mapping relationship between the material adsorption platform and the image coordinate system: Among them, xp, yp are the offsets corresponding to the rotation center, H a is the nine-point calibration matrix, H b is the final mapping matrix.

[0017] S7, by moving the material adsorption platform to the top of the camera, and then recording the reference coordinates of the material adsorption platform at this time, the obtained reference coordinates are stored, and the reference coordinates are used as the position reference of the corresponding camera.

[0018] S8, through the camera installed under the loading platform, the pixel coordinates of the object feature points are obtained by using the image processing algorithm, and then the pixel coordinates of the object feature points are converted into offset values ​​from the position reference obtained in step S7 using the calibration matrix calculated in step S6.

[0019] S9, the coordinate system of the rotation center of the mechanism obtained by the material adsorption platform in step S5 and the posture offset of the workpiece on the platform obtained in step S6 are calculated to obtain the (x, y, θ) offset relative to the set reference. The material loading platform is based on the reference set in step S6 and the new position obtained in step S7 and the (x, y, θ) offset from the set reference.

[0020] S10 , superimposing the offsets of the material adsorption platform and the loading platform calculated in step S9 to obtain a final motion amount (x, y, θ), and performing the assembly action according to the final motion amount.

[0021] A visual calibration method comprises the following steps: first, making a calibration plate with point coordinates, wherein the point coordinates are the coordinates of the point coordinates in a dot matrix, wherein the calibration plate coordinate system is the coordinate system in which the dot matrix is ​​located; placing the calibration plate in the field of view of each camera so that the camera can capture the calibration plate with the coordinate system dot matrix; controlling each camera to capture an image, reading the point coordinates on the calibration plate, and then converting the pixel coordinates of the read points into actual mechanical coordinates through nine-point calibration, and obtaining a radial transformation matrix from each camera coordinate system to the calibration plate coordinate system, at which point the calibration of the camera field of view and the mechanical platform is completed. (X t ,Y t ) represents the coordinates in the output coordinate system, (X i ,Y i ) represents the coordinates in the image coordinate system, (X o,Y o ) represents the position of the origin of the output coordinate system in the image coordinate system, α is the angle between the output coordinate system and the image coordinate system, and the conversion formula is:

[0022]

[0023] Since the calibration plate can calibrate the field of view and the platform coordinates, the point coordinate transfer carries the coordinate information of the point coordinate transfer center in the calibration plate coordinate system. During the calibration process, there is no need to fill in the coordinates of the grid corner points. Therefore, the above calibration process is relatively simple and can be completed without the need for professional personnel.

[0024] The calibration plate is based on a CTP (rubber anti-scorch agent) substrate. This CTP-based standard plate features a low coefficient of thermal expansion, high strength, high hardness, excellent wear resistance, low thermal conductivity, and good acid and alkali resistance. Its well-treated surface diffuse reflectance eliminates the problem of glare from glass calibration plates in front-light applications, enabling better identification of the calibration plate's pattern details, resulting in higher calibration and measurement accuracy.

[0025] Therefore, the core of using the CTP calibration plate with point coordinates is simplicity and efficiency, which can cope with various complex scenarios.

[0026] The calibration plate of the present invention is easy to manufacture and has low cost. Each local coordinate system pattern group can be used to establish a correspondence between the pattern of the calibration plate image and the real object, which greatly reduces the requirements for the calibration plate image during calibration and improves the calibration accuracy and efficiency. The calibration method is easy to execute on a computer, has low requirements for the camera and the captured calibration plate image, is not prone to errors, improves the calibration efficiency, and has high calibration accuracy.

[0027] For accurate calibration, the camera model is ideally constrained when the camera sees the calibration target filling most of the image. In layman's terms, using a small calibration plate, many combinations of camera parameters can explain the observed image. As a rule of thumb, the area of ​​the calibration plate should be at least half the available pixel area when viewed head-on. Calibration plates with point coordinates overcome the limitation of traditional origin calibration plates, which must capture images within 1 / 4 of the field of view, making them more convenient for use in a variety of environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a design drawing of a calibration plate according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of a calibration reference position of an upper unit according to an embodiment of the present invention;

[0030] Figure 3 is a schematic diagram of a calibration reference position of a lower unit according to an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the installation process of the upper and lower unit workpieces according to an embodiment of the present invention;

[0032] Figure 5 This is a flow chart of a method for visual calibration and alignment lamination of special-shaped products according to an embodiment of the present invention;

[0033] Figure 6 is a flow chart of a visual calibration method according to an embodiment of the present invention;

[0034] Figure 7 It is a partial schematic diagram of the calibration plate in the visual calibration method;

[0035] Figure 8 It is a schematic diagram of the application scenario of the visual calibration method;

[0036] Figure 9 It is a schematic diagram of the application scenario of the visual calibration method;

[0037] Figure 10 This is a schematic diagram of the layout of the camera and calibration board in the visual calibration method. DETAILED DESCRIPTION

[0038] The present invention is further illustrated below with reference to specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0039] like Figure 1 As shown, a visual calibration and alignment method for the bonding of special-shaped products is described, wherein the material adsorption platform is a platform that can perform xyθ movements, the loading platform is a supporting platform for fixing assembled objects, and the calibration plate is a set of dot matrix calibration plates with digital physical coordinate information, and the points are arranged at equal intervals.

[0040] like Figure 5 As shown, a method for visual calibration and alignment of special-shaped product lamination includes the following steps:

[0041] S1: The material adsorption platform moves to the location of the calibration plate to collect the calibration plate image; the calibration plate is placed parallel to the platform movement direction, and the calibration plate coordinate system direction is consistent with the material adsorption platform movement direction. It must be ensured that the camera field of view contains at least one complete dot matrix set area with digital coordinate information when in the photo position.

[0042] First, the material adsorption platform is in the calibration reference position, specifically, see Figure 2(The upper unit in the figure refers to the material adsorption platform), wherein, by adjusting the image acquisition device (ie, the camera), the field of view of the image acquisition device (ie, the camera) can clearly obtain the calibration plate of the material adsorption platform similar to Figure 1 image.

[0043] S2: Select 9 points randomly according to the image in the camera field of view, set the coordinate data, and complete the conversion relationship between the mechanism (material adsorption platform) coordinate system and the image coordinate system.

[0044] See Figure 1 , set the position of 9 calibration points, and then set the coordinate data. The calculation here is mainly to build the relationship between camera pixels and physical distance Pixel coordinates [xy 1] in the image and physical coordinates

[0045] The relationship between [XY 1] After expansion, we get Among them, a, b, c, d, e, and f represent the rotation and translation relationship between the camera pixel coordinates and the physical coordinates. According to the linear equation system, six equations are needed to calculate the calibration matrix T. Here, 9 points can construct 18 equations, using the minimum variance Find the corresponding 6 values, where Ax i It represents the new value obtained by combining one set of pixel coordinates with the physical coordinate set, and β represents the value under ideal conditions.

[0046] S3: Move the material adsorption platform to transfer the calibration plate to the loading platform so that the camera corresponding to the loading platform acquires the image of the calibration plate.

[0047] See Figure 3 , translate the coordinate plate to the loading platform, so that the cameras of the upper and lower units can uniformly describe the size and posture of an object. The upper unit in the figure refers to the material adsorption platform, and the lower unit refers to the loading platform.

[0048] S4: arbitrarily select 9 points based on the image within the camera field of view, set the calibration coordinate data, and complete the conversion relationship between the mechanism (loading platform) coordinate system and the image coordinate system.

[0049] See Figure 3 , according to the image, set the conversion relationship matrix of the physical distance between the camera pixel and the calibration plate in the same way as step 2.

[0050] S5: Set an image feature within the upper unit field of view to rotate the platform for five angles to calculate the rotation center of the material adsorption platform. If there are multiple cameras, calculate and average the angles to obtain the rotation center of the material adsorption platform.

[0051] See Figure 2This step is the most critical step. Since the execution of the platform action requires the execution of the material adsorption platform, the position of the rotation center of the mechanism will determine the translation calculation after the rotation. By recording the pixel coordinates of the feature points [xy 1] collected by the camera of the material adsorption platform, it is converted into the coordinates represented by the lock on the corresponding calibration plate. Here, 5 groups of points are selected and the least squares method is used to calculate the rotation center coordinates O(x, y) of the material adsorption mechanism. Then, the originally calculated offset matrix is ​​translated to the origin centered on o.

[0052] S6: Translate the calibration matrix of each camera according to the calculated rotation center.

[0053] S7: Set the position reference of each camera.

[0054] S8: Calculate the new position reference in each camera to calculate the posture information of the workpiece.

[0055] S9: Superimpose the workpiece posture information obtained by the upper unit and the lower unit and then move the upper unit.

[0056] S10: Execute the docking action.

[0057] See Figure 4 According to the information of the feature points of the upper and lower units collected by the upper and lower unit cameras, the center offset (Δx, Δy) and angle deviation Δθ of the upper and lower workpieces and the standard posture are calculated respectively. The translation amount of the two platforms is applied to the upper unit, and the angles of the two workpieces are superimposed to complete the transfer of the two workpieces.

[0058] In summary, the present invention provides a method for visual calibration and alignment of the bonding of special-shaped products. This method is to transfer the motion relationship coordinates of the upper unit to the immovable lower unit by transferring the calibration plate. This method does not require trial installation or a small amount of trial adjustment to complete the bonding of the product. For products of different shapes, due to the established standard and stable coordinate system, there is no need to transfer too much workpiece size information. At the same time, it can complete the precise measurement task of the workpiece based on the acquisition of the camera's installation position. In actual use, it can complete the partial bonding work of special-shaped products quickly, efficiently and economically.

[0059] A visual calibration method, such as Figure 6 As shown, the following steps are included:

[0060] S1, make a dot matrix calibration plate with digital coordinate information according to the size of the field of view.

[0061] In order to fit the field of view and be able to capture all 9 circular hole points, the circular hole size should be designed to be smaller than 1 / 4 of the field of view.

[0062] S2, place the calibration plate in the field of view of each camera.

[0063] like Figure 8 The calibration plate shown, each number represents the sequential coordinates of the hole where the center is missing, with Y in front and X in the back.

[0064] S3 controls the camera in each field of view to take pictures, selects the dot matrix area, binarizes it, and extracts the corresponding circular hole area through the clumping algorithm. The digital coordinates corresponding to the selected dot matrix area are filled in, and the affine transformation matrix from the camera pixel coordinate system to the calibration plate is obtained through the nine-point calibration algorithm.

[0065] Capturing images in the camera Figure 8 As shown in the figure, (after entering the camera calibration interface, a draggable ROI rectangular box will be automatically generated on the interface), drag the ROI control box to place it on the dot matrix area (generally, as long as the range contains nine circular holes and the numerical coordinate information is not blocked, it is considered appropriate), and then by setting the high and low thresholds of the grayscale value in the cluster processing algorithm, click the preprocessing button to see the final positioning result. The selected dots are covered by the corresponding circular cluster area, which means that the positioning threshold is set correctly (if there is no corresponding circular cluster area covering the dot matrix selected by the ROI box, or the cluster area exceeds the range of the dot matrix image, it is considered that the threshold setting is unreasonable and needs to be reset). After that, you only need to click the execute button, and the calibration algorithm will generate a calibration matrix according to the nine-point calibration algorithm based on the cluster area obtained through the above preprocessing operation, as shown in the figure. Figure 9 As shown, the 9 calibration points in the positioning box will be filled with numbers in sequence, completing the calibration of the physical relationship between the camera pixels and the coordinate system of the calibration plate.

[0066] like Figure 7 As shown in the figure, this calibration method uses a special dot matrix calibration plate with spatial coordinate information. By setting a set of 9 calibration points on the calibration plate, and then setting the coordinate data, the calculation here is mainly to build the relationship between the physical distance between the camera pixel and the area to be calibrated; the relationship between the pixel coordinate [xy 1] and the physical coordinate [XY 1] in the image Among them, a, b, c, d, e, and f represent the rotation and translation relationship from the camera pixel coordinates to the physical coordinates. After expansion, we get According to the linear equation system, six equations are needed to find the calibration matrix T. Here, 9 points can construct 18 equations, using the minimum variance Find the corresponding 6 values, where Ax i represents the new value obtained by solving the above equations, and b represents the value under ideal conditions.

[0067] like Figure 10As shown in the figure, when cameras are installed in different areas, in order to unify the coordinate systems of multiple cameras into one coordinate system, the traditional method requires solving a set of equations simultaneously, which is a cumbersome process. However, this experiment uses the position information on the calibration plate to directly unify multiple cameras into the same coordinate system, and the camera adjustment and calibration matrix settings are the same, which is convenient for users to operate. At the same time, the CTP plate is cheap, precise and wear-resistant, and can calibrate multiple cameras of large size and different orientations within a certain accuracy range, thereby completing the precision measurement of objects.

Claims

1. A visual calibration and alignment method for laminating special-shaped products, characterized by: The assembly object is transported through the xyθ material adsorption platform, and the material adsorption platform can move up and down and left and right, where up and down refers to the θ direction and left and right refers to the x and y directions; the assembly object is fixed by the loading platform; the calibration material adsorption platform is moved by the control mechanism to obtain the motion coordinate system of the control mechanism, and then a dot matrix calibration plate containing X and Y coordinate information is used to transfer the calibration plate to the immovable loading platform through the material adsorption platform, and the calibration information on the calibration plate is collected by at least one camera installed on the loading platform. By arbitrarily selecting 9 coordinate points in the collected calibration plate image and setting the calibration plate coordinate data, the conversion relationship between the material adsorption platform coordinate system and the coordinate system of the camera field of view associated with the immovable loading platform is completed, and the calibration mapping matrix between the camera position and the loading platform is calculated, thereby obtaining the center reference position and reference angle of the installed workpiece. At this time, by controlling the corresponding angle and translation amount of the material adsorption platform movement, the workpiece can be installed according to the specified position; The calculated offsets xp and yp corresponding to the rotation center are translated into the nine-point calibration matrix corresponding to each camera to obtain the final accurate physical mapping relationship between the material adsorption platform and the image coordinate system: Among them, xp, yp are the offsets corresponding to the rotation center, H a is the nine-point calibration matrix, H b is the final mapping matrix; By moving the material adsorption platform to the top of the camera, the reference coordinates of the material adsorption platform at this time are recorded and stored, and the reference coordinates are used as the position reference of the corresponding camera; The camera installed under the loading platform is used to image and the pixel coordinates of the feature points of the object are obtained by using the image processing algorithm. The pixel coordinates of the feature points of the workpiece at the debugging installation position and the new loading position on the loading platform are recorded and mapped through the mapping matrix H. b Obtaining the position offset and angle offset between the new loading position and the debugging installation position; Through the camera imaging below the material adsorption platform, the image processing algorithm is used to obtain the pixel coordinates of the feature points on the loading object. Based on the pixel coordinates, the debugging reference position of the loading object on the material adsorption platform and the position offset and angle offset of the installed loading object are calculated; the material adsorption platform is moved to the loading position of the product on the loading platform to perform the assembly action.

2. The visual calibration and alignment laminating method for special-shaped products according to claim 1, characterized in that: Move the material adsorption platform to the location of the calibration plate to capture the calibration plate image. The calibration plate coordinate system direction is consistent with the movement direction of the material adsorption platform. It is necessary to ensure that the camera's field of view contains at least one complete dot matrix set area with digital coordinate information when taking pictures. The calibration plate is a dot matrix calibration plate with digital coordinate information.

3. The visual calibration and alignment laminating method for special-shaped products according to claim 1, characterized in that: Control the material adsorption platform to transfer the calibration plate to the loading platform, so that the camera corresponding to the transfer platform can obtain the image of the calibration plate; Select 9 coordinate points at random on the image within the camera field of view, set the calibration coordinate data, and complete the correlation calculation matrix conversion relationship between the loading platform and the material adsorption platform through the nine-point calibration algorithm. (X t ,Y t ) represents the coordinates of the output loading platform, (X i ,Y i ) represents the image coordinates in the camera field of view of the loading platform, (X o ,Y o ) represents the position of the output loading platform origin in the image coordinate system, α is the angle between the output loading platform coordinate system and the image coordinate system, and the conversion formula is: The calibration plate is fixed on the material adsorption platform and rotated at five different angles within the corresponding camera field of view. The image features are then recorded to finally calculate the rotation center of the material adsorption platform. If there are two or more cameras on the material adsorption platform, the rotation center (x2, y2) of the material adsorption platform can be obtained by calculating the average value in sequence. x=(x1-x2)cosθ-(y1-y2)sinθ+x2 y=(y1-y2)cosθ+(x1-x2)sinθ+y2 x2, y2 are the rotation centers, x1, y1 are the physical coordinates calculated by calibration, θ is the rotation angle, and x, y are the coordinates of the selected calibration point after rotating by θ around the rotation center (x2, y2).

4. The visual calibration and alignment laminating method for special-shaped products according to claim 1, characterized in that: The calibration plate is a calibration plate with a CTP substrate.

Images