Multi-lens Circular Image Mosaic Algorithm for Glue Coating Defect Detection Equipment

Abstract

The present invention discloses a multi-lens circular image stitching algorithm for glue application defect detection equipment, which relates to the technical field of visual inspection of glue application defects in intelligent manufacturing. In order to solve the problem that in the existing technology based on two types of features, namely the glue strip and the substrate, if only the stitching algorithm based on feature points is used, due to the small number of feature types and even no obvious features can be extracted from some images, the stability of the algorithm is poor. Step 1: Number the three cameras in sequence as the first camera, the second camera, and the third camera; Step 2: The position of the end of the glue gun in the image coincides with the center position of the circle constrained by the centers of the three camera images; Step 3: Establish a base coordinate system A0 at the center point of the glue gun end image, and establish coordinate systems A1, A2, and A3 at the centers of the respective camera images in sequence; Step 4: According to the above parameters, substitute specific numerical values; Step 5: Convert all pixel positions of the camera 1 image to A0, and convert all pixel positions of the second camera and the third camera images to A0.

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection of gluing defects, and particularly to a multi-lens circular image stitching algorithm for a gluing defect detection device. Background Art

[0002] In the field of visual inspection of gluing defects in intelligent manufacturing, an image collector in the form of a hollow structure is used, and a plurality of cameras are evenly distributed circumferentially on the collector to detect surface defects of an object. Surface defect detection uses advanced machine vision detection technology to detect defects such as spots, pits, scratches, color differences, and defects on the surface of a workpiece. Gluing defect detection also includes continuity, width, and position, etc.

[0003] In the images of visual inspection of gluing defects, since a plurality of cameras on the collector are circumferentially distributed and the glue gun passes through the middle through-hole of the collector, each photo taken by a camera is only a rectangular window within an angular range along the projection plane of the glue gun. There is partial overlap between the windows. According to the different gluing running directions, the glue strip only exists in 1-2 images. The area outside the glue strip is the base material of the glue application, which is generally a uniform texture. As Figure 2 shown, the image characteristics of the gluing scene generally include two types of features: the glue strip and the base. The glue strip has a relatively simple edge feature, and the features of the base are not obvious. Some of the pictures taken by the cameras only contain uniform texture features. For image stitching of this scene, if only the feature point-based stitching algorithm is used, due to the small number of feature types and even some images having no obvious features to extract, the algorithm stability is poor. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-lens circular image stitching algorithm for a gluing defect detection device to solve the problem raised in the above background art that for the existing two types of features of the glue strip and the base, if only the feature point-based stitching algorithm is used, due to the small number of feature types and even some images having no obvious features to extract, the algorithm stability is poor.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A multi-lens circular image stitching algorithm for a gluing defect detection device, including a collection device, a glue gun, and a camera group: The following steps are included:

[0006] Step 1: Number the camera group, which are the first camera, the second camera, and the third camera in sequence;

[0007] Step 2: The position of the end of the glue gun in the image coincides with the center position of the circle constrained by the image center of the camera group;

[0008] Step 3: Establish a base coordinate system A0 at the center point of the image at the glue gun end, and establish coordinate systems A1, A2, and A3 at the image centers of the camera group in sequence;

[0009] Step 4: According to the above parameters, substitute specific values.

[0010] Step 5: Convert all pixel positions of the first camera image to under A0, convert all pixel positions of the second camera image to under A0, and convert all pixel positions of the third camera image to under A0.

[0011] Step 6: Determine the overlapping part between the images, and perform feature extraction and matching, image registration, and image fusion within the small neighborhood range of the overlapping area.

[0012] Step 7: If the base is a uniform solid color and feature points cannot be extracted within the overlapping part of the two images and its small neighborhood, directly perform weighted fusion.

[0013] By adopting the above technical solution, first, the images captured by each camera are converted to the same coordinate system using the principle of coordinate transformation, and then feature extraction, registration, and stitching processing are performed on the images. For images with poor feature extraction quality, after image coordinate transformation, feature extraction and matching are not performed, and the overlapping part is directly fused and stitched.

[0014] Further, the glue gun is fixed inside the acquisition device. The camera group includes a first camera, a second camera, and a third camera, and they are equally spaced and distributed inside the acquisition device.

[0015] By adopting the above technical solution, the assembly dimensions between multiple cameras distributed in a circle are known and fixed, and the optical axes of each camera are parallel.

[0016] Further, the imaging range of the camera group is rectangular and evenly distributed at a 120° angle along the circumferential direction. The camera group takes the glue gun as the center, and there is an overlapping part in the coverage range of the camera group.

[0017] By adopting the above technical solution, the image stitching algorithm uses the spatial coordinate transformation method to convert the images of different cameras to the same coordinate system, and performs re-registration and fusion stitching on the overlapping parts.

[0018] Further, a base coordinate system A0 is established based on the center point of the image at the end of the glue gun in Step 5. Coordinate systems A1, A2, and A3 are sequentially established at the image centers of the camera group. The origin of A0 is located at the center of the image at the end of the glue gun, the y0 axis points to the origin of the coordinate system A1, and the x0 axis is along the clockwise tangent direction at the arc where it is located.

[0019] By adopting the above technical solution, a base coordinate system A0 is established at the center point of each camera distribution arc, and camera coordinate systems A1, A2, and A3 described relative to the base coordinate system A0 are established at the center points of each image, and the position coordinates of pixel points in each image in the camera coordinate system are converted to the A0 coordinate system.

[0020] Further, define A0 as the identity matrix I(4×4). Taking R = 30 as an example according to the layout of A1, A2, and A3 in the previous step, the values of each coordinate system are as follows:

[0021]

[0022]

[0023]

[0024]

[0025] By adopting the above technical solution, calculation formulas are obtained based on the values of each coordinate system in the A1, A2, and A3 layouts.

[0026] Further, the image coordinate position taken by the first camera is converted to the A0 coordinate system. The position of a point p1 in the image is described as [x1, y1] in A1, and it is expressed as p1 = [x1, y1, 0, 1] T , and the position coordinate of this point in the A0 coordinate system is p 01 , represented by [x 01 , y 01 , 0, 1] T to describe p 01 , then p 01 = A0A1p1. Taking x1 = 10 and y1 = 10 as an example, substituting into each variable gives:

[0027] x 01 = x1 = 10

[0028] y 01 = y1 + R = 40

[0029] Following the calculation principle of the previous step, the formula for converting the position of a point in the image taken by the second camera to the A0 coordinate system is as follows:

[0030]

[0031]

[0032] The formula for converting the position of a point in the image taken by the third camera to the A0 coordinate system is as follows:

[0033]

[0034]

[0035] By adopting the above technical solution, features are further extracted and registration is performed within a small neighborhood range of the overlapping part of the images.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] 1. For the multi-lens annular image stitching algorithm of the glue application defect detection device, according to the structural characteristics of the multi-camera lenses of the collector in the multi-camera annular image stitching algorithm of the glue application defect detection device, that is, the structural installation positions of each camera are fixed and the assembly accuracy is relatively high. By using the method of coordinate transformation, the images of each camera are converted to the same coordinate system, and relatively good stitching accuracy can be achieved. Then, features are further extracted and matched within a small neighborhood range of the overlapping part of the images, which can improve the stitching accuracy and robustness of the algorithm. For a uniform substrate in the glue application scenario, matching feature points may not be extractable, in which case feature extraction and matching can be skipped and image fusion can be directly performed, which also has relatively good stitching accuracy and high algorithm robustness.

[0038] 2. For the multi-lens annular image stitching algorithm of the glue application defect detection device, since feature extraction is only performed within a small neighborhood range of the overlapping part of the images, the calculation amount is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic structural diagram of the collector of the present invention;

[0040] Figure 2 It is a schematic diagram of the collector shooting of the present invention;

[0041] Figure 3 It is a schematic diagram of the coordinate system established for the images shot by the collector of the present invention;

[0042] Figure 4 It is a schematic diagram of the position dimensions of the coordinate system of the present invention;

[0043] Figure 5 It is a schematic flowchart of the stitching algorithm of the embodiment of the present invention.

[0044] In the figure: 1. Acquisition device; 2. Glue gun; 3. Camera group; 4. First camera; 5. Second camera; 6. Third camera. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] A multi-lens circular image stitching algorithm for glue application defect detection equipment. The present invention provides the following technical solutions:

[0047] As Figures 1-5 shown, it includes the following steps:

[0048] Step 1: Number the camera group 3, which are the first camera 4, the second camera 5, and the third camera 6 in sequence;

[0049] Step 2: The position of the end of the glue gun 2 in the image coincides with the center position of the circle constrained by the image center of the camera group 3;

[0050] Step 3: Establish a base coordinate system A0 at the center point of the image at the end of the glue gun 2, and establish coordinate systems A1, A2, and A3 at the image centers of the camera group 3 in sequence;

[0051] Step 4: According to the above parameters, substitute specific values;

[0052] Step 5: Convert all pixel positions of the image of the first camera 4 to A0, convert all pixel positions of the image of the second camera 5 to A0, and convert all pixel positions of the image of the third camera 6 to A0;

[0053] Step 6: Judge the overlapping part between the images, and perform feature extraction and matching, image registration, and image fusion within a small neighborhood range of the overlapping area;

[0054] Step 7: If due to the base being a uniform solid color, feature points cannot be extracted within the overlapping part of the two images and its small neighborhood, directly perform weighted fusion;

[0055] The glue gun 2 is fixed in the acquisition device 1. The camera group 3 includes the first camera 4, the second camera 5, and the third camera 6, and is evenly distributed within the acquisition device 1;

[0056] The imaging range of the camera group 3 is rectangular, and is evenly distributed at an angle of 120° along the circumferential direction. The camera group 3 takes the glue gun 2 as the center, and there is an overlapping part in the coverage range of the camera group 3;

[0057] Establish a base coordinate system A0 based on the center point of the image at the end of the glue gun 2 in step 5. Successively establish coordinate systems A1, A2, and A3 at the image centers of the camera group 3. The origin of A0 is located at the image center at the end of the glue gun 2. The y0 axis points to the origin of the coordinate system A1, and the x0 axis is along the clockwise tangent direction at the arc where it is located.

[0058] Define A0 as the identity matrix I(4×4) according to the layouts of A1, A2, and A3 in the previous step.

[0059] Convert the image coordinate position captured by the first camera 4 to the A0 coordinate system. The position of a point p1 in the image is described as [x1, y1] in A1, and it is expressed as p1 = [x1, y1, 0, 1]. T , and the position coordinate of this point in the A0 coordinate system is p 01 , represented by [x 01 , y 01 , 0, 1] T to describe p 01 , then there is p 01 = A0A1p1.

[0060] The multi-camera circular image stitching algorithm for the glue application defect detection device aims at the structural characteristics of the multi-camera lenses of the collector, that is, the structural installation positions of each camera are fixed and the assembly accuracy is high. By using the method of coordinate transformation to convert the images of each camera to the same coordinate system, better stitching accuracy can be achieved. Then, feature extraction and matching are further carried out in the small neighborhood range of the overlapping part of the images, which can improve the stitching accuracy and robustness of the algorithm. For a uniform substrate in the glue application scenario, it may not be possible to extract matching feature points, so feature extraction and matching can be skipped and image fusion can be directly performed, which also has good stitching accuracy and high algorithm robustness.

[0061] Working principle: Install the acquisition device 1 on the glue application robot. The robot controls the operation of the glue application program. When the glue application robot is working, the camera group 3 simultaneously acquires and captures images. First, number the camera group 3 respectively. The position of the end of the glue gun 2 in the image coincides with the center position of the circle constrained by the image centers of the camera group 3, as Figure 3As shown in the figure, a base coordinate system A0 is established at the center point of the image at the end of the glue gun 2, and coordinate systems A1, A2, and A3 are established at the centers of each camera image in turn. Assuming that the value of the radius R of the arc where the origin is located is 30 mm, then all pixel positions of the image of the first camera 4 are converted to A0, all pixel positions of the image of the second camera 5 are converted to A0, and all pixel positions of the image of the third camera 6 are converted to A0, so as to judge the overlapping part between the images. Feature extraction and matching, image registration, and image fusion are carried out within the small neighborhood range of the overlapping area. If no feature points can be extracted in the overlapping part of the two images and its small neighborhood due to the base being a uniform solid color, then weighted fusion is directly carried out, and thus the image stitching algorithm is completed.

[0062] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be encompassed within the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

Claims

1. A multi-lens circular image stitching algorithm for glue application defect detection equipment, characterized in that: It includes a collection device (1), a glue gun (2), and a camera group (3); it includes the following steps: Step 1: Number the camera group (3) as the first camera (4), the second camera (5), and the third camera (6) in sequence; Step 2: The position of the end of the glue gun (2) in the image coincides with the center position of the circle constrained by the image center of the camera group (3); Step 3: Establish a base coordinate system A0 at the center point of the image at the end of the glue gun (2), and establish coordinate systems A1, A2, and A3 at the image centers of the camera group (3) in sequence; Step 4: According to the above parameters, substitute specific values; Step 5: Convert all pixel positions of the image of the first camera (4) to under A0, convert all pixel positions of the image of the second camera (5) to under A0, and convert all pixel positions of the image of the third camera (6) to under A0; Step 6: Judge the overlapping part between the images, and perform feature extraction and matching, image registration, and image fusion within the small neighborhood range of the overlapping area; Step 7: If the base is a uniform solid color and feature points cannot be extracted in the overlapping part of the two images and its small neighborhood, directly perform weighted fusion.

2. A multi-lens circular image stitching algorithm for a glue application defect detection device according to claim 1, characterized in that: The glue gun (2) is fixed inside the collection device (1), the camera group (3) includes the first camera (4), the second camera (5), and the third camera (6), and they are evenly distributed inside the collection device (1).

3. The multi-lens circular image stitching algorithm for a glue application defect detection device according to claim 1, characterized in that: The imaging range of the camera group (3) is rectangular, and it is evenly distributed at an angle of 120° along the circumferential direction. The camera group (3) takes the glue gun (2) as the center, and there is an overlapping part in the coverage range of the camera group (3).

4. The multi-lens circular image stitching algorithm for a glue application defect detection device according to claim 1, wherein: Based on the center point of the image at the end of the glue gun (2) described in Step 5, establish a base coordinate system A0, and establish coordinate systems A1, A2, and A3 at the image centers of the camera group (3) in sequence. The origin of A0 is located at the center of the image at the end of the glue gun (2), the coordinate axis y0 points to the origin of the coordinate system A1, and the coordinate axis x0 is along the clockwise tangent direction at the arc where it is located.

5. A multi-lens circular image stitching algorithm for a glue application defect detection device according to claim 4, characterized in that: A0 is the unit matrix I(4×4), according to the layout of A1, A2, and A3 described in the previous step.

6. The multi-lens circular image stitching algorithm for a glue application defect detection device according to claim 1, characterized in that: The image coordinate position captured by the first camera (4) is converted to the A0 coordinate system. The position of a point p1 in the image is described as [x1, y1] in A1, and it is expressed as p1 = [x1, y1, 0, 1] T , and the position coordinate of this point after being converted to the A0 coordinate system is p 01 , which is described by [x 01 , y 01 , 0, 1] T for p 01 . Then, we have p 01 = A0A1p1.

Images