Image stitching system and method

Abstract

The present invention discloses an image stitching system and method for PCBA detection, comprising: a camera, a mechanism for fixing and adjusting the position of the camera and a light source, a moving device for moving the camera position, and a controller. The present invention sequentially shoots multiple images for stitching, and changes the stitching quality by adjusting the stitching parameters, so as to obtain a stitching image with high pixels and a large field of view, solve the problem of incomplete information shooting due to imaging range limitation, and improve the disadvantage of poor real-time performance of general image stitching.

Description

Technical Field

[0001] The present invention relates to the field of two-dimensional image detection, and in particular to an image stitching system and method. Background Art

[0002] PCBA has a large appearance area and small components. In order to detect the defects of device welding, a large field of view and high-pixel image are required for detection, which is difficult to meet with a single industrial camera, so image stitching technology is used. The current mainstream stitching technology uses image feature extraction to perform feature matching stitching and fusion. The application scenarios of this technology are very flexible, but the real-time performance is far from enough and cannot meet engineering needs. Summary of the invention

[0003] The object of the present invention is to provide an image stitching system and method to solve the above-mentioned multiple defects caused by the prior art.

[0004] An image stitching system, the system comprising: a camera arranged above an image acquisition station, a two-dimensional driving mechanism for driving the camera to move, and a height adjustment mechanism for adjusting the height of the camera, the height adjustment mechanism being connected to the two-dimensional driving mechanism, and the two-dimensional driving mechanism being connected to a host computer;

[0005] The two-dimensional driving mechanism drives the camera to move above the image acquisition station according to the displacement command transmitted by the host computer, and adjusts the vertical distance between the camera and the product to be inspected through the height adjustment mechanism. The host computer performs image stitching based on the images of each area of ​​the product to be inspected collected by the camera.

[0006] Preferably, the system further comprises a feeding mechanism, which comprises: a driving wheel driven by a feeding drive motor, a driven wheel connected to the driving wheel via a conveyor belt, and a carrier arranged on the conveyor belt for carrying the product to be inspected.

[0007] Preferably, the system further comprises a photoelectric sensor communicatively connected to the host computer. When the carrier moves to the image acquisition station, the photoelectric sensor sends a signal to the host computer, and the host computer controls the camera to start image acquisition.

[0008] Preferably, the height adjustment mechanism includes: a connecting plate, a gear fixed relative to the connecting plate, a rack meshing with the gear, and a handwheel for driving the gear to rotate; the connecting plate is fixed on the two-dimensional driving mechanism, the camera is fixedly connected to the rack through a slide, and the slide includes a concave slide fixedly connected to the connecting plate and a convex slide slidably connected to the concave slide.

[0009] Preferably, the two-dimensional driving mechanism includes a Y-axis linear module laid along the Y-axis direction and an X-axis linear module laid along the X-axis direction, the X-axis linear module is supported on the Y-axis linear module, the Y-axis linear module is connected to a Y-axis driving motor for driving the X-axis linear module to move along the Y-axis direction, and the X-axis linear module is connected to an X-axis driving motor for driving the camera to move along the X-axis direction.

[0010] Preferably, the camera is also connected to a coaxial light source coaxial with the camera lens.

[0011] An image stitching method, the method is implemented by using any of the above systems, and specifically comprises the following steps:

[0012] The camera collects images of each area of ​​the inspected product according to the preset stitching parameters;

[0013] The images of each area of ​​the inspected product are stitched together in sequence to obtain a complete image of the inspected product.

[0014] Preferably, the stitching parameters include: a camera shooting path, a stepping distance, a tilt offset, and a return error offset set according to the size of the product to be inspected;

[0015] The tilt offset includes:

[0016] X-axis tilt offset: used to correct the error of non-coincidence between the scanning X-axis and the image X-axis;

[0017] Y-axis tilt offset: used to correct the error of the scanning Y-axis and the image Y-axis not coinciding with each other;

[0018] The return error offset includes:

[0019] X-axis return error offset: used to correct the return error in the X direction;

[0020] Y-axis return error offset: used to correct the return error in the Y direction.

[0021] Preferably, the shape of the camera shooting path includes an "S" shape.

[0022] Preferably, during the image stitching process, when a blank portion is generated, the images are overlapped and covered, and the overlap amount is the ratio of the width of a single image.

[0023] The advantages of the present invention are as follows: the image stitching system can capture multiple images in sequence for stitching, and can change the stitching quality by adjusting the stitching parameters, thereby obtaining a stitched image with high pixels and a large field of view, solving the problem of incomplete information capture due to imaging range limitations, and improving the shortcoming of poor real-time performance of general image stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the device of the present invention.

[0025] Figure 2 Schematic diagram of the scanning path of the present invention.

[0026] Figure 3 Schematic diagram of the tilt offset of the present invention.

[0027] Figure 4 Schematic diagram of return error in the present invention.

[0028] Figure 5 Schematic diagram of the overlap amount in the present invention.

[0029] Figure 6 It is a partial cross-sectional schematic diagram of the height adjustment mechanism in the present invention.

[0030] Figure 7 It is a left side schematic diagram of the height adjustment mechanism in the present invention.

[0031] Among them: 1-fixed device platform, 2-servo motor, 3-convex belt, 4-X direction linear module, 5-Y direction linear module, 6-carrier plate, 7-photoelectric sensor, 8-coaxial light source, 9-camera, 10-knob, 11-fixed device, 12-connecting plate, 13-concave slide, 14-convex slide, 15-handwheel, 16-rack, 17-gear. DETAILED DESCRIPTION

[0032] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0033] like Figures 1 to 7 As shown, an image stitching system, the system comprises: a camera arranged above an image acquisition station, a two-dimensional driving mechanism for driving the camera to move, and a height adjustment mechanism for adjusting the height of the camera, the height adjustment mechanism is connected to the two-dimensional driving mechanism, and the two-dimensional driving mechanism is connected to a host computer;

[0034] The two-dimensional driving mechanism drives the camera to move above the image acquisition station according to the displacement command transmitted by the host computer, and adjusts the vertical distance between the camera and the product to be inspected through the height adjustment mechanism. The host computer performs image stitching based on the images of each area of ​​the product to be inspected collected by the camera.

[0035] In this embodiment, the system also includes a loading mechanism, which includes: a driving wheel driven by a loading drive motor, a driven wheel connected to the driving wheel through a conveyor belt, and a carrier arranged on the conveyor belt for carrying the product to be inspected.

[0036] In this embodiment, the system further includes a photoelectric sensor that is communicatively connected to the host computer. When the carrier moves to the image acquisition station, the photoelectric sensor sends a signal to the host computer, and the host computer controls the camera to start image acquisition.

[0037] In this embodiment, the height adjustment mechanism includes: a connecting plate, a gear fixed relative to the connecting plate, a rack meshing with the gear, and a handwheel for driving the gear to rotate; the connecting plate is fixed on the two-dimensional driving mechanism, the camera is fixedly connected to the rack through a slide, and the slide includes a concave slide fixedly connected to the connecting plate and a convex slide slidably connected to the concave slide.

[0038] In this embodiment, the two-dimensional driving mechanism includes a Y-axis linear module laid along the Y-axis direction and an X-axis linear module laid along the X-axis direction. The X-axis linear module is supported on the Y-axis linear module. The Y-axis linear module is connected to a Y-axis driving motor that drives the X-axis linear module to move along the Y-axis direction. The X-axis linear module is connected to an X-axis driving motor that drives the camera to move along the X-axis direction.

[0039] In this embodiment, the camera is also connected to a coaxial light source coaxial with the camera lens.

[0040] The system includes:

[0041] Fixing device platform 1;

[0042] Servo motor 2, used to move the camera, light source and conveyor belt, controlled by a PLC (Programmable Logic Controller) system;

[0043] Conveyor belt 3, for transferring PCBA to be inspected;

[0044] X-direction linear module 4;

[0045] Y direction linear module 5;

[0046] A carrier plate 6, used for loading test objects such as test PCBA;

[0047] Photoelectric sensor 7, used to sense the position of the carrier plate;

[0048] Coaxial light source 8 makes the brightness of the object surface uniform;

[0049] Camera 9 is a 5-megapixel Daheng MER-503-20GC-P industrial camera;

[0050] Knob 10, controls the camera height;

[0051] The device 11 for fixing the camera and the light source enables the camera lens and the light source to be coaxially fixed.

[0052] The specific implementation steps of the method are:

[0053] Step 1: The product to be inspected is fed through a conveyor belt, and the photoelectric detector detects the product and fixes the product to the inspection position;

[0054] Step 2: The staff inputs the stitching parameters, and the algorithm calculates the path and step distance of the camera and light source;

[0055] Step 3: PLC controls the camera to move according to the calculated path and step, and takes multiple images in sequence;

[0056] Step 4: While shooting, the central control machine uses a stitching algorithm to stitch images in multiple threads;

[0057] Step 5: According to the quality of the stitched image, the technician reasonably adjusts the stitching parameters and repeats steps 2, 3, and 4 until the appropriate parameters are adjusted;

[0058] Step 6: Perform image defect detection based on the stitched image.

[0059] Explanation of the moving path in step 2:

[0060] The number of small images is calculated according to the specified frame size. Figure 2 As shown in the figure, starting from the upper left corner, the images are taken in a zigzag pattern. Each time a picture is taken, it is passed to the stitching algorithm for stitching.

[0061] The main splicing parameters used in splicing are:

[0062] X-direction tilt offset: Figure 3 As shown, in order to correct the error of non-coincidence between the scanning X-axis and the image X-axis;

[0063] Y-direction tilt offset: Figure 3 As shown, in order to correct the error of the non-coincidence between the scanning Y axis and the image Y axis;

[0064] X-direction return error offset: Figure 4 As shown, the return error in the X direction is corrected;

[0065] Y direction return error offset: Figure 4 As shown, the return error in the Y direction is corrected;

[0066] Figure 3 After the error correction, blank areas will be generated, so the images need to be overlapped. The overlap rate is required here and is set to Figure 5 The amount of overlap shown is relative to the width of a single image.

[0067] Based on the above parameters, the single-step spacing of the camera movement can be calculated, and then the full image can be pieced together.

[0068] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. An image stitching system, characterized in that: The system comprises: a camera arranged above an image acquisition station, a two-dimensional driving mechanism for driving the camera to move, and a height adjustment mechanism for adjusting the height of the camera, wherein the height adjustment mechanism is connected to the two-dimensional driving mechanism, and the two-dimensional driving mechanism is connected to a host computer; The two-dimensional driving mechanism drives the camera to move above the image acquisition station according to the displacement instruction transmitted by the host computer, and adjusts the vertical distance between the camera and the product to be inspected through the height adjustment mechanism. The host computer performs image splicing based on the images of each area of ​​the product to be inspected collected by the camera; The camera is also connected to a coaxial light source coaxial with the camera lens; The host computer performs image stitching based on the images of each area of ​​the product to be inspected collected by the camera, including the following steps: The camera collects images of each area of ​​the inspected product according to the preset stitching parameters; Sequentially stitch the images of each area of ​​the inspected product to obtain a complete image of the inspected product; Wherein, the stitching parameters include: the camera shooting path, step distance, tilt offset and return error offset set according to the size of the product to be inspected; The tilt offset includes: X-axis tilt offset: used to correct the error of non-coincidence between the scanning X-axis and the image X-axis; Y-axis tilt offset: used to correct the error of the scanning Y-axis and the image Y-axis not coinciding with each other; The return error offset includes: X-axis return error offset: used to correct the return error in the X direction; Y-axis return error offset: used to correct the return error in the Y direction; The splicing parameters are adjusted and obtained by the following method: Input the current stitching parameters and calculate the movement path and stepping distance of the camera and coaxial light source; The PLC controls the camera to move according to the calculated path and step distance, and takes multiple images in sequence; While shooting, the central control machine uses a stitching algorithm to stitch images in multiple threads; Adjust the current stitching parameters according to the stitching image quality, use the adjusted stitching parameters as the current stitching parameters, repeat the above steps until the ideal image stitching quality is obtained, and use the corresponding stitching parameters as the final preset stitching parameters; During the image stitching process, when blank parts are generated, the images are overlapped and the overlap amount is the ratio of the width of a single image.

2. The image stitching system according to claim 1, characterized in that: The system further comprises a feeding mechanism, which comprises: a driving wheel driven by a feeding driving motor, a driven wheel connected to the driving wheel through a conveyor belt, and a carrier arranged on the conveyor belt for carrying the product to be inspected.

3. The image stitching system according to claim 2, characterized in that: The system also includes a photoelectric sensor that is communicatively connected to the host computer. When the carrier runs to the image acquisition station, the photoelectric sensor sends a signal to the host computer, and the host computer controls the camera to start image acquisition.

4. The image stitching system according to claim 1, characterized in that: The height adjustment mechanism includes: a connecting plate, a gear fixed relative to the connecting plate, a rack meshing with the gear, and a handwheel for driving the gear to rotate; the connecting plate is fixed on the two-dimensional driving mechanism, the camera is fixedly connected to the rack through a slide, and the slide includes a concave slide fixedly connected to the connecting plate and a convex slide slidably connected to the concave slide.

5. The image stitching system according to claim 1, characterized in that: The two-dimensional driving mechanism includes a Y-axis linear module laid along the Y-axis direction and an X-axis linear module laid along the X-axis direction. The X-axis linear module is supported on the Y-axis linear module. The Y-axis linear module is connected to a Y-axis driving motor that drives the X-axis linear module to move along the Y-axis direction. The X-axis linear module is connected to an X-axis driving motor that drives the camera to move along the X-axis direction.

6. The image stitching system according to claim 1, characterized in that: The shape of the camera shooting path includes an "S" shape.

Images