Moving part detection system and method using single vision and galvanometer cooperation

Abstract

The application discloses a mobile part detection system and method using single-vision and galvanometer cooperation, belongs to the technical field of part detection, and comprises a galvanometer module, a visual sensing module, a motion control module and a processing module. The galvanometer module comprises a rotating motor and a galvanometer, the galvanometer is connected with the rotating motor and rotates under the driving of the rotating motor, the visual sensing module comprises an industrial camera, the industrial camera is used for shooting through the galvanometer on the rotating motor, and the image of a measured part in a moving state on a conveying belt can be collected. The application can acquire a relatively complete image of the part by tracking and collecting the part image multiple times through single-vision technology and the galvanometer, can better perform edge processing on the part by performing gray processing and Gaussian convolution processing on the collected part image and then performing edge detection on the part image after convolution, and can more accurately judge whether the part has defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of part detection, and in particular to a mobile part detection system and method using monocular vision and a galvanometer. BACKGROUND

[0002] With the development of society, the effective combination of visual systems and robots in production and manufacturing enterprises reduces the consumption of manpower and material resources, speeds up the time, and improves the efficiency. Today, the application of visual technology has been popular in multiple fields. At present, visual technology is widely used in workpiece positioning, logistics sorting, and part assembly. However, the most commonly used technology in these fields is mature 2D visual technology, i.e., monocular camera, which has certain limitations. However, with the development of robot vision technology and the increasing improvement of part recognition and positioning technology, monocular cameras can achieve the desired purpose.

[0003] Traditional monocular cameras detect static planar objects, and are mainly used to determine whether the target has defects (such as small round holes, rectangular holes, etc.). With the development of production and manufacturing, if the parts on the assembly line are detected, the time taken to capture the parts on the conveyor belt by moving the camera will be too long, and the field of view will be limited during the shooting process, which will cause certain errors in the detection of parts. Therefore, a mobile part detection system using monocular vision and a galvanometer is proposed. SUMMARY

[0004] The technical problem to be solved by the present application is how to more effectively detect the parts on the conveyor belt. The present application provides a mobile part detection system using monocular vision and a galvanometer.

[0005] The present application solves the above technical problems by the following technical solutions. The present application comprises a galvanometer module, a vision sensing module, a motion control module, and a processing module. The galvanometer module comprises a rotary motor and a galvanometer. The galvanometer is connected to the rotary motor and rotates under the drive of the rotary motor. The vision sensing module comprises an industrial camera. The industrial camera captures images of the moving parts on the conveyor belt through the galvanometer on the rotary motor. The motion control module comprises a single-chip microcomputer and a control unit. The single-chip microcomputer is connected to the rotary motor through the control unit. The processing module comprises a computer. The computer processes the images of the measured parts collected by the industrial camera and performs defect detection. The single-chip microcomputer and the computer are in communication connection.

[0006] Further, the center of the industrial camera lens and the center of the rotary motor controlled galvanometer are at the same horizontal height, and the industrial camera is fixedly arranged.

[0007] Further, when the industrial camera captures the measured part, the conversion relationship between the world coordinates and the pixel coordinates is as follows:

[0008]

[0009] wherein t is a translation matrix, R is a rotation matrix, t x ,t y ,t z represents translation along the X, Y, Z axes of the world coordinate system; r1 represents the element obtained by multiplying the X axis of the world coordinate system with the X axis of the camera coordinate system, r2 represents the element obtained by multiplying the Y axis of the camera coordinate system with the X axis of the world coordinate system; r3 represents the element obtained by multiplying the Z axis of the camera coordinate system with the X axis of the world coordinate system; r4 represents the element obtained by multiplying the X axis of the camera coordinate system with the Y axis of the world coordinate system; r5 represents the element obtained by multiplying the Y axis of the camera coordinate system with the Y axis of the world coordinate system; r6 represents the element obtained by multiplying the Z axis of the camera coordinate system with the Y axis of the world coordinate system; r7 represents the element obtained by multiplying the X axis of the camera coordinate system with the Z axis of the world coordinate system; r8 represents the element obtained by multiplying the Y axis of the camera coordinate system with the Z axis of the world coordinate system; and r9 represents the element obtained by multiplying the Z axis of the camera coordinate system with the Z axis of the world coordinate system.

[0010] For example, assuming that the world coordinate system is A and the camera coordinate system is B, the following is obtained:

[0011]

[0012] Further, the industrial camera coordinate system under the mirror reflection surface of the mirror is rotated by the rotation motor, and the coordinate value calculation formula of the new point p' of the mirror after rotating by an angle θ is as follows:

[0013]

[0014] wherein X' W , Y' W , and Z' W are the three-axis coordinate values of the point p' in the world coordinates, X W , and Z W are the two-axis coordinate values of the point p in the world coordinates before rotation.

[0015] Further, the parts on the conveying belt are regarded as a parallel and sequential moving mode, and each batch of parts is regarded as a whole, and the batch of parts is detected, and the conveying belt process expression is as follows:

[0016]

[0017] Wherein, T is the total cycle of the batch of parts, n is the batch quantity of parts, m is the number of parts processing, t i is the processing time of the i-th part.

[0018] The present application also provides a moving part detection method using monocular vision and galvanometer cooperation, which is used for detecting defects of moving parts by using the moving part detection system described above, and includes the following steps:

[0019] S1: the galvanometer rotates under the driving of the rotating motor, and the industrial camera takes pictures through the galvanometer on the rotating motor, so as to collect images of the measured parts in a moving state on the conveyor belt;

[0020] S2: the collected part images are grayed and subjected to Gaussian convolution processing;

[0021] S3: the part images processed in step S2 are subjected to edge detection by a Canny operator, and whether the measured part has defects is judged according to the edge detection result.

[0022] Further, in the step S2, the specific process of convolution processing is as follows:

[0023] S21: the collected part images are subjected to gray processing;

[0024] S22: Gaussian smoothing function is used for convolution processing to obtain new pixel points, and then the effective information of the image is retained.

[0025] It should be noted that the defects of the measured part have two types: one is that the part produces cracks; the other is that the part produces scars in the production process.

[0026] Compared with the prior art, the moving part detection system using monocular vision and galvanometer cooperation has the following advantages: the system can obtain a relatively complete image of the part by tracking and collecting the part image multiple times through monocular vision technology and galvanometer, can better perform edge processing on the part by performing gray and Gaussian convolution processing on the collected part image and then performing edge detection on the convolved part image, and can better perform continuous edge processing and more accurately judge whether the part has defects. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a structural schematic diagram of the moving part detection system using monocular vision and galvanometer cooperation in the embodiment one of the present application;

[0028] Figure 2 is a geometric model schematic diagram of the imaging of the industrial camera in the embodiment one of the present application;

[0029] Figure 3 This is a schematic diagram illustrating the conversion process between the world coordinate system and the pixel coordinate system in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram (top view) of the cooperation between the industrial camera and the galvanometer in Embodiment 1 of the present invention;

[0031] Figure 5 This is a graph showing the relationship between the new coordinate value and the original coordinate value of point P after the rotating mirror of the rotating motor is at an angle θ in Embodiment 1 of the present invention.

[0032] Figure 6 This is a schematic diagram of the implementation process of the moving part detection method using monocular vision and galvanometer combined in Embodiment 2 of the present invention;

[0033] Figure 7(a) is an image of the part (original image) in Embodiment 2 of the present invention;

[0034] Figure 7(b) is a grayscale image of the part in Embodiment 2 of the present invention;

[0035] Figure 7(c) is a part image processed by Gaussian convolution in Embodiment 2 of the present invention;

[0036] Figure 7(d) is a part image after Canny edge detection in Embodiment 2 of the present invention.

[0037] exist Figure 1 The components include: 1. Microcontroller; 2. Rotary motor; 3. Galvanometer; 4. Part under test; 5. Industrial camera; 6. Computer; 7. Conveyor belt. Detailed Implementation

[0038] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0039] Example 1

[0040] like Figure 1 As shown, this embodiment provides a technical solution: a moving part inspection system employing monocular vision and galvanometer collaboration (hereinafter referred to as the "monocular vision galvanometer system"), mainly comprising a vision sensing module, a motion control module, and a processing module; the motion control module includes a microcontroller 1 and a control unit, used to control a rotary motor 2 to drive the galvanometer 3 to rotate; the vision sensing module is an industrial camera 5, which captures images of the parts being tested on the conveyor belt 7 through the galvanometer 3 on the rotary motor 2; the processing module is a computer 6, which processes the images of the parts being tested captured by the industrial camera 5. Figure 1As can be seen, the center of the lens of industrial camera 5 and the center of the galvanometer 3, which is controlled by rotary motor 2 to rotate, are at the same level. Industrial camera 5 is fixed in place, and the image of the part being tested is acquired by industrial camera 5 through the rotation of galvanometer 3 controlled by rotary motor 2. The rotation of galvanometer 3 allows for the capture of moving objects, thus avoiding blind spots caused by objects moving too fast or not being captured. Furthermore, Canny edge detection is performed on the part image acquired by industrial camera 5 to detect defects and ensure the part meets acceptance standards.

[0041] Regarding the perspective imaging model of industrial cameras: Based on the principle of pinhole imaging, the imaging position of any point in space in the image can be represented by a pinhole. There is a linear relationship between spatial coordinates (coordinates in the world coordinate system) and image coordinates, which can represent the planar relationship between the real scene in three-dimensional space and the two-dimensional image.

[0042] Geometric models of industrial camera imaging, such as Figure 2 As shown, where P(X) C ,Y C Z C Let P be a point in the camera coordinate system, p(X,Y) be the projection of point P onto the imaging plane, and P' be the projection of point P onto the camera coordinate system X. C OZ C The projection point of the surface, f is the camera focal length;

[0043] Based on the geometric model of industrial camera imaging, and using the principle of triangle similarity, the transformation relationship between the camera coordinate system and the image coordinate system shown in equation (1) can be obtained:

[0044]

[0045] Among them, Z C This represents the value of the camera's Z-axis coordinates, i.e., the distance from the target to the camera. (X,Y) represents any coordinate point in the image coordinate system. C ,Y C Z C () represents a three-dimensional coordinate point in the world coordinate system;

[0046] In the camera plane, the image pixel coordinate system and the image physical coordinate system together form the image coordinate system. If the coordinate point (u0, v0) is the position of the origin O of the image physical coordinate system in the image pixel coordinate system.

[0047] Let the physical size of each unit pixel in the x-axis and y-axis directions be d. x ,d yBased on the positional relationship between image coordinates and pixel coordinates, the transformation between image coordinates and pixel coordinates for any pixel (u,v) can be obtained, and thus the transformation between 3D coordinates and pixel coordinates can be derived; the transformation relationship is as follows:

[0048]

[0049] Where t is the translation matrix. R is the rotation matrix.

[0050] Since industrial camera coordinates are usually inconsistent with world coordinates, rotation and translation transformations are required to achieve the transformation between the two coordinate systems. The flowchart of the transformation relationship is as follows: Figure 3 As shown.

[0051] In this embodiment, the detection of moving objects (moving parts) is achieved by rotating a galvanometer driven by a rotary motor, thereby detecting the object's relative motion, and then capturing images with an industrial camera. Taking a camera and galvanometer as an example within the system, combined with a motor model, such as... Figure 4 As shown, when the part moves, the galvanometer rotates accordingly to capture images of the part. A rotary motor is used in conjunction with a microcontroller. Under the control of the rotary motor, the galvanometer rotates. For each pulse output by the microcontroller, the rotary motor controls the galvanometer to rotate by an angle to track the movement of the object, thereby tracking and capturing images of the part.

[0052] Assuming that point P, after rotation, has world coordinates calculated using the transformation relationship described above, the coordinates are P(X). W ,0,Z W The relationship between the new coordinates of point P after the rotating mirror of the rotating motor at an angle θ and the original coordinates is shown in the figure below. Figure 5 As shown, O W -x w y w z w Let P(X) be the world coordinate system centered on the rotating electric motor platform. W ,0,Z W Let P'(X') be a point on the line from the camera to the galvanometer. W ,Y' W ,Z' W Let P be the new coordinate position of point P after it rotates by an angle θ under the drive of the motor, where θ is the rotation angle.

[0053] Each time a part is transferred, the galvanometer is rotated at a certain angle and the object is photographed using the galvanometer to obtain a better image of the part being tested. The image is then processed by a computer to inspect the part.

[0054] The relationship between the rotation angle θ controlled by the rotary motor under the galvanometer and the fixed angle α when each part of the conveyor belt passes is as follows:

[0055] θ=n·α

[0056] Where n represents the number of conveyor belt parts.

[0057] Since point p' is in plane O w -x w y w The projection point on the light plane O, according to the geometric relationship in the above figure, is due to the light plane O w -x w The coordinates of point p within yw and the rotation angle of the galvanometer controlled by the motor can be used to calculate the coordinates of the new point p' after point p rotates by an angle θ. The formula is as follows:

[0058]

[0059] If we consider the parts on the (parallel) conveyor belt as moving in a parallel sequence, and if we consider each batch of parts as a whole, and if we are to inspect this batch of parts, the expression for the conveyor belt process (in parts manufacturing companies, where parts are transported and processed using a conveyor belt) is as follows:

[0060]

[0061] Where T is the total cycle time for this batch of parts, n is the batch size of the parts, m is the number of parts processed, and t is the total cycle time for the batch of parts. i Let be the processing time for the i-th part.

[0062] Example 2

[0063] In this embodiment, the acquired part images are processed by a computer, and experimental simulation is used to determine whether the tested part has defects. The specific process is as follows: Figure 6 As shown.

[0064] In this embodiment, the industrial camera is model A7A20MU201.

[0065] The experiment involved computer processing of images of parts captured by a monocular camera, and the resulting images are shown in the figure. Figures 7(a)-7(d) As shown.

[0066] The system tracks and acquires multiple images of the part using a monocular vision galvanometer system, obtaining relatively complete images. The acquired images are first converted to grayscale and then subjected to Gaussian convolution. Edge detection is then performed on the convolved part image. As can be seen from the above processing steps and results, this system can better process the edges of the part, exhibiting better continuity and enabling a more accurate determination of whether defects exist.

[0067] Edge detection can better detect the integrity of parts, while scars and cracks may be caused by collisions between parts or improper human operation. During edge detection, scars and cracks can be used as valid information to detect defects such as scars and cracks.

[0068] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A moving parts inspection system employing monocular vision and galvanometer in synergy, characterized in that, include: The system comprises a galvanometer module, a vision sensing module, a motion control module, and a processing module. The galvanometer module includes a rotary motor and a galvanometer, the galvanometer being connected to the rotary motor and rotating under its drive. The vision sensing module includes an industrial camera that captures images of the workpiece moving on a conveyor belt via the galvanometer on the rotary motor. The motion control module includes a microcontroller and a control unit, the microcontroller being connected to the rotary motor via the control unit. The processing module includes a computer that processes the images of the workpiece captured by the industrial camera and performs defect detection. The microcontroller is communicatively connected to the computer. The center of the industrial camera lens and the center of the galvanometer controlled by the rotary motor are at the same horizontal height, and the industrial camera is fixedly installed. The rotation of the galvanometer is driven by the rotary motor. The coordinates of the new point p' on the reflecting surface of the galvanometer in the industrial camera coordinate system after the galvanometer rotates by an angle θ are calculated using the following formula: Where X' W Y' W Z' W The three coordinates of point p' in world coordinates, X, Y, and Z. W Z W These are the two-axis coordinates of point p in world coordinates before rotation; The parts on the conveyor belt are considered to move in a parallel sequence, and each batch of parts is considered as a whole. The batch of parts is then inspected. The conveyor belt process expression is as follows: Where T is the total cycle time for this batch of parts, n is the batch size of the parts, m is the number of parts processed, and t is the total cycle time for the batch of parts. i Let i be the processing time for the i-th part; The moving parts inspection system described above is used to detect defects in moving parts, including the following steps: S1: The galvanometer rotates under the drive of the rotary motor, and at the same time the industrial camera takes pictures through the galvanometer on the rotary motor to realize the acquisition of images of the test parts that are moving on the conveyor belt. S2: Convert the acquired part images to grayscale and perform Gaussian convolution processing; S3: For the part image processed in step S2, perform edge detection using the Canny operator, and determine whether there are defects in the tested part based on the edge detection results; In step S2, the specific process of convolution processing is as follows: S21: Perform grayscale processing on the acquired part images; S22: The Gaussian smoothing function is used for convolution to obtain new pixels while preserving the effective information of the image.

2. The moving part detection system employing monocular vision and galvanometer coordination as described in claim 1, characterized in that: When the industrial camera captures images of the part under test, the transformation relationship between world coordinates and pixel coordinates is as follows: Where t is the translation matrix. R is the rotation matrix. t x ,t y ,t z The values ​​represent translations along the X, Y, and Z axes of the world coordinate system; r1 represents the element obtained by multiplying the X-axis of the world coordinate system by the X-axis of the camera coordinate system; r2 represents the element obtained by multiplying the Y-axis of the camera coordinate system by the X-axis of the world coordinate system; r3 represents the element obtained by multiplying the Z-axis of the camera coordinate system by the X-axis of the world coordinate system; r4 represents the element obtained by multiplying the X-axis of the camera coordinate system by the Y-axis of the world coordinate system; r5 represents the element obtained by multiplying the Y-axis of the camera coordinate system by the Y-axis of the world coordinate system; r6 represents the element obtained by multiplying the Z-axis of the camera coordinate system by the Y-axis of the world coordinate system; r7 represents the element obtained by multiplying the X-axis of the camera coordinate system by the Z-axis of the world coordinate system; r8 represents the element obtained by multiplying the Y-axis of the camera coordinate system by the Z-axis of the world coordinate system; and r9 represents the element obtained by multiplying the Z-axis of the camera coordinate system by the Z-axis of the world coordinate system.

Images