A plug-in method for special-shaped machines based on image recognition

Abstract

The present invention discloses a special-shaped machine plug-in method based on image recognition, including: a. obtaining a plug-in hole image of a PCB, and performing image processing on the image to obtain a hole position contour; b. calculating the position and angle of the plug-in hole; c. obtaining a pin image of a component; d. matching the pin image with a preset template, and performing image processing on the area matching the template to obtain a pin pattern; e. calculating the position and angle of the component pin according to the pin pattern; f. calculating the angle that needs to be compensated for the plug-in operation, and rotating the compensated angle to perform the plug-in operation. The present invention obtains the position and angle of the plug-in hole by shooting an image of the PCB plug-in hole, and at the same time, obtains the position and angle of the component pin by shooting an image of the component pin. When performing the plug-in operation, the angle of the plug-in hole and the angle of the component pin are superimposed to form a compensation angle, thereby realizing accurate plug-in operation and ensuring product quality.

Description

Technical Field

[0001] The invention relates to the technical field of special-shaped machine plug-in, and in particular to a special-shaped machine plug-in method based on image recognition. Background Art

[0002] A special-shaped plug-in machine is a mechanical device that automatically inserts some regular electronic components into the conductive through-holes of a PCB. During the plug-in operation, in order to realize the automatic insertion of electronic components, the special-shaped plug-in machine needs to perform programmed operations according to the position of the electronic components, which requires the plug-in machine to accurately insert the components to be plugged into the plug-in holes. However, during the plug-in process, the position and angle of the plug-in holes are usually affected by various factors and offset. In addition, when the plug-in machine absorbs electronic components, the position of the pins of the components may change or the angle may shift due to the influence of some factors. Once the position or angle of the plug-in hole or the pin of the component changes, it is difficult for the plug-in machine to accurately complete the plug-in operation, which not only affects the plug-in efficiency, but also has a serious impact on the quality of the product. Therefore, how to make the plug-in machine's precise plug-in operation become an objective need. Summary of the invention

[0003] The present invention provides a special-shaped machine plug-in method based on image recognition, which is used to solve the problem that the plug-in operation of the existing plug-in machine affects product quality due to low plug-in accuracy.

[0004] To achieve the purpose of the present invention, the present invention provides a special-shaped machine plug-in method based on image recognition, the method comprising the following steps:

[0005] a. Obtain the plug-in hole image of the PCB and perform image processing on the image to obtain the hole position contour;

[0006] b. Calculate the position and angle of the plug-in hole according to the hole profile;

[0007] c. Obtain the pin image of the component;

[0008] d. Matching the pin image with a preset pin image template, and performing image processing on the area matching the template to obtain the pin pattern;

[0009] e. Calculate the position and angle of the component pins based on the pin pattern;

[0010] f. When plugging in, superimpose the plug-in hole angle of step b and the component angle of step e, calculate the angle that needs to be compensated for the plug-in operation, and rotate the compensated angle to perform the plug-in operation.

[0011] In step a, the step of performing image processing on the image to obtain the hole position contour includes:

[0012] a1. Performing Gaussian filtering on the image;

[0013] a2. Detect the edges in the image using the Canny operator;

[0014] a3. Obtain all hole contours in the image according to the edge.

[0015] Step b includes:

[0016] b1. Extract the position information of each plug-in hole according to the hole contour;

[0017] b2. Calculate the center coordinates of the entire image based on the position of each plug-in hole, and then obtain the actual coordinates based on the center coordinates, which is the position of the plug-in hole;

[0018] b3. Obtain the convex hull of the plug-in hole of the image, and calculate the angle of the plug-in hole based on the convex hull of the plug-in hole.

[0019] Furthermore, in step b3, the plug-in hole convex hull is the smallest polygon formed by all the plug-in holes in the image, and the plug-in hole angle is the included angle between the plug-in hole convex hull and the preset standard plug-in hole convex hull.

[0020] In step c, the method for obtaining the pin image of the component is: the head of the special-shaped machine passes through six predetermined shooting positions in sequence, and when the head of the special-shaped machine enters a certain range of the predetermined position, the camera is triggered to shoot and record the actual position of the current xy axis.

[0021] Step d includes:

[0022] d1. Pre-photograph templates of different component pin images, obtain the pin convex hull of each template, and perform a horizontal operation on the template pin convex hull;

[0023] d2. Match the pin image with a preset pin image template;

[0024] d3. If the match is successful, the area matching the template is image processed to obtain the pin pattern.

[0025] Furthermore, in step d3, the step of performing image processing on the area matching the template to obtain the foot print includes:

[0026] d31, perform Gaussian filtering on the area image matching the template;

[0027] d32, detect the edge in the image through the Canny operator;

[0028] d33. Acquire all contours in the image according to the edge, so as to obtain the pin pattern.

[0029] Further, step d32 includes:

[0030] d321, calculate the gradient magnitude and gradient direction of the image by applying the Sobel operator;

[0031] d322, scan the image, retain the local maximum point of the local gradient amplitude, and suppress the non-maximum value;

[0032] d323, set two thresholds to threshold the image that is not the maximum threshold;

[0033] d324. Further refine the image to remove irrelevant edge segments and disconnected edge segments, thereby obtaining the image edge.

[0034] Step e includes:

[0035] e1. forming a pin convex hull of the component pin image based on the component pin pattern, and performing a horizontal operation on the pin convex hull;

[0036] e2. Match the horizontal pin convex hull with the template pin convex hull preset in step d1. If the match is successful, a matched component image is obtained; if the match is unsuccessful, the material is discarded;

[0037] e3. Calculate the center coordinates of the entire pin according to the actual position of each pin pattern in the successfully matched component image, and determine whether the pin offset is too large according to the actual position of each pin. If the offset is too large, perform material removal.

[0038] e4. Calculate the offset value of the actual coordinates of the component relative to the shooting position according to the pixel coordinates of the component in the image, the camera calibration information and the shooting position, and calculate the position of the component pin based on the offset value;

[0039] e5. Calculate the angle of the component according to the pin convex hull of the component pin image, where the angle of the component is the included angle between the pin convex hull of the component pin image and the pin convex hull of the template.

[0040] Furthermore, in step e2, the step of performing a horizontal operation on the pin convex hull includes:

[0041] The horizontalization operation is performed based on the long side and the short side of the pin convex hull. If the pin pattern in the pin convex hull is an asymmetric pin, the long side includes a first long side and a second long side.

[0042] The beneficial effects of the present invention are as follows: the image recognition-based special-shaped machine plug-in method of the present invention obtains the position and angle of the plug-in hole by taking an image of the PCB plug-in hole, and at the same time obtains the position and angle of the component pin by taking an image of the component pin. When performing the plug-in operation, the angle of the plug-in hole and the angle of the component pin are superimposed to form a compensation angle, thereby realizing accurate plug-in operation and ensuring product quality.

[0043] On the other hand, by matching the image of the component pin with a template that has pre-captured images of different component pins, components with poor size or structure can be screened out; further, by matching the pin convexity of the component with the pin convexity of the component template, components with excessive pin deviation can be screened out, which not only ensures the quality of the component but also improves the plug-in efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the method of the present invention.

[0045] Figure 2 It is the plug-in hole image of the PCB of the present invention.

[0046] Figure 3 It is the hole position profile of the plug-in hole of the present invention.

[0047] Figure 4 It is a plug-in hole convex hull formed by the plug-in hole of the present invention.

[0048] Figure 5 is a pin image of the component of the present invention.

[0049] Figure 6 The present invention relates to a template pin convex bump formed by the pin template of the component.

[0050] Figure 7 is the pin pattern of the component of the present invention.

[0051] Figure 8 It is a pin convex hull formed by the pin image of the component of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] See also Figure 1The special-shaped machine plug-in method based on image recognition in this embodiment recognizes the position and angle of the plug-in hole and the position and angle of the component pin, and then performs a compensation plug-in operation according to the position and angle of the plug-in hole and the component pin, thereby realizing a precise plug-in operation. The special-shaped machine plug-in method includes the following steps:

[0054] S10, acquiring the plug-in hole image of the PCB in real time, and performing image processing on the acquired image to obtain the hole position contour.

[0055] In this step, a high-precision industrial camera is used to capture images of PCB plug-in holes in real time, and the real-time captured images are processed to obtain the hole position contours, such as Figure 2 The image shown is an image of the plug-in hole taken at a certain moment, such as Figure 3 The image shown is Figure 2 The hole position contour of the image is obtained by processing the real-time captured image.

[0056] S11, performing Gaussian filtering on the image.

[0057] In this step, the image is Gaussian filtered to reduce the variation of pixel values ​​in the image by weighted averaging, thereby blurring the image and reducing the impact of image noise.

[0058] S12. Detect edges in the image using the Canny operator.

[0059] This step includes:

[0060] S121. Calculate the gradient magnitude and gradient direction of the image by applying the Sobel operator.

[0061] S122, scanning the image, retaining the local maximum point of the local gradient amplitude, and suppressing the non-maximum value.

[0062] In this step, the detected edge is refined by retaining the local maximum point of the local gradient amplitude and suppressing the non-maximum value, so that it matches the actual edge more accurately.

[0063] S123, setting two thresholds, and thresholding the image that is not the maximum threshold.

[0064] In this step, two thresholds, i.e., a high threshold and a low threshold, are set to threshold the image after non-maximum suppression. Specifically, pixels with gradient magnitudes higher than the high threshold are marked as strong edges, and pixels with gradient magnitudes lower than the low threshold are marked as weak edges. Starting from the strong edge pixels, the connected weak edge pixels are tracked and marked as strong edges until they can no longer be connected to new edge pixels.

[0065] S124, further thinning the image to remove irrelevant edge segments and disconnected edge segments, thereby obtaining the image edge.

[0066] S13, obtaining all hole contours in the image according to the edges.

[0067] S20, calculating the position and angle of the plug-in hole according to the hole profile.

[0068] In this step, since the plug-in hole may not be set in the direction preset by the standard plug-in hole, the plug-in hole and the direction preset by the standard plug-in hole may form a certain angle, which is the angle of the plug-in hole and needs to be compensated in the subsequent plug-in operation. It specifically includes:

[0069] S21. Extracting the position information of each plug-in hole according to the hole position contour.

[0070] S22, calculating the center coordinates of the entire image according to the position of each plug-in hole, and then obtaining the actual coordinates according to the center coordinates, which are the positions of the plug-in holes.

[0071] In this step, if Figure 3 As shown, the center coordinates of the entire image are calculated according to the position of each plug-in hole, and then the pixel coordinates are converted into actual coordinates according to the calibration parameters of the camera and the shooting position. The actual coordinates obtained are the positions of the plug-in holes.

[0072] S23, obtaining the convex hull of the plug-in hole of the image, and calculating the angle of the plug-in hole according to the convex hull of the plug-in hole.

[0073] In this step, the minimum polygon composed of all plug-in holes in the image forms the plug-in hole convex hull, such as Figure 4 In the embodiment shown, the plug-in hole convex hull is a rectangle, and the angle of the plug-in hole is calculated based on the plug-in hole convex hull. The angle of the plug-in hole is the angle between the plug-in hole convex hull and the preset standard plug-in hole convex hull. Specifically, the image of the standard plug-in hole is obtained in advance, and the standard plug-in hole convex hull is formed for all the plug-in holes in the standard plug-in hole image (not shown in the figure), and the plug-in hole convex hull is matched with the standard plug-in hole convex hull. After the match is successful, the long side or short side of the standard plug-in hole convex hull is used as a reference, and the angle between the long side or short side of the plug-in hole convex hull and the long side or short side of the standard plug-in hole convex hull is the angle of the plug-in hole. In order to avoid rotation operation, if the plug-in hole in the plug-in hole convex hull is an asymmetric structure, it is necessary to distinguish different long sides. Specifically, the long sides can be divided into a first long side and a second long side.

[0074] S30, obtaining a pin image of a component.

[0075] In this step, the method for obtaining the pin image of the component is: the head of the special-shaped machine passes through six predetermined shooting positions in sequence. When the head of the special-shaped machine enters a certain range of the predetermined position, the camera is triggered to shoot and record the actual position of the current xy axis. The pin image of the component shot is as follows: Figure 5 As shown in the figure, when the pin image is captured, 6 sets of working heads and 6 components pass over the camera at a speed of 2m / s without stopping. The rotation center of the shaft is moved to the center of the camera image, and the current position is recorded to obtain the predetermined shooting position of the shaft.

[0076] S40, matching the pin image with a preset pin image template, and performing image processing on the area matching the template to obtain a pin pattern.

[0077] This step includes:

[0078] S41, pre-photographing templates of pin images of different components, obtaining the pin convex hull of each template, and performing a horizontal operation on the pin convex hull of the template.

[0079] In this step, different types of pin image templates are pre-photographed to facilitate template matching of the pin images to be taken later. Figure 6 As shown, a convex hull is formed for the pin image template, and the convex hull of the template pin is horizontalized. The convex hull of the template pin forms a rectangle including all pin patterns with the center of the outermost pin pattern.

[0080] S42, matching the pin image with a preset pin image template.

[0081] In this step, the captured pin image is matched with the templates of all pin images preset in step S41. If the match is successful, step S43 is executed; if the match is unsuccessful, it means that the component corresponding to the pin image deviates from the standard pin image and does not meet the requirements, and the component corresponding to the pin image needs to be discarded.

[0082] S43. If the match is successful, image processing is performed on the area matching the template to obtain a pin pattern.

[0083] This step includes:

[0084] S431, performing Gaussian filtering on the area image that matches the template.

[0085] In this step, the image is Gaussian filtered to reduce the variation of pixel values ​​in the image by weighted averaging, thereby blurring the image and reducing the impact of image noise.

[0086] S432: Detect edges in the image using the Canny operator.

[0087] This step includes:

[0088] S4321. Calculate the gradient magnitude and gradient direction of the image by applying the Sobel operator.

[0089] S4322. Scan the image, retain the local maximum points of the local gradient amplitude, and suppress non-maximum values.

[0090] In this step, the detected edge is refined by retaining the local maximum point of the local gradient amplitude and suppressing the non-maximum value, so that it matches the actual edge more accurately.

[0091] S4323. Set two thresholds and threshold the image that is not the maximum threshold.

[0092] In this step, two thresholds, i.e., a high threshold and a low threshold, are set to threshold the image after non-maximum suppression. Specifically, pixels with gradient magnitudes higher than the high threshold are marked as strong edges, and pixels with gradient magnitudes lower than the low threshold are marked as weak edges. Starting from the strong edge pixels, the connected weak edge pixels are tracked and marked as strong edges until they can no longer be connected to new edge pixels.

[0093] S4324, further refine the image to remove irrelevant edge segments and disconnected edge segments, thereby obtaining the image edge.

[0094] S433, all contours in the image are obtained according to the edge, that is, the pin pattern is obtained, such as Figure 7 shown.

[0095] S50, calculating the position and angle of the component pins according to the pin pattern.

[0096] In this step, since the component pins may not be set strictly in the direction of the standard pins or there may be deviations during production or transportation, the direction of the component pins and the standard pins may form a certain angle, which is the angle of the component and needs to be compensated in the subsequent plug-in operation. Specifically, it includes:

[0097] S51 . Form a pin convex hull of the component pin image based on the component pin pattern, and perform a horizontal operation on the pin convex hull.

[0098] In this step, if Figure 8As shown, the center of the outermost pin pattern forms a rectangle including all pin patterns, and the rectangle is the pin convex hull of the component pin image. The horizontalization operation is performed based on the long side and short side of the pin convex hull. In order to avoid the rotation operation, if the pin pattern in the pin convex hull is an asymmetric pin, it is necessary to distinguish different long sides. Specifically, the long side can be divided into a first long side and a second long side. When the pin convex hull is horizontalized, the first long side and the short side or the second long side and the short side are used as the reference.

[0099] S52, matching the pin convex hull after horizontalization in step S51 with the template pin convex hull preset in step S41, if the match is successful, a matching component image is obtained; if the match is unsuccessful, a material discarding process is performed.

[0100] In this step, after the pin convex hull is horizontalized, the horizontalized pin convex hull is matched with all preset template pin convex hulls. If the match is successful, a matched component image is obtained; if the match is unsuccessful, it means that the pin deviation of the component is too large and needs to be polished.

[0101] S53, calculating the center coordinates of the entire pin according to the actual position of each pin pattern in the successfully matched component image, and judging whether the pin offset is too large according to the actual position of each pin, and performing material removal processing if the offset is too large.

[0102] S54, calculating the offset value of the actual coordinates of the component relative to the shooting position according to the pixel coordinates of the component in the image, the camera calibration information and the shooting position, and calculating the position of the component pin based on the offset value.

[0103] S55. Calculate the angle of the component according to the pin convex hull of the component pin image.

[0104] In this step, the angle of the component is the angle between the pin bump and the template pin bump. Specifically, taking the long side or short side of the template pin bump as a reference, the angle between the long side or short side of the pin bump and the long side or short side of the template pin bump is the angle of the component. If the long sides of the pin bump are inconsistent, it is necessary to distinguish the first long side and the second long side.

[0105] S60, during the plug-in operation, the plug-in hole angle of step S20 and the component angle of step S50 are superimposed to calculate the angle that needs to be compensated for the plug-in operation, and the plug-in operation is performed by rotating the compensated angle.

[0106] In this step, when plugging in, the hole rotation angle of the convex bulge formed by the plug-in hole is superimposed with the component rotation angle of the pin convex bulge formed by the component pattern to obtain a compensation angle, and precise plug-in operation can be achieved by rotating the compensation angle.

[0107] The special-shaped machine plug-in method based on image recognition in this embodiment obtains the position and angle of the plug-in hole by taking an image of the PCB plug-in hole. At the same time, the position and angle of the component pin is obtained by taking an image of the component pin. When performing the plug-in operation, the angle of the plug-in hole and the angle of the component pin are superimposed to form a compensation angle, thereby realizing accurate plug-in operation and ensuring product quality.

[0108] On the other hand, by matching the image of the component pin with a template that has pre-captured images of different component pins, components with poor size or structure can be screened out; further, by matching the pin convexity of the component with the pin convexity of the component template, components with excessive pin deviation can be screened out, which not only ensures the quality of the component but also improves the plug-in efficiency.

[0109] Although the present invention is disclosed through the above embodiments, the protection scope of the present invention is not limited thereto. Without departing from the concept of the present invention, any deformation, replacement, etc. made to the above components shall fall within the scope of the claims of the present invention.

Claims

1. A special-shaped machine plug-in method based on image recognition, characterized in that: The method comprises the following steps: a. Obtain the plug-in hole image of the PCB and perform image processing on the image to obtain the hole position contour; b. Calculate the position and angle of the plug-in hole according to the hole profile; c. Obtain the pin image of the component; d. Matching the pin image with a preset pin image template, and performing image processing on the area matching the template to obtain the pin pattern; e. Calculate the position and angle of the component pins based on the pin pattern; f. When plugging in, superimpose the plug-in hole angle of step b and the component angle of step e, calculate the angle that needs to be compensated for the plug-in operation, and rotate the compensated angle to perform the plug-in operation.

2. The image recognition-based special-shaped machine plug-in method according to claim 1, characterized in that: In step a, the step of performing image processing on the image to obtain the hole position contour includes: a1. Performing Gaussian filtering on the image; a2. Detect the edges in the image using the Canny operator; a3. Obtain all hole contours in the image according to the edge.

3. The special-shaped machine plug-in method based on image recognition as claimed in claim 1, characterized in that: Step b includes: b1. Extract the position information of each plug-in hole according to the hole contour; b2. Calculate the center coordinates of the entire image based on the position of each plug-in hole, and then obtain the actual coordinates based on the center coordinates, which is the position of the plug-in hole; b3. Obtain the convex hull of the plug-in hole of the image, and calculate the angle of the plug-in hole based on the convex hull of the plug-in hole.

4. The image recognition-based special-shaped machine plug-in method according to claim 3, characterized in that: In step b3, the plug-in hole convex hull is the smallest polygon formed by all the plug-in holes in the image, and the plug-in hole angle is the included angle between the plug-in hole convex hull and the preset standard plug-in hole convex hull.

5. The special-shaped machine plug-in method based on image recognition as claimed in claim 1, characterized in that: In step c, the method for obtaining the pin image of the component is: the head of the special-shaped machine passes through six predetermined shooting positions in sequence, and when the head of the special-shaped machine enters a certain range of the predetermined position, the camera is triggered to shoot and record the actual position of the current xy axis.

6. The method for inserting a special-shaped machine based on image recognition as claimed in claim 5, characterized in that: Step d includes: d1. Pre-photograph templates of different component pin images, obtain the pin convex hull of each template, and perform a horizontal operation on the template pin convex hull; d2. Match the pin image with a preset pin image template; d3. If the match is successful, the area matching the template is image processed to obtain the pin pattern.

7. The special-shaped machine plug-in method based on image recognition as claimed in claim 6, characterized in that: In step d3, the step of performing image processing on the area matching the template to obtain the pin pattern includes: d31, perform Gaussian filtering on the area image matching the template; d32, detect the edge in the image through the Canny operator; d33. Acquire all contours in the image according to the edge, so as to obtain the pin pattern.

8. The method for inserting a special-shaped machine based on image recognition as claimed in claim 7, characterized in that: Step d32 includes: d321, calculate the gradient magnitude and gradient direction of the image by applying the Sobel operator; d322, scan the image, retain the local maximum point of the local gradient amplitude, and suppress the non-maximum value; d323, set two thresholds to threshold the image that is not the maximum threshold; d324. Further refine the image to remove irrelevant edge segments and disconnected edge segments, thereby obtaining the image edge.

9. The method for inserting a special-shaped machine based on image recognition as claimed in claim 6, characterized in that: Step e includes: e1. forming a pin convex hull of the component pin image based on the component pin pattern, and performing a horizontal operation on the pin convex hull; e2. Match the horizontal pin convex hull with the template pin convex hull preset in step d1. If the match is successful, a matched component image is obtained; if the match is unsuccessful, the material is discarded; e3. Calculate the center coordinates of the entire pin according to the actual position of each pin pattern in the successfully matched component image, and determine whether the pin offset is too large according to the actual position of each pin. If the offset is too large, perform material removal. e4. Calculate the offset value of the actual coordinates of the component relative to the shooting position according to the pixel coordinates of the component in the image, the camera calibration information and the shooting position, and calculate the position of the component pin based on the offset value; e5. Calculate the angle of the component according to the pin convex hull of the component pin image, where the angle of the component is the included angle between the pin convex hull of the component pin image and the pin convex hull of the template.

10. The special-shaped machine plug-in method based on image recognition as claimed in claim 9, characterized in that: In step e2, the step of performing a horizontal operation on the pin convex hull includes: The horizontalization operation is performed based on the long side and the short side of the pin convex hull. If the pin pattern in the pin convex hull is an asymmetric pin, the long side includes a first long side and a second long side.

Images