Intelligent detection method and system for glue overflow defect on surface of electronic product bracket

By using a high-resolution line scan camera and Kirigami folding algorithm segmentation technology, the accuracy problem of detecting adhesive overflow defects on the surface of electronic product brackets has been solved, achieving efficient and accurate adhesive overflow detection, and improving product quality and work efficiency.

CN119027392APending Publication Date: 2024-11-26深圳市智弦科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411114812.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing technologies are insufficient to accurately detect subtle adhesive overflow defects on the surface of electronic product brackets, leading to high rates of missed or over-detection, which affects product quality.

Method used

A high-resolution line scan camera is used to scan the surface of the bracket, creating multiple image positioning templates. The image is repositioned using the three-point positioning method, and the image is segmented using the Kirigami folding algorithm. The overflow defect area is segmented by instance segmentation. Combining affine transformation and instance segmentation techniques, the bracket's qualification is determined.

Benefits of technology

It achieves high detection rate and low false negative rate for glue overflow detection, improves detection accuracy and speed, ensures product quality, and reduces labor and time costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027392A_ABST
    Figure CN119027392A_ABST
Patent Text Reader

Abstract

The invention relates to an electronic product support surface glue overflow defect intelligent detection method and system which are ingenious in design and have the advantages of being high in detection rate, low in omission ratio, high in detection precision, high in accuracy, high in accuracy and the like for glue overflow of a support. A plurality of image positioning templates are created through a plurality of image features at different positions of a picture, and then non-detection areas can be removed to improve the detection speed by detecting the glue overflow areas of the bracket in blocks; the to-be-detected material picture is repositioned, so that the position consistency of the bracket image is ensured, the glue overflow detection area is reduced again, and the detection precision is improved; carrying out folding processing and instance segmentation processing on the segmented sub-pictures independently to obtain the specific position of the glue overflow defect of each detection area; then, the detection results of the sub-images are combined, the complete detection result is inversely calculated to the original image, whether the bracket is qualified or not is judged according to the complete detection result and the defect judgment standard, and the detection accuracy of the glue overflow defect is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electronic product appearance defect detection, and more specifically, to an intelligent detection method and system for electronic product bracket surface glue overflow defects. Background Art

[0002] The quality of electronic products has become a key factor that more and more consumers consider. Electronic products usually need to go through a variety of complex process flows during the production process, among which the glue spraying and bonding of the middle frame bracket is a key link in the display assembly process. Glue overflow defects generally occur at the edge of the middle frame bracket; the middle frame bracket is often made of metal, and it is very easy to produce glue overflow defects on the surface during the glue spraying process, that is, the glue overflows from the edge of the bracket during the spraying process. The overflowed glue usually attached to the surface of the bracket will cause the gap to be too large in the subsequent bonding process, resulting in poor sealing of the electronic product. This will seriously affect the overall quality of the product. Therefore, how to accurately detect glue overflow defects on the appearance of electronic product brackets is the key to ensuring product quality. Most of the existing technologies use strip lights and line scan cameras to capture image effects. Existing glue overflow brackets such as Figure 1 and Figure 2 As shown, there are respectively a partial enlarged picture of the overflow glue in the upper left corner of the electronic equipment bracket, and a partial enlarged picture of the overflow glue in the lower right corner of the electronic equipment bracket. It can be seen from the image that the edge of the bracket has a relatively obvious overflow glue defect. Most traditional visual inspection algorithms can detect relatively obvious overflow glue defects, but for materials with no obvious overflow glue, most detection algorithms find it difficult to achieve good results. They are either prone to miss detection or easy to pass inspection. Therefore, the accuracy of the overflow glue detection on the material surface cannot be guaranteed. This requires that the surface defect detection of the middle frame bracket should have higher accuracy and intelligence. It should not only be able to detect the relatively obvious overflow glue defects on the bracket surface, but also have better learning ability, and have better recognition ability for various types and forms of overflow glue defects. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for intelligently detecting glue overflow defects on the surface of electronic product brackets in view of the above-mentioned defects in the prior art.

[0004] On the one hand, the technical solution adopted by the present invention to solve its technical problem is: an intelligent detection method for glue overflow defects on the surface of an electronic product bracket, wherein the detection method comprises the following steps:

[0005] The camera is used to scan and photograph the entire material surface to determine the glue overflow area on the bracket surface, extract all contours in the glue overflow area, and filter out the required image feature contours; then, multiple image positioning templates are created based on multiple image features at different positions of the picture;

[0006] Match the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; reposition the image of the material to be inspected and define the specific ROI area for defect detection;

[0007] Cut the original image of the material to be inspected and fold the obtained sub-image N times through the folding algorithm; then perform instance segmentation on the cut and folded image to separate the foreground and background of the overflow defect and obtain the specific location of the overflow defect in each inspection area;

[0008] The inspection results of all sub-images are summarized, and the specific location and defect area of ​​the overflow glue defect on the original image are obtained by reverse calculation. The reverse calculation process is summarized and restored and merged according to the above-mentioned cutting process to determine whether the bracket is qualified;

[0009] In the intelligent detection method for the overflow of glue on the surface of the electronic product bracket described in the present invention, the repositioning method of the image of the material to be tested is as follows:

[0010] The three-point positioning method is used to establish the row and column coordinates of the image template and the row and column coordinates of the image to be detected according to the feature contours in three different orientations to calculate the rigid affine transformation matrix, and then the image to be detected is rotated and translated until the position of the bracket in all images remains consistent;

[0011] The method for intelligently detecting the overflow of glue on the surface of the electronic product support of the present invention comprises obtaining the size of the image after affine transformation, comparing the height and width of the image, and taking the minimum value of the height and width of the image as the basis for the next step of cutting and folding;

[0012] The intelligent detection method for the overflow defect of the electronic product support surface of the present invention, wherein the x-direction and y-direction scaling factors of the affine matrix calculated by the rigid affine transformation are 1, the rotation angle is 0.031786, and the bevel angle is 0;

[0013] In the intelligent detection method for the overflow defect on the surface of the electronic product support described in the present invention, the folding algorithm logic of the sub-image is as follows:

[0014] The width and height of the image are w and h respectively. If h>w, the height is cut based on the width w, and the sub-image obtained by cutting is folded based on w. The size of the folded image is 2w*h / 2. If 2w≥h, the folding algorithm ends, otherwise the next cutting and folding is performed, and the iteration is continued until the termination condition of the folding algorithm 2w≥h is met, and the folding algorithm ends;

[0015] In the intelligent detection method for the overflow of glue on the surface of the electronic product bracket of the present invention, the image of the rectangular area is divided into a plurality of square areas based on the short side, and the image of each square area is segmented by instance;

[0016] On the other hand, the present invention provides an intelligent detection system for adhesive overflow defects on the surface of electronic product brackets, which includes:

[0017] The scanning and extraction module is used to scan and photograph the entire material surface to determine the glue overflow area on the bracket surface, extract all contours in the glue overflow area, and filter out the required image feature contours;

[0018] A creation module is used to create multiple image positioning templates according to multiple image features at different positions of the image;

[0019] The contour matching module is used to match the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; reposition the image of the material to be inspected and define the specific ROI area for defect detection.

[0020] The cutting and folding module is used to cut the original image of the material to be inspected and fold the obtained sub-image N times through the folding algorithm; then the cut and folded image is instance segmented to separate the foreground and background of the overflow defect, and the specific location of the overflow defect in each inspection area is obtained;

[0021] The summary and reverse calculation module is used to summarize the detection results of all sub-images, reversely calculate the specific location and defect area of ​​the overflow glue defect on the original image, and the summary and reverse calculation process is reversely restored and merged according to the above-mentioned cutting process to determine whether the bracket is qualified;

[0022] The beneficial effects of the present invention are as follows: the intelligent detection method and system for the overflow glue defect on the surface of the electronic product bracket are cleverly designed, have a high detection rate and a low missed detection rate for the overflow glue of the bracket, and have high detection accuracy. By using a high-resolution line scan camera to scan and photograph the entire bracket, and then creating multiple image positioning templates according to multiple image features at different positions of the picture, the overflow glue area on the surface of the bracket is detected in blocks to remove the undetected area to improve the detection speed; then the characteristic contour of the image positioning template is matched with the contour of the corresponding area of ​​the image of the material to be detected; the image of the material to be detected is repositioned to ensure the consistency of the position of the bracket image, and the overflow glue detection area is further reduced to improve the detection accuracy; the specific position of the overflow glue defect in each detection area is obtained by folding the segmented sub-images and performing instance segmentation processing separately; then the detection results of the sub-images are merged, and the complete detection results are back-calculated to the original image, and whether the bracket is qualified is determined based on the complete detection results and defect discrimination criteria, and the detection accuracy of the overflow glue defect is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work:

[0024] Figure 1 It is a partial enlarged view of the glue overflow at the upper left corner of the electronic equipment bracket in the background technology;

[0025] Figure 2 It is a partial enlarged view of the glue overflow at the lower right corner of the electronic equipment bracket in the background technology;

[0026] Figure 3 This is a flow chart of an intelligent detection method for adhesive overflow defects on the surface of an electronic product bracket according to a first embodiment of the present invention;

[0027] Figure 4 is a result diagram of defect detection of an electronic device bracket in the first embodiment of the present invention;

[0028] Figure 5 It is a schematic diagram of an intelligent detection system for adhesive overflow defects on the surface of an electronic product bracket according to a second embodiment of the present invention. DETAILED DESCRIPTION

[0029] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present invention and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] "Multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0032] Moreover, the terms "up, down, front, back, left, right, upper end, lower end, longitudinal" and the like indicating directions are all based on the posture and position of the device or equipment described in this solution during normal use.

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0034] Generally, the glue overflow phenomenon only exists at the edge of the frame bracket of the electronic product. The present invention uses a high-resolution camera to collect the edge image of the bracket for detection.

[0035] 1. Embodiment 1:

[0036] The present invention provides an intelligent detection method for adhesive overflow defects on the surface of an electronic product bracket, such as Figure 3 As shown, the detection method comprises the following steps:

[0037] S10: Scan the entire material surface with a camera to determine the glue overflow area on the bracket surface, delineate the area where the characteristic contour is located for the selected bracket image, extract all contours in the glue overflow area, and filter out the required image characteristic contours; then create multiple image positioning templates based on multiple image features at different positions of the image;

[0038] S20: Matching the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; in order to avoid positional displacement of the image during the inspection process, it is necessary to select a picture with a correct position and a clear image from multiple pictures. In the embodiment of the present invention, a three-point positioning method can be used to reposition the image of the material to be inspected and define a specific ROI area for defect detection;

[0039] Among them, the relocation method of the image of the material to be tested is as follows:

[0040] The three-point positioning method is used to establish the row and column coordinates of the image template and the row and column coordinates of the image to be tested according to the characteristic contours of three different positions, such as the characteristic contours of any three positions in the lower right corner, upper right corner, upper left corner and upper right corner of the frame bracket of the electronic product, and calculate the rigid affine transformation matrix. Then, the image to be tested is rotated and translated until the position of the bracket in all images is consistent and the positioning is accurate, which is convenient for subsequent manual demarcation of the specific area of ​​the overflow defect. In this embodiment, Figure 1Taking the overflow glue bracket in the upper left corner as the material to be detected as an example, the x- and y-direction scaling factors of the affine matrix calculated by the rigid affine transformation are both 1, the rotation angle is 0.031786, the bevel angle is 0, the x-direction translation distance is 78.821104mm, and the y-direction translation distance is 506.820466mm. The displacement accuracy is high, which is convenient for subsequent precise detection.

[0041] Furthermore, the detection method of the present invention further includes obtaining the size of the image after affine transformation, comparing the height and width of the image, and taking the minimum value of the image height and width as the basis for the next step of cutting and folding;

[0042] S30: The present invention processes the image using a folding algorithm, such as using the Kirigami folding form to combine cutting and folding. Kirigami is a paper art technique originating from Japan. Unlike traditional origami, Kirigami includes not only folding but also cutting steps. This art form creates three-dimensional structures by cutting and folding on paper, and its name comes from the Japanese words "cut" and "origami". The characteristic of Kirigami technology is that by pre-scoring and folding the paper, its shape can be easily changed to form a variety of interesting and practical three-dimensional designs, thereby realizing the segmentation of the image into multiple sub-pictures, continuously narrowing the detection range, and simple operation.

[0043] Specifically, the folding and segmentation operations of the present invention are as follows: cutting the original image of the material to be inspected and folding the obtained sub-image N times through a folding algorithm, where N is a positive integer; in the present invention, the width and height of the image are w and h respectively. If the image size is 8192*15000 and the height is greater than the width, the height is cut based on the width value w, and the sub-image obtained by cutting is folded based on w. The size of the folded image is 2w*h / 2. If 2w≥h, the folding algorithm is terminated, otherwise the next cutting and folding is performed, and the iteration is continued until the termination condition 2w≥h of the folding algorithm is met, and the folding algorithm is terminated.

[0044] Perform instance segmentation on the cut and folded images to mark different entities in the same category. Divide the rectangular area into several square areas based on the short side, perform instance segmentation on each square area, separate the foreground and background of the overflow defect, achieve pixel-level object separation, and obtain the specific location of the overflow defect in each detection area; instance segmentation can not only identify the objects in the image, but also draw accurate outlines for each object, which is easy to use.

[0045] like Figure 4 As shown, a result diagram of defect detection of electronic equipment bracket is shown;

[0046] S40: Summarize the detection results of all sub-images, and reversely calculate the specific position and defect area of ​​the overflow glue defect on the original image. The reverse calculation process is reversely restored and merged according to the above-mentioned cutting process to determine whether the bracket is qualified.

[0047] Among them, after detecting the specific position where the glue overflows on the bracket, it can be determined that the bracket is an NG product; conversely, if no glue overflow defect is detected, the bracket is an OK product.

[0048] The algorithm involved in the process of detecting the overflow of glue on the surface of the bracket in the present invention can obtain the bracket image in synchronization with the camera, and then can realize the automatic detection of the overflow of glue on the bracket of the electronic product; the size data of the overflow of glue in the detection method of the present invention can be output by the computer, and the specific location of the overflow of glue is marked in the data, and then the entire bracket is marked as NG (defective product), and the computer can automatically perform statistical analysis on the unqualified products. The algorithm involved in this detection process has a certain learning function, and its learning process is very simple and easy to operate, and the algorithm can be continuously updated, which greatly improves the accuracy of the product, improves work efficiency, and saves labor costs and time costs.

[0049] The intelligent detection method for the surface overflow of glue defects on the electronic product bracket is cleverly designed, has a high detection rate and a low missed detection rate for the overflow of glue on the bracket, and has high detection accuracy. The entire bracket is scanned and photographed by a high-resolution line scan camera, and then multiple image positioning templates are created according to multiple image features at different positions of the picture, and then the overflow of glue area on the bracket surface is detected in blocks to remove the undetected area to improve the detection speed; then the feature contour of the image positioning template is matched with the contour of the corresponding area of ​​the image of the material to be detected; the image of the material to be detected is relocated to ensure the consistency of the position of the bracket image, and the overflow of glue detection area is further reduced to improve the detection accuracy; the specific position of the overflow of glue defects in each detection area is obtained by folding and segmenting the segmented sub-images separately; then the detection results of the sub-images are merged, and the complete detection results are back-calculated to the original image, and whether the bracket is qualified is determined according to the complete detection results and defect discrimination criteria, and the detection accuracy of the overflow of glue defects is high.

[0050] 2. Embodiment 2:

[0051] The present invention also provides an intelligent detection system for adhesive overflow defects on the surface of an electronic product bracket, which includes the following modules:

[0052] The scanning and extraction module is used to scan and photograph the entire material surface to determine the glue overflow area on the bracket surface, extract all contours in the glue overflow area, and filter out the required image feature contours;

[0053] A creation module is used to create multiple image positioning templates according to multiple image features at different positions of the image;

[0054] The contour matching module is used to match the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; reposition the image of the material to be inspected and define the specific ROI area for defect detection.

[0055] The cutting and folding module is used to cut the original image of the material to be inspected and fold the obtained sub-image N times through the folding algorithm; N is a positive integer; then the cut and folded image is instance segmented to separate the foreground and background of the overflow defect, and the specific location of the overflow defect in each inspection area is obtained;

[0056] The summary and inverse calculation module is used to summarize the detection results of all sub-images, inversely calculate the specific location and defect area of ​​the overflow glue defect in the original image, and the summary and inverse calculation process is reversely restored and merged according to the above-mentioned cutting process to determine whether the bracket is qualified.

[0057] The intelligent detection system for the surface overflow of glue defects on the bracket of electronic products is cleverly designed. It has a high detection rate and a low missed detection rate for the overflow of glue on the bracket, and has high detection accuracy. It uses a high-resolution line scan camera to scan and photograph the entire bracket, and then creates multiple image positioning templates based on multiple image features at different positions of the picture. The overflow of glue area on the surface of the bracket is then detected in blocks to remove the undetected area to increase the detection speed. The feature contour of the image positioning template is then matched with the contour of the corresponding area of ​​the image of the material to be detected. The image of the material to be detected is repositioned to ensure the consistency of the position of the bracket image, and the overflow of glue detection area is further reduced to improve the detection accuracy. The specific position of the overflow of glue defects in each detection area is obtained by folding and segmenting the segmented sub-images separately. Then the detection results of the sub-images are merged, and the complete detection results are back-calculated onto the original image. The qualified or unqualified bracket is determined based on the complete detection results and defect discrimination criteria, and the detection accuracy of the overflow of glue defects is high.

[0058] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. An intelligent detection method for adhesive overflow defects on the surface of electronic product brackets, characterized in that: The detection method comprises the following steps: The camera is used to scan and photograph the entire material surface to determine the glue overflow area on the bracket surface, extract all contours in the glue overflow area, and filter out the required image feature contours; then, multiple image positioning templates are created based on multiple image features at different positions of the picture; Match the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; reposition the image of the material to be inspected and define the specific ROI area for defect detection; Cut the original image of the material to be inspected and fold the obtained sub-image N times through the folding algorithm; then perform instance segmentation on the cut and folded image to separate the foreground and background of the overflow defect and obtain the specific location of the overflow defect in each inspection area; The inspection results of all sub-images are summarized, and the specific location and defect area of ​​the overflow glue defect on the original image are obtained by reverse calculation. The reverse calculation process is summarized and restored and merged in reverse according to the above-mentioned cutting process to determine whether the bracket is qualified.

2. The intelligent detection method for adhesive overflow defects on the surface of electronic product brackets according to claim 1 is characterized in that: The relocation method of the material image to be tested is as follows: The three-point positioning method is used to establish the row and column coordinates of the image template and the row and column coordinates of the image to be detected according to the feature contours in three different orientations to calculate the rigid affine transformation matrix, and then the image to be detected is rotated and translated until the position of the bracket in all images remains consistent.

3. The intelligent detection method for adhesive overflow defects on the surface of electronic product brackets according to claim 2 is characterized in that: Get the size of the image after affine transformation, compare the height and width of the image, and use the minimum value of the image height and width as the basis for the next step of cutting and folding.

4. The intelligent detection method for adhesive overflow defects on the surface of electronic product brackets according to claim 2 or 3, characterized in that: in, The affine matrix calculated by the rigid affine transformation has a scaling factor of 1 in the x-direction and y-direction, a rotation angle of 0.031786, and a bevel angle of 0.

5. The intelligent detection method for adhesive overflow defects on the surface of electronic product brackets according to any one of claims 1 to 3, characterized in that: The logic of the sub-image folding algorithm is as follows: The width and height of the image are w and h respectively. If h>w, the height is cut based on the width w, and the sub-image obtained by cutting is folded based on w. The size of the folded image is 2w*h / 2. If 2w≥h, the folding algorithm is terminated, otherwise the next cutting and folding is performed. The iteration is continued until the termination condition of the folding algorithm 2w≥h is met, and the folding algorithm is terminated.

6. The intelligent detection method for adhesive overflow defects on the surface of electronic product brackets according to claim 1, characterized in that: The rectangular area image is divided into several square areas based on the short side, and instance segmentation is performed on the image in each square area.

7. An intelligent detection system for adhesive overflow defects on the surface of electronic product brackets, characterized in that: include: The scanning and extraction module is used to scan and photograph the entire material surface to determine the glue overflow area on the bracket surface, extract all contours in the glue overflow area, and filter out the required image feature contours; A creation module is used to create multiple image positioning templates according to multiple image features at different positions of the image; The contour matching module is used to match the characteristic contour of the image positioning template with the contour of the corresponding area of ​​the image of the material to be inspected; reposition the image of the material to be inspected and define the specific ROI area for defect detection. The cutting and folding module is used to cut the original image of the material to be inspected and fold the obtained sub-image N times through the folding algorithm; then the cut and folded image is instance segmented to separate the foreground and background of the overflow defect, and the specific location of the overflow defect in each inspection area is obtained; The summary and inverse calculation module is used to summarize the detection results of all sub-images, inversely calculate the specific location and defect area of ​​the overflow glue defect in the original image, and the summary and inverse calculation process is reversely restored and merged according to the above-mentioned cutting process to determine whether the bracket is qualified.