SIM card package glue breaking visual detection method, system, storage medium and device

Abstract

The application discloses a SIM card packaging glue breaking visual detection method and system, a storage medium and a device. The method comprises the following steps: collecting a multispectral image of a SIM card, wherein the multispectral image comprises at least four original images; performing registration and fusion based on the multispectral image to obtain a multi-channel fusion image tensor; inputting the multi-channel fusion image tensor into a trained segmentation model to obtain a glue segmentation mask; performing glue breaking defect analysis based on the glue segmentation mask to obtain defect data, wherein the analysis process comprises connected region analysis, skeletonization and density analysis, and internal contour detection; extracting the multispectral image, the glue segmentation mask and the defect data to generate a detection report, and visualizing the defect area. The application can automatically detect the glue breaking defects of the SIM card packaging with high precision and high robustness, effectively identifies various complex defects, significantly improves the detection efficiency and adaptability, and reduces the false detection rate and maintenance cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a visual inspection method, system, storage medium, and device for SIM card encapsulation adhesive breakage. Background Technology

[0002] In the manufacturing process of mobile phone SIM cards (modules), one of the key steps in bonding and encapsulating the chip (die) onto the substrate is dispensing and curing. The quality of the adhesive application is crucial during this process, with "adhesive breakage" being a common and highly detrimental defect. This refers to a partial interruption, a sharp narrowing of the adhesive path, or complete absence, resulting in insufficient chip bonding strength. Consequently, the module may fail or detach during subsequent testing, transportation, or use, directly impacting product reliability and yield. As SIM card sizes continue to shrink (e.g., eSIM), the dispensing path becomes increasingly intricate, posing extreme challenges to the accuracy and reliability of testing systems.

[0003] Traditional inspection methods primarily rely on manual visual inspection, where operators observe the dispensed product under a microscope. This method is not only inefficient and costly, but also highly susceptible to missed detections and misjudgments due to visual fatigue and subjective biases. This is especially problematic on high-volume, high-paced production lines, becoming a bottleneck for improving both capacity and quality. Therefore, adopting machine vision technology to automate adhesive breakage detection has become an inevitable trend in the industry. Summary of the Invention

[0004] The purpose of this invention is to provide a visual inspection method, system, storage medium, and device for SIM card encapsulation adhesive breakage, which solves the problems of existing SIM card adhesive breakage detection technology, which relies on a single light source and simple image algorithms, resulting in sensitivity to light, high false negative rate, and inability to identify complex defects.

[0005] The first aspect of this invention provides a visual inspection method for SIM card encapsulation adhesive breakage, comprising the following steps:

[0006] Acquire multispectral images of the SIM card, wherein the multispectral images include at least four original images;

[0007] The multi-channel fused image tensor is obtained by registration and fusion of the multispectral image.

[0008] The multi-channel fused image tensor is input into the trained segmentation model to obtain the glue segmentation mask;

[0009] Based on the glue segmentation mask, defect data is obtained by analyzing glue breakage defects. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection.

[0010] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized.

[0011] In this solution, the acquisition of multispectral images from the SIM card specifically includes:

[0012] The system controls a pre-set precision motion platform to move the SIM card to a pre-set imaging position, and triggers the camera to turn on via a pre-set trigger sensor.

[0013] When the coaxial white light source is turned on, the acquired image is obtained. ;

[0014] When the red low-angle light source is turned on, the acquired image is obtained. ;

[0015] When the blue low-angle light source is turned on, the acquired image is obtained. ;

[0016] When the ultraviolet light source is turned on, the acquired image is obtained. ;

[0017] Based on the acquired images ,image ,image and images The multispectral image is obtained.

[0018] In this scheme, the step of registering and fusing the multispectral image to obtain a multi-channel fused image tensor specifically includes:

[0019] Select image Based on the baseline, calculate the image ,image and images The affine transformation matrix to the reference;

[0020] Apply the affine transformation matrix to the image ,image and images Transform to image Registered images obtained in the same coordinate system Image registration and registered images ;

[0021] Image Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor.

[0022] In this scheme, the step of inputting the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask specifically includes:

[0023] The segmentation model includes the U-Net++ segmentation model, which includes an encoder and a decoder;

[0024] The multi-channel fused image tensor is input into the U-Net++ segmentation model to obtain the glue category probability for each pixel;

[0025] The glue segmentation mask is obtained by binarizing the probability map output by the model using a threshold. .

[0026] In this solution, the defect data obtained by analyzing adhesive breakage defects based on the adhesive segmentation mask specifically includes:

[0027] When performing connected region analysis, the number of independent connected regions in the glue-segmented mask is specifically calculated. If the number is greater than a preset value, a complete break defect is determined, and the position and area of ​​each broken part are recorded.

[0028] During skeletonization and density analysis, the central skeleton line of the glue region is extracted, and the width of each sampling point is calculated along the skeleton line to obtain the width sequence. Calculate the width sequence mean and standard deviation The fine glue threshold is thus calculated based on a preset formula. , where, if the width sequence Medium sampling point width Less than the fine glue threshold If so, it is determined that there is a fine glue defect;

[0029] When performing internal contour detection, specifically, internal holes are detected within the glue-splitting mask. If the hole area is greater than a preset hole threshold... If so, it is determined that a hollow defect exists.

[0030] In this solution, the multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized. Specifically, this includes:

[0031] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a structured inspection report, which includes an overall conclusion, a list of defect types, defect pixel coordinates, and quantization parameters.

[0032] When visualizing, multispectral images, segmented contours, skeleton lines, and defect areas marked with different colors are displayed on the same screen.

[0033] A second aspect of the present invention also provides a visual inspection system for SIM card encapsulation adhesive breakage, comprising a memory and a processor. The memory includes a program for visual inspection of SIM card encapsulation adhesive breakage, which, when executed by the processor, performs the following steps:

[0034] Acquire multispectral images of the SIM card, wherein the multispectral images include at least four original images;

[0035] The multi-channel fused image tensor is obtained by registration and fusion of the multispectral image.

[0036] The multi-channel fused image tensor is input into the trained segmentation model to obtain the glue segmentation mask;

[0037] Based on the glue segmentation mask, defect data is obtained by analyzing glue breakage defects. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection.

[0038] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized.

[0039] In this solution, the acquisition of multispectral images from the SIM card specifically includes:

[0040] The system controls a pre-set precision motion platform to move the SIM card to a pre-set imaging position, and triggers the camera to turn on via a pre-set trigger sensor.

[0041] When the coaxial white light source is turned on, the acquired image is obtained. ;

[0042] When the red low-angle light source is turned on, the acquired image is obtained. ;

[0043] When the blue low-angle light source is turned on, the acquired image is obtained. ;

[0044] When the ultraviolet light source is turned on, the acquired image is obtained. ;

[0045] Based on the acquired images ,image ,image and images The multispectral image is obtained.

[0046] In this scheme, the step of registering and fusing the multispectral image to obtain a multi-channel fused image tensor specifically includes:

[0047] Select image Based on the baseline, calculate the image ,image and images The affine transformation matrix to the reference;

[0048] Apply the affine transformation matrix to the image ,image and images Transform to image Registered images obtained in the same coordinate system Image registration and registered images ;

[0049] Image Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor.

[0050] In this scheme, the step of inputting the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask specifically includes:

[0051] The segmentation model includes the U-Net++ segmentation model, which includes an encoder and a decoder;

[0052] The multi-channel fused image tensor is input into the U-Net++ segmentation model to obtain the glue category probability for each pixel;

[0053] The glue segmentation mask is obtained by binarizing the probability map output by the model using a threshold. .

[0054] In this solution, the defect data obtained by analyzing adhesive breakage defects based on the adhesive segmentation mask specifically includes:

[0055] When performing connected region analysis, the number of independent connected regions in the glue-segmented mask is specifically calculated. If the number is greater than a preset value, a complete break defect is determined, and the position and area of ​​each broken part are recorded.

[0056] During skeletonization and density analysis, the central skeleton line of the glue region is extracted, and the width of each sampling point is calculated along the skeleton line to obtain the width sequence. Calculate the width sequence mean and standard deviation The fine glue threshold is thus calculated based on a preset formula. , where, if the width sequence Medium sampling point width Less than the fine glue threshold If so, it is determined that there is a fine glue defect;

[0057] When performing internal contour detection, specifically, internal holes are detected within the glue-splitting mask. If the hole area is greater than a preset hole threshold... If so, it is determined that a hollow defect exists.

[0058] In this solution, the multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized. Specifically, this includes:

[0059] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a structured inspection report, which includes an overall conclusion, a list of defect types, defect pixel coordinates, and quantization parameters.

[0060] When visualizing, multispectral images, segmented contours, skeleton lines, and defect areas marked with different colors are displayed on the same screen.

[0061] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for a visual inspection method of SIM card encapsulation adhesive breakage, wherein when executed by a processor, the SIM card encapsulation adhesive breakage visual inspection method program implements the steps of the visual inspection method of SIM card encapsulation adhesive breakage as described in any of the preceding claims.

[0062] A fourth aspect of the present invention provides a visual inspection device for SIM card encapsulation adhesive breakage, comprising:

[0063] The system comprises an acquisition unit, a multispectral light source unit, a triggering unit, and a computing unit.

[0064] The acquisition unit includes a high-resolution color industrial camera for acquiring color images of the SIM card under different lighting conditions;

[0065] The multispectral light source unit includes several independently controlled light sources used to generate different lighting conditions;

[0066] The carrier triggering unit includes a precision motion platform and a trigger sensor. The precision motion platform is used to carry and position the SIM card, and the trigger sensor is used to trigger the camera to turn on.

[0067] The computing unit includes an industrial computer with a high-performance GPU, used to execute the visual inspection method for SIM card encapsulation adhesive breakage described in any of the above-mentioned embodiments.

[0068] In this solution, the device also includes an optical filter wheel, which is installed in front of a high-resolution color industrial camera, and the light source in the multispectral light source unit includes a coaxial white light source, a red low-angle bar light source, a blue low-angle bar light source, and an ultraviolet light source.

[0069] This invention discloses a visual inspection method, system, storage medium, and device for SIM card encapsulation adhesive breakage. It enables high-precision and robust automatic detection of SIM card encapsulation adhesive breakage defects, effectively identifying various complex defects, significantly improving detection efficiency and adaptability, while reducing false detection rate and maintenance costs. Specific beneficial effects are as follows:

[0070] 1. Extremely high detection robustness and adaptability: The multispectral imaging scheme enhances the characteristics of the adhesive from multiple physical dimensions (front, double edges, fluorescence), which greatly reduces the system's sensitivity to changes in illumination, substrate color, and adhesive material (including transparent adhesive), resulting in extremely high stability.

[0071] 2. Excellent defect detection capability: The semantic segmentation model based on U-Net++ can achieve pixel-level accurate segmentation, and its localization of glue boundaries far surpasses traditional edge detection. Combined with skeleton width analysis and internal contour detection, it can effectively identify complex defects such as "complete breakage", "thin glue", and "hollow", with extremely low false negative and false positive rates.

[0072] 3. Refined quantitative analysis beyond binary judgment: This invention not only determines the "presence" or "absence" of defects, but also provides quantitative data such as the type, location, and size (length, width, area) of the defects. This provides invaluable data support for optimizing production processes and tracing problems, achieving a leap from "detection" to "analysis."

[0073] 4. Strong generalization ability and low maintenance cost: Deep learning models can learn the essential characteristics of glue through training. For new products or new glue types, they can quickly adapt by simply providing new sample data for fine-tuning, without the need for complex feature engineering and parameter retuning, which significantly reduces maintenance costs and time.

[0074] 5. High degree of overall automation and intelligence: The entire process, from image acquisition to defect report generation, is fully automated without human intervention, which greatly improves detection efficiency and consistency and meets the high-speed and high-quality requirements of modern production lines. Attached Figure Description

[0075] Figure 1 A flowchart of a visual inspection method for SIM card encapsulation adhesive breakage according to the present invention is shown;

[0076] Figure 2 This diagram illustrates the visualization of the detection results in a visual inspection method for SIM card encapsulation adhesive breakage according to the present invention.

[0077] Figure 3 A block diagram of a visual inspection system for SIM card encapsulation adhesive breakage according to the present invention is shown.

[0078] Figure 4 A schematic diagram of the structure of a visual inspection device for SIM card encapsulation adhesive breakage according to the present invention is shown. Detailed Implementation

[0079] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0080] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0081] Existing machine vision-based detection schemes mainly revolve around two-dimensional imaging and image processing algorithms. The implementation schemes listed in this invention include grayscale image processing based on single-angle illumination, contour analysis methods based on edge detection, and methods based on traditional machine learning feature classification.

[0082] Among these, the global threshold segmentation method based on single-angle bright field illumination is the most basic and common scheme. It typically consists of a vertically oriented camera and a coaxial or ring light source to acquire an image of the front of the SIM card module. The algorithm assumes a fixed grayscale difference between the adhesive area and the substrate background. The processing flow is usually: image preprocessing (e.g., Gaussian filtering for noise reduction) -> global threshold segmentation (e.g., Otsu's method) -> morphological operations (e.g., closing operations to connect broken adhesive areas) -> connected component analysis. By calculating the connectivity, total area, length, and other features of the binarized adhesive area, it is compared with preset ideal template parameters. If the area is too small or there is a significant break, it is determined to be a broken adhesive. The advantage of this scheme is its simplicity and speed. However, its disadvantages are extremely prominent: First, it is extremely sensitive to illumination. Slight fluctuations in the light source, batch differences in substrate color, or the reflective properties of the adhesive itself (e.g., transparent or translucent adhesive) can cause drastic changes in grayscale values, rendering the fixed threshold ineffective and generating a large number of false alarms or missed alarms. Secondly, it cannot effectively distinguish between dents on the glue surface (which may not be true glue breakage) and actual fractures, nor can it detect defects such as glue width narrowing but not completely broken ("thin glue").

[0083] Secondly, the edge detection method based on low-angle dark field illumination uses a low-angle incident strip light source to enhance the contrast between the adhesive and the substrate boundary. Under near-horizontal illumination, the three-dimensional edges of the adhesive produce significant specular reflection or scattering, forming bright edges in the image, while the flat substrate area appears darker. The core of the algorithm lies in using edge detection operators (such as Canny and Sobel) to extract these bright edges, and then reconstructing the adhesive path through Hough transform or contour tracking. The continuity and parallelism of the edge lines (the adhesive should have two edges) are analyzed to determine if a break exists. This method has strong contour capture capabilities for the adhesive. However, its limitations are: it is highly sensitive to the mounting angle and easily affected by shadows from chip height and substrate warping. When the adhesive surface is flat and the edges are indistinct (such as some UV adhesives), the edge features are very weak. More importantly, it primarily focuses on the continuity of the "edge," making it difficult to effectively detect "hollow" breaks in the adhesive path (i.e., the edge line remains, but the adhesive in the middle is gone).

[0084] In addition to detection methods based on handcrafted features and classifiers, some approaches attempt to extract more features from images and utilize machine learning models for classification to improve robustness. First, a series of handcrafted features are extracted from the preprocessed image, such as statistical features based on gray-level histograms (mean, variance, entropy, etc.), texture features based on gray-level co-occurrence matrices (GLCM) (contrast, correlation, energy), and shape features based on HOG (Histogram of Oriented Gradients). These feature vectors are then input into a classifier (such as Support Vector Machine (SVM), AdaBoost, or Random Forest) for training. The model learns the feature space distribution of good products and broken glue defects and makes classification decisions for new samples. This method reduces dependence on single lighting conditions to some extent. However, its feature engineering process is cumbersome, time-consuming, and heavily reliant on expert prior knowledge. For complex and varied broken glue defects, especially novel or rare defect types, handcrafted features often lack sufficient representational power, have limited generalization performance, and incur high model maintenance and update costs.

[0085] Based on the above in-depth analysis of the existing technical solutions, the following key drawbacks can be clearly identified:

[0086] 1. Limited information dimension and poor anti-interference ability: Relying on a single light source and two-dimensional grayscale images, it cannot cope with common industrial interferences such as light fluctuations, material color differences, and reflections, resulting in poor system stability.

[0087] 2. Insufficient ability to detect low contrast and complex defects: For transparent / semi-transparent glue, glue with a gray level close to the background, and atypical glue breaks such as "thin glue" and "hollow", the existing methods have low sensitivity and a high risk of missed detection.

[0088] 3. Limited feature representation capabilities: Whether it is global threshold or manual features, it is difficult to comprehensively and accurately describe the complete shape, texture and contextual information of the glue, resulting in insufficient differentiation of defects.

[0089] 4. Poor flexibility and adaptability: The thresholds and parameters are mostly fixed values. When the product is changed, the type of adhesive is changed, or the process is adjusted, a lot of manual re-adjustment is required, resulting in poor adaptability.

[0090] 5. Lack of interpretability of test results: The system usually only outputs a "pass / fail" judgment, which is difficult to provide detailed information such as the specific location, type and severity of defects, which is not conducive to the traceability of the production process and process optimization.

[0091] To address these shortcomings, the present invention aims to provide a visual detection method for SIM card encapsulation adhesive breakage based on multispectral imaging and deep semantic segmentation. This method aims to achieve high-precision, high-robustness, and high-adaptive adhesive breakage detection by acquiring richer adhesive information and utilizing a powerful deep learning model for pixel-level accurate analysis.

[0092] Specifically, Figure 1 A flowchart of a visual inspection method for SIM card encapsulation adhesive breakage according to this application is shown.

[0093] like Figure 1 As shown, this application discloses a visual inspection method for SIM card encapsulation adhesive breakage, including the following steps:

[0094] S102, Acquire a multispectral image of the SIM card, wherein the multispectral image includes at least four original images;

[0095] S104, Based on the multispectral image, registration and fusion are performed to obtain a multichannel fused image tensor;

[0096] S106, Input the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask;

[0097] S108, based on the glue segmentation mask, perform glue breakage defect analysis to obtain defect data. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection.

[0098] S110, extract the multispectral image, the glue segmentation mask, and the defect data to generate a detection report, and visualize the defect area.

[0099] It should be noted that, in this embodiment, a multispectral image of the SIM card is first acquired, thereby enhancing the characteristics of the adhesive from different physical angles through a multispectral illumination scheme, overcoming the limitations of a single light source, and then registering and fusing the multispectral image to obtain a multi-channel fused image tensor, which serves as the input to the subsequent segmentation model.

[0100] Furthermore, in this embodiment, the multi-channel fused image tensor is input into the trained segmentation model to obtain a glue segmentation mask. By employing a semantic segmentation model, pixel-level accurate segmentation of the glue region is achieved, eliminating the reliance on manual features and fixed thresholds. Further, based on the glue segmentation mask, glue breakage defect analysis is performed to obtain defect data. The analysis process includes connected component analysis, skeletonization and density analysis, and internal contour detection, thereby accurately identifying and locating various glue breakage defects, including complete breakage, thin glue, and hollow areas. Finally, the multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect region is visualized. The detailed defect report provides guidance for daily production, exhibits good generalization ability, and can quickly adapt to new products and processes.

[0101] According to an embodiment of the present invention, the acquisition of multispectral images of the SIM card specifically includes:

[0102] The system controls a preset precision motion platform to move the SIM card to a preset imaging position, and triggers the camera to turn on via a preset trigger sensor.

[0103] When the coaxial white light source is turned on, the acquired image is obtained. ;

[0104] When the red low-angle light source is turned on, the acquired image is obtained. ;

[0105] When the blue low-angle light source is turned on, the acquired image is obtained. ;

[0106] When the ultraviolet light source is turned on, the acquired image is obtained. ;

[0107] Based on the acquired images ,image ,image and images The multispectral image is obtained.

[0108] It should be noted that, in this embodiment, for each SIM card, a preset precision motion platform is controlled to move the SIM card to a preset imaging station. A preset trigger sensor then triggers the camera to start. During application, the multispectral light source unit sequentially triggers coaxial light, red low-angle light, blue low-angle light, and ultraviolet light (UV light), thereby enabling the simultaneous acquisition of four images using a high-resolution color industrial camera. Specifically, the images... ,image ,image and images .

[0109] According to an embodiment of the present invention, the step of registering and fusing the multispectral image to obtain a multichannel fused image tensor specifically includes:

[0110] Select image Based on the baseline, calculate the image ,image and images The affine transformation matrix to the reference;

[0111] Apply the affine transformation matrix to the image ,image and images Transform to image Registered images obtained in the same coordinate system Image registration and registered images ;

[0112] Image Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor.

[0113] It should be noted that, in this embodiment, due to the fixed camera position, there are inherent perspective and slight positional deviations between the four images. To achieve pixel-level accurate fusion and analysis, high-precision image registration is required. Therefore, this embodiment employs an affine transformation model based on feature points (such as ORB) to obtain the images. Using the reference image as the reference image, calculate the transformation matrices from the other three images to the reference image. , , They are then transformed to the same coordinate system to obtain the registered images. Image registration and registered images .

[0114] Furthermore, in this embodiment, the registered four-channel image data are stacked as a four-channel input tensor, specifically the image... Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor. This tensor contains information about the frontal surface of the glue, information about both sides of the edge, and potential fluorescence information, providing a rich data foundation for subsequent segmentation.

[0115] According to an embodiment of the present invention, the step of inputting the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask specifically includes:

[0116] The segmentation model includes the U-Net++ segmentation model, which includes an encoder and a decoder;

[0117] The multi-channel fused image tensor is input into the U-Net++ segmentation model to obtain the glue category probability for each pixel;

[0118] The glue segmentation mask is obtained by binarizing the probability map output by the model using a threshold. .

[0119] It should be noted that, in this embodiment, the U-Net network architecture is selected as the basis for the segmentation model. U-Net enhances the information flow between the encoder and decoder layers through dense skip connections, which can better integrate shallow details and deep semantics, and is beneficial for accurately segmenting the boundaries of the glue. Specifically, the encoder (downsampling path) uses a pre-trained ResNet-34 as the backbone network to extract multi-level features. The decoder (upsampled path) is connected to dense skip connections, assuming Indicates the first The downsampling layer and the first The feature map of each upsampled node is calculated as follows:

[0120] ;

[0121] in, This represents the convolution operation. Indicates an upsampling operation. This indicates feature concatenation. Using this structure allows the decoder to acquire richer multi-scale features from the encoder, thus enabling improvements at multiple levels of the decoder. Adding auxiliary outputs and deep supervision helps alleviate the vanishing gradient problem and improve training performance. The final output of the model network is a single-channel probability map with the same resolution as the input image. Each pixel value represents the probability that the point belongs to the glue area, and a threshold is set. When applying this method, you can use "0.5" to obtain a binarized glue segmentation mask. :

[0122] .

[0123] According to an embodiment of the present invention, the step of obtaining defect data by performing glue breakage defect analysis based on the glue segmentation mask specifically includes:

[0124] When performing connected region analysis, the number of independent connected regions in the glue-segmented mask is specifically calculated. If the number is greater than a preset value, a complete break defect is determined, and the position and area of ​​each broken part are recorded.

[0125] During skeletonization and density analysis, the central skeleton line of the glue region is extracted, and the width of each sampling point is calculated along the skeleton line to obtain the width sequence. Calculate the width sequence mean and standard deviation The fine glue threshold is thus calculated based on a preset formula. , where, if the width sequence Medium sampling point width Less than the fine glue threshold If so, it is determined that there is a fine glue defect;

[0126] When performing internal contour detection, specifically, internal holes are detected within the glue-splitting mask. If the hole area is greater than a preset hole threshold... If so, it is determined that a hollow defect exists.

[0127] It should be noted that, in this embodiment, a precise glue segmentation mask is obtained. Then, connectivity analysis is performed to identify independent glue regions, where, ideally, for SIM card glue dots, it should be a continuous connected region.

[0128] Specifically, firstly, the glue is used to divide the mask. Morphological closing operations are performed to fill any possible tiny holes. Then, the minimum bounding rectangle of the connected region is calculated. Finally, a skeletonization algorithm (such as the Zhang-Suen algorithm) is used to extract the center skeleton line of the glue region. Among them, the skeleton line can simplify the topology of the glue path, along the skeleton line Sampling is performed, and the width of the glue region at each sampling point is calculated (specifically by calculating the length of the intersection between the normal at that point and the glue boundary), thus obtaining a width sequence. Then, defect assessment is performed, specifically including:

[0129] (1) Complete break detection: If the number of connected regions is greater than the preset value (which can be set to 1 in application), then a complete break is determined to exist, and the position and area of ​​each broken part are recorded.

[0130] (2) Fine glue / fracture detection: based on width sequence Make a judgment and calculate the average width. and standard deviation The fine glue threshold is calculated based on a preset formula. The preset formula is: ,in This is the sensitivity coefficient, which can be set to "2" in application. Furthermore, if there are multiple consecutive sampling points... If the narrow section has a fine glue defect, its severity can be determined by the length of the narrow section. and average width To quantify.

[0131] (3) Hollow defect detection: On the original glue-splitting mask The above steps involve performing internal contour detection on connected regions. If an internal hole (inner contour) is found, and the hole area is greater than a certain hole threshold, the detection process is repeated. If so, it is determined that a hollow defect exists.

[0132] According to an embodiment of the present invention, extracting the multispectral image, the glue segmentation mask, and the defect data to generate a detection report, and visually representing the defect region, specifically includes:

[0133] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a structured inspection report, which includes an overall conclusion, a list of defect types, defect pixel coordinates, and quantization parameters.

[0134] When visualizing, multispectral images, segmented contours, skeleton lines, and defect areas marked with different colors are displayed on the same screen.

[0135] It should be noted that, in this embodiment, the multispectral image, the glue segmentation mask, and the defect data are finally extracted to generate a structured inspection report. The inspection report includes an overall conclusion (qualified / unqualified), a list of defect types (complete breakage, thin glue, hollow), the precise pixel coordinates of each type of defect, and quantization parameters (such as the length of the thin glue, the minimum width, the hollow area, etc.).

[0136] Furthermore, in this embodiment, when visualizing the representation, such as Figure 2As shown, the display is a visualization of the detection results. The display interface shows the original image, the segmentation results, and the defect annotations on the same screen, and clearly marks the location and range of various defects with different colored highlight boxes and lines.

[0137] Figure 3 A block diagram of a visual inspection system for SIM card encapsulation adhesive breakage according to the present invention is shown.

[0138] like Figure 3 As shown, this invention discloses a visual inspection system for SIM card encapsulation adhesive breakage, including a memory and a processor. The memory includes a program for visual inspection of SIM card encapsulation adhesive breakage. When the processor executes the program for visual inspection of SIM card encapsulation adhesive breakage, it performs the following steps:

[0139] Acquire multispectral images of the SIM card, wherein the multispectral images include at least four original images;

[0140] The multi-channel fused image tensor is obtained by registration and fusion of the multispectral image.

[0141] The multi-channel fused image tensor is input into the trained segmentation model to obtain the glue segmentation mask;

[0142] Based on the glue segmentation mask, defect data is obtained by analyzing glue breakage defects. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection.

[0143] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized.

[0144] It should be noted that, in this embodiment, a multispectral image of the SIM card is first acquired, thereby enhancing the characteristics of the adhesive from different physical angles through a multispectral illumination scheme, overcoming the limitations of a single light source, and then registering and fusing the multispectral image to obtain a multi-channel fused image tensor, which serves as the input to the subsequent segmentation model.

[0145] Furthermore, in this embodiment, the multi-channel fused image tensor is input into the trained segmentation model to obtain a glue segmentation mask. By employing a semantic segmentation model, pixel-level accurate segmentation of the glue region is achieved, eliminating the reliance on manual features and fixed thresholds. Further, based on the glue segmentation mask, glue breakage defect analysis is performed to obtain defect data. The analysis process includes connected component analysis, skeletonization and density analysis, and internal contour detection, thereby accurately identifying and locating various glue breakage defects, including complete breakage, thin glue, and hollow areas. Finally, the multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect region is visualized. The detailed defect report provides guidance for daily production, exhibits good generalization ability, and can quickly adapt to new products and processes.

[0146] According to an embodiment of the present invention, the acquisition of multispectral images of the SIM card specifically includes:

[0147] The system controls a preset precision motion platform to move the SIM card to a preset imaging position, and triggers the camera to turn on via a preset trigger sensor.

[0148] When the coaxial white light source is turned on, the acquired image is obtained. ;

[0149] When the red low-angle light source is turned on, the acquired image is obtained. ;

[0150] When the blue low-angle light source is turned on, the acquired image is obtained. ;

[0151] When the ultraviolet light source is turned on, the acquired image is obtained. ;

[0152] Based on the acquired images ,image ,image and images The multispectral image is obtained.

[0153] It should be noted that, in this embodiment, for each SIM card, a preset precision motion platform is controlled to move the SIM card to a preset imaging station. A preset trigger sensor then triggers the camera to start. During application, the multispectral light source unit sequentially triggers coaxial light, red low-angle light, blue low-angle light, and ultraviolet light (UV light), thereby enabling the simultaneous acquisition of four images using a high-resolution color industrial camera. Specifically, the images... ,image ,image and images .

[0154] According to an embodiment of the present invention, the step of registering and fusing the multispectral image to obtain a multichannel fused image tensor specifically includes:

[0155] Select image Based on the baseline, calculate the image ,image and images The affine transformation matrix to the reference;

[0156] Apply the affine transformation matrix to the image ,image and images Transform to image Registered images obtained in the same coordinate system Image registration and registered images ;

[0157] Image Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor.

[0158] It should be noted that, in this embodiment, due to the fixed camera position, there are inherent perspective and slight positional deviations between the four images. To achieve pixel-level accurate fusion and analysis, high-precision image registration is required. Therefore, this embodiment employs an affine transformation model based on feature points (such as ORB) to obtain the images. Using the reference image as the reference image, calculate the transformation matrices from the other three images to the reference image. , , They are then transformed to the same coordinate system to obtain the registered images. Image registration and registered images .

[0159] Furthermore, in this embodiment, the registered four-channel image data are stacked as a four-channel input tensor, specifically the image... Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor. This tensor contains information about the frontal surface of the glue, information about both sides of the edge, and potential fluorescence information, providing a rich data foundation for subsequent segmentation.

[0160] According to an embodiment of the present invention, the step of inputting the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask specifically includes:

[0161] The segmentation model includes the U-Net++ segmentation model, which includes an encoder and a decoder;

[0162] The multi-channel fused image tensor is input into the U-Net++ segmentation model to obtain the glue category probability for each pixel;

[0163] The glue segmentation mask is obtained by binarizing the probability map output by the model using a threshold. .

[0164] It should be noted that, in this embodiment, the U-Net network architecture is selected as the basis for the segmentation model. U-Net enhances the information flow between the encoder and decoder layers through dense skip connections, which can better integrate shallow details and deep semantics, and is beneficial for accurately segmenting the boundaries of the glue. Specifically, the encoder (downsampling path) uses a pre-trained ResNet-34 as the backbone network to extract multi-level features. The decoder (upsampled path) is connected to dense skip connections, assuming Indicates the first The downsampling layer and the first The feature map of each upsampled node is calculated as follows:

[0165] ;

[0166] in, This represents the convolution operation. Indicates an upsampling operation. This indicates feature concatenation. Using this structure allows the decoder to acquire richer multi-scale features from the encoder, thus enabling improvements at multiple levels of the decoder. Adding auxiliary outputs and deep supervision helps alleviate the vanishing gradient problem and improve training performance. The final output of the model network is a single-channel probability map with the same resolution as the input image. Each pixel value represents the probability that the point belongs to the glue area, and a threshold is set. When applying this method, you can use "0.5" to obtain a binarized glue segmentation mask. :

[0167] .

[0168] According to an embodiment of the present invention, the step of obtaining defect data by performing glue breakage defect analysis based on the glue segmentation mask specifically includes:

[0169] When performing connected region analysis, the number of independent connected regions in the glue-segmented mask is specifically calculated. If the number is greater than a preset value, a complete break defect is determined, and the position and area of ​​each broken part are recorded.

[0170] During skeletonization and density analysis, the central skeleton line of the glue region is extracted, and the width of each sampling point is calculated along the skeleton line to obtain the width sequence. Calculate the width sequence mean and standard deviation The fine glue threshold is thus calculated based on a preset formula. , where, if the width sequence Medium sampling point width Less than the fine glue threshold If so, it is determined that there is a fine glue defect;

[0171] When performing internal contour detection, specifically, internal holes are detected within the glue-splitting mask. If the hole area is greater than a preset hole threshold... If so, it is determined that a hollow defect exists.

[0172] It should be noted that, in this embodiment, a precise glue segmentation mask is obtained. Then, connectivity analysis is performed to identify independent glue regions, where, ideally, for SIM card glue dots, it should be a continuous connected region.

[0173] Specifically, firstly, the glue is used to divide the mask. Morphological closing operations are performed to fill any possible tiny holes. Then, the minimum bounding rectangle of the connected region is calculated. Finally, a skeletonization algorithm (such as the Zhang-Suen algorithm) is used to extract the center skeleton line of the glue region. Among them, the skeleton line can simplify the topology of the glue path, along the skeleton line Sampling is performed, and the width of the glue region at each sampling point is calculated (specifically by calculating the length of the intersection between the normal at that point and the glue boundary), thus obtaining a width sequence. Then, defect assessment is performed, specifically including:

[0174] (1) Complete break detection: If the number of connected regions is greater than the preset value (which can be set to 1 in application), then a complete break is determined to exist, and the position and area of ​​each broken part are recorded.

[0175] (2) Fine glue / fracture detection: based on width sequence Make a judgment and calculate the average width. and standard deviation The fine glue threshold is calculated based on a preset formula. The preset formula is: ,in This is the sensitivity coefficient, which can be set to "2" in application. Furthermore, if there are multiple consecutive sampling points... If the narrow section has a fine glue defect, its severity can be determined by the length of the narrow section. and average width To quantify.

[0176] (3) Hollow defect detection: On the original glue-splitting mask The above steps involve performing internal contour detection on connected regions. If an internal hole (inner contour) is found, and the hole area is greater than a certain hole threshold, the detection process is repeated. If so, it is determined that a hollow defect exists.

[0177] According to an embodiment of the present invention, extracting the multispectral image, the glue segmentation mask, and the defect data to generate a detection report, and visually representing the defect region, specifically includes:

[0178] The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a structured inspection report, which includes an overall conclusion, a list of defect types, defect pixel coordinates, and quantization parameters.

[0179] When visualizing, multispectral images, segmented contours, skeleton lines, and defect areas marked with different colors are displayed on the same screen.

[0180] It should be noted that, in this embodiment, the multispectral image, the glue segmentation mask, and the defect data are finally extracted to generate a structured inspection report. The inspection report includes an overall conclusion (qualified / unqualified), a list of defect types (complete breakage, thin glue, hollow), the precise pixel coordinates of each type of defect, and quantization parameters (such as the length of the thin glue, the minimum width, the hollow area, etc.).

[0181] Furthermore, in this embodiment, when visualizing the representation, such as Figure 2 As shown, the display is a visualization of the detection results. The display interface shows the original image, the segmentation results, and the defect annotations on the same screen, and clearly marks the location and range of various defects with different colored highlight boxes and lines.

[0182] A third aspect of the present invention provides a computer-readable storage medium comprising a program for visually detecting adhesive breakage in a SIM card package. When executed by a processor, the program implements the steps of the visually detecting adhesive breakage in a SIM card package as described in any of the preceding claims.

[0183] A fourth aspect of the present invention provides a visual inspection device for SIM card encapsulation adhesive breakage, comprising:

[0184] The system comprises an acquisition unit, a multispectral light source unit, a triggering unit, and a computing unit.

[0185] The acquisition unit includes a high-resolution color industrial camera for acquiring color images of the SIM card under different lighting conditions;

[0186] The multispectral light source unit includes several independently controlled light sources used to generate different lighting conditions;

[0187] The carrier triggering unit includes a precision motion platform and a trigger sensor. The precision motion platform is used to carry and position the SIM card, and the trigger sensor is used to trigger the camera to turn on.

[0188] The computing unit includes an industrial computer with a high-performance GPU, used to execute the visual inspection method for SIM card encapsulation adhesive breakage described in any of the above-mentioned embodiments.

[0189] It should be noted that, in this embodiment, reference is made to... Figure 4 The diagram shows the structure of the device, in which the carrier triggering unit and the computing unit are not shown. The carrier triggering unit specifically includes a precision motion platform and a trigger sensor. The precision motion platform is used to carry and position the SIM card. The trigger sensor is used to activate the trigger camera. The computing unit includes an industrial computer with a high-performance GPU, used to execute the visual inspection method for SIM card encapsulation adhesive breakage described above.

[0190] Furthermore, in this embodiment, the acquisition unit includes a high-resolution color industrial camera for acquiring color images of the SIM card under different lighting conditions. The multispectral light source unit includes several independently controlled light sources for generating different lighting conditions. Specifically, the multispectral light source unit consists of four independently controllable light sources arranged around the lens of the high-resolution color industrial camera. The light sources in the multispectral light source unit include a coaxial white light source, a red low-angle bar light source, a blue low-angle bar light source, and an ultraviolet light source. Specifically, the coaxial white light source provides uniform illumination in the vertical direction for... The system captures the color and overall shape of the adhesive from the front. A red low-angle strip light source is incident from one side at a low angle (15°-30° in application) to highlight the edge contour of one side of the adhesive. A blue low-angle strip light source is incident from the other side at a low angle to highlight the edge contour of the other side of the adhesive. The alternating or simultaneous use of red and blue light sources can capture images with directional edge information. Ultraviolet light (UV light): UV light of specific wavelengths can be used to excite fluorescent substances in some adhesives, causing the adhesive to emit a specific color of light against a dark background, greatly enhancing the contrast between the adhesive and the background, especially suitable for transparent adhesives.

[0191] Furthermore, in this embodiment, the device also includes an optical filter wheel (not shown), which is installed in front of a high-resolution color industrial camera and can switch between different wavelength bandpass filters as needed to further suppress ambient light interference and improve the signal-to-noise ratio of UV-excited images.

[0192] This invention discloses a visual inspection method, system, storage medium, and device for SIM card encapsulation adhesive breakage, which can perform high-precision and robust automatic detection of SIM card encapsulation adhesive breakage defects, effectively identify a variety of complex defects, significantly improve detection efficiency and adaptability, and reduce false detection rate and maintenance costs.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0194] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0195] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0196] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A visual inspection method for SIM card encapsulation adhesive breakage, characterized in that, Includes the following steps: Acquire multispectral images of the SIM card, wherein the multispectral images include at least four original images; The multi-channel fused image tensor is obtained by registration and fusion of the multispectral image. The multi-channel fused image tensor is input into the trained segmentation model to obtain the glue segmentation mask; Based on the glue segmentation mask, defect data is obtained by analyzing glue breakage defects. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection. The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized.

2. The visual inspection method for SIM card encapsulation adhesive breakage according to claim 1, characterized in that, The acquisition of multispectral images from the SIM card specifically includes: The system controls a preset precision motion platform to move the SIM card to a preset imaging position, and triggers the camera to turn on via a preset trigger sensor. When the coaxial white light source is turned on, the acquired image is obtained. ; When the red low-angle light source is turned on, the acquired image is obtained. ; When the blue low-angle light source is turned on, the acquired image is obtained. ; When the ultraviolet light source is turned on, the acquired image is obtained. ; Based on the acquired images ,image ,image and images The multispectral image is obtained.

3. The visual inspection method for SIM card encapsulation adhesive breakage according to claim 2, characterized in that, The process of registering and fusing the multispectral image to obtain the multichannel fused image tensor specifically includes: Select image Based on the baseline, calculate the image ,image and images The affine transformation matrix to the reference; Apply the affine transformation matrix to the image ,image and images Transform to image Registered images obtained in the same coordinate system Image registration and registered images ; Image Image registration Image registration and registered images Stacking along the channel dimension yields a four-channel multi-channel fused image tensor.

4. The visual inspection method for SIM card encapsulation adhesive breakage according to claim 3, characterized in that, The step of inputting the multi-channel fused image tensor into the trained segmentation model to obtain the glue segmentation mask specifically includes: The segmentation model includes the U-Net++ segmentation model, which includes an encoder and a decoder; The multi-channel fused image tensor is input into the U-Net++ segmentation model to obtain the glue category probability for each pixel; The glue segmentation mask is obtained by binarizing the probability map output by the model using a threshold. .

5. The visual inspection method for SIM card encapsulation adhesive breakage according to claim 4, characterized in that, The defect data obtained by analyzing adhesive breakage defects based on the adhesive segmentation mask specifically includes: When performing connected region analysis, the number of independent connected regions in the glue-segmented mask is specifically calculated. If the number is greater than a preset value, a complete break defect is determined, and the position and area of ​​each broken part are recorded. During skeletonization and density analysis, the central skeleton line of the glue region is extracted, and the width of each sampling point is calculated along the skeleton line to obtain the width sequence. Calculate the width sequence mean and standard deviation The fine glue threshold is thus calculated based on a preset formula. , where, if the width sequence Medium sampling point width Less than the fine glue threshold If so, it is determined that there is a fine glue defect; When performing internal contour detection, specifically, internal holes are detected within the glue-splitting mask. If the hole area is greater than a preset hole threshold... If so, it is determined that a hollow defect exists.

6. The visual inspection method for SIM card encapsulation adhesive breakage according to claim 5, characterized in that, Extracting the multispectral image, the glue segmentation mask, and the defect data to generate a detection report, and visually representing the defect region, specifically includes: The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a structured inspection report, which includes an overall conclusion, a list of defect types, defect pixel coordinates, and quantization parameters. When visualizing, multispectral images, segmented contours, skeleton lines, and defect areas marked with different colors are displayed on the same screen.

7. A visual inspection system for SIM card encapsulation adhesive breakage, characterized in that, The system includes a memory and a processor. The memory includes a program for visually detecting adhesive breakage in SIM card packaging. When the processor executes the program for visually detecting adhesive breakage in SIM card packaging, it performs the following steps: Acquire multispectral images of the SIM card, wherein the multispectral images include at least four original images; The multi-channel fused image tensor is obtained by registration and fusion of the multispectral image. The multi-channel fused image tensor is input into the trained segmentation model to obtain the glue segmentation mask; Based on the glue segmentation mask, defect data is obtained by analyzing glue breakage defects. The analysis process includes connected region analysis, skeletonization and density analysis, and internal contour detection. The multispectral image, the glue segmentation mask, and the defect data are extracted to generate a detection report, and the defect area is visualized.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a SIM card encapsulation adhesive breakage visual inspection method program. When the SIM card encapsulation adhesive breakage visual inspection method program is executed by a processor, it implements the steps of the SIM card encapsulation adhesive breakage visual inspection method as described in any one of claims 1 to 6.

9. A visual inspection device for SIM card encapsulation adhesive breakage, characterized in that, include: The system comprises an acquisition unit, a multispectral light source unit, a triggering unit, and a computing unit. The acquisition unit includes a high-resolution color industrial camera for acquiring color images of the SIM card under different lighting conditions; The multispectral light source unit includes several independently controlled light sources used to generate different lighting conditions; The carrier triggering unit includes a precision motion platform and a trigger sensor. The precision motion platform is used to carry and position the SIM card, and the trigger sensor is used to trigger the camera to turn on. The computing unit includes an industrial computer with a high-performance GPU, used to execute a visual inspection method for SIM card encapsulation adhesive breakage as described in any one of claims 1-6.

10. A visual inspection device for SIM card encapsulation adhesive breakage according to claim 9, characterized in that, The device also includes an optical filter wheel, which is mounted in front of a high-resolution color industrial camera, and the light source in the multispectral light source unit includes a coaxial white light source, a red low-angle bar light source, a blue low-angle bar light source, and an ultraviolet light source.

Images