Conductive particle detection method based on hybrid detection strategy

Abstract

The invention relates to the technical field of image processing and intelligent detection, and particularly discloses a conductive particle detection method based on a hybrid detection strategy, which integrates a rule-based traditional image processing method and a deep learning-based intelligent detection algorithm, and adopts a dual-mode switching mechanism. And high-precision and high-efficiency conductive particle detection at different production stages is realized. Specifically, a parameter configuration mechanism of a traditional detection algorithm is guided through a deep learning detection model, a mapping relation from internal features of the deep model and a prediction result to traditional configuration parameters is established, and rapid adaptation and automatic deployment of the configuration parameters required by the detection algorithm when a novel panel is imported are realized. The method gives consideration to both stability and flexibility, and is suitable for conducting particle intelligent detection scenes of a semiconductor panel production line.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial visual inspection, and particularly relates to a hybrid detection method suitable for automatic recognition and classification of conductive particles, and suitable for conductive particle detection and counting tasks in the production of semiconductor display panels. BACKGROUND

[0002] With the continuous progress of display panel manufacturing technology, conductive particles, as the key electrical connection unit in the TFT-LCD packaging process, the integrity of their microstructure, distribution state and physical appearance have become important indicators affecting the yield and reliability of finished products. The evolution of automatic conductive particle detection algorithms mainly experienced three stages: traditional non-learning algorithm (NL), machine learning method (ML) to the current mainstream deep learning method (DL). In the early research of conductive particle detection algorithm, considering that the conductive particles usually show a typical "light / dark symmetric structure" in the imaging process, a series of particle recognition strategies based on gray extremum extraction and gradient information enhancement are developed based on image gray change. This kind of method can be collectively referred to as the traditional detection method based on gray gradient. The existing patent discloses a conductive particle detection method (publication number: CN119850616 A), which detects the maximum value of the local area by using a sliding window method. This kind of method has the characteristics of low sensitivity to imaging conditions and high calculation efficiency, but it is easy to cause segmentation error in the case of particle adhesion or edge blur. The deep learning network model widely used in the current conductive particle detection task is mainly represented by U-Net and its variants. The existing patent discloses a conductive particle detection method based on circular convolution (publication number: CN118247218A), which designs a neural network model based on "circular convolution" sampling strategy. However, this kind of method lacks adaptive updating mechanism after deployment and does not fully consider the geometric features of the samples to be tested, which has the risk of precision fluctuation in long-term detection tasks.

[0003] In summary, most of the existing methods are based on image segmentation or object detection network framework, and excessively rely on the distribution prior of training data, but fail to fully integrate the structural physical properties of conductive particles. Due to the rapid iteration of panel models and production processes, the existing detection algorithms generally lack dynamic learning ability and structural adaptive updating mechanism. When facing new panel models (such as changes in conductive layer thickness and electrode size), the existing methods often have problems such as precision drop, false detection and missed detection, which need to rely on manual feedback and manual adjustment, greatly increasing the resource consumption and time cost. Therefore, the present application proposes a conductive particle detection method based on a hybrid detection strategy. SUMMARY

[0004] To solve the problems of poor adaptability, complicated parameter configuration and low detection stability of the existing conductive particle detection method, the application discloses a conductive particle detection method based on a hybrid detection strategy, a hybrid detection strategy combining a deep learning model and a traditional image processing algorithm, an automatic mapping mechanism from deep features to traditional parameter configuration is established, and a conductive particle automatic detection system with high adaptability, high efficiency and high stability is realized.

[0005] To achieve the application purpose, the specific technical solutions of the application are as follows:

[0006] A conductive particle detection method based on a hybrid detection strategy, comprising the following steps:

[0007] S1, image acquisition and input: acquiring a conductive particle detection area image and inputting it to a deep learning detection module;

[0008] S2, model prediction and parameter extraction: generating a particle mask image through a deep segmentation model, and extracting intermediate feature parameters of the model;

[0009] S3, parameter mapping: based on the intermediate features and the output mask, a parameter mapping function is used to generate the convolution kernel scale, binary threshold and area threshold required by the traditional image processing algorithm;

[0010] S4, detection mode switching: in the test stage, the deep learning detection model is used to output the detection result; in the mass production stage, the parameters generated in step S3 are used to call the traditional image processing module for conductive particle detection;

[0011] S5, abnormal feedback and automatic rollback: when an abnormality is found in the traditional detection, the deep learning model is automatically switched back and the parameter template is regenerated.

[0012] The application further comprises an image preprocessing, edge enhancement, gradient threshold segmentation and connected domain analysis.

[0013] The application further comprises a U-shaped structure network with introduced circular convolution and channel attention mechanism, which is used to enhance the modeling ability of edge details and structural texture.

[0014] The application further comprises the following sub-steps in the parameter mapping process:

[0015] a, calculating the target average area and average outline size of the model prediction mask image;

[0016] b, extracting the size control factor and confidence response value from the model intermediate layer;

[0017] c. The above information is converted into three sets of configuration parameters required by traditional algorithms according to a set of preset functions. The parameter mapping process avoids the manual parameter adjustment step each time the sample type is changed, greatly improving the deployment efficiency and detection consistency.

[0018] Advantages of the present application:

[0019] (1) High applicability to various types of panels, automatic extraction of parameter templates, no manual parameter adjustment is required;

[0020] (2) High stability of the detection process, preferentially using lightweight traditional algorithms in the stable running stage;

[0021] (3) Automatic rollback mechanism: ensures automatic correction of configuration when misdetected, maintains detection reliability;

[0022] (4) Suitable for batch panel detection scenarios: supports industrial production rhythm and robustness requirements. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of a conductive particle detection method based on a hybrid detection strategy of the present application.

[0024] Figure 2 is a flowchart of a traditional conductive particle detection algorithm in a hybrid detection strategy of the present application.

[0025] Figure 3 is a mapping process of a neural network model to a traditional method configuration recommendation parameter in a hybrid detection strategy of the present application. DETAILED DESCRIPTION

[0026] The present application will be further illustrated below in conjunction with the drawings and specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application. It should be noted that the words "front", "back", "left", "right", "up" and "down" used in the following description refer to the directions in the drawings, and the words "in" and "out" refer to the directions towards or away from the geometric center of a particular component.

[0027] The conductive particle detection method based on the hybrid detection strategy is described in detail. The present application aims to integrate the advantages of deep learning methods and traditional image processing methods, and to realize efficient and stable detection of conductive particles through a parameter mapping mechanism, especially suitable for automatic optical detection scenarios in the display panel pressing process.

[0028] Referring to Figure 1 , in the present application, an industrial camera is first used to collect high-resolution images of the target detection area. After uniform size scaling and normalization processing, the images are input into the conductive particle detection neural network model.

[0029] The conductive particle detection neural network model used by the present application is based on an improved U-Net structure, and introduces a circular convolution structure and an attention mechanism to enhance the response capability to the edges and context of different morphological conductive particles. The output of the model includes two parts: one is a conductive particle mask map X of the same size as the original map, which is used to represent the spatial distribution of the particles; the other is the internal structure parameters of the model after the prediction is completed.

[0030] The internal structure parameters of the model in the output results of the neural network model of the present application are control parameters scale_n for controlling the scale of the circular convolution sampling region.

[0031] Referring to Figure 2 , the conventional conductive particle detection algorithm used by the present application mainly consists of two parts: convolution & binarization and connected domain clustering & splitting. The convolution & binarization part is responsible for feature extraction of the original image, and the connected domain clustering & splitting part is responsible for original feature decoding and post-processing. After the detection algorithm receives the signal of the input image source, it will first expand the one-way Sobel operator weight according to the operator size parameter n, and then use the convolution kernel composed of the weight to perform the convolution process, extract the features of the original image, and then binarize the results according to the binarization threshold parameter s1, and filter out the parts that meet the requirements of matching the standard conductive particle gray template, while removing false points and noise; after the feature extraction is completed, the filtered results close to each other are clustered into groups through connected domain marking, and the area of each group is calculated. If the area of the group exceeds twice the area threshold s2, it is considered that the group is formed by multiple conductive particles adhering to each other, and for such a group, the area threshold s2 is used to continue to split it, and the number of splits is determined by the quotient of the area of the group and s2, and the integer value after rounding off, and the center of mass point is allocated to the group according to the principle of uniform distribution, as the center of each conductive particle after splitting; otherwise, it is considered that the group is a single conductive particle, and the center of mass and the area of the group are the center and size of the conductive particle. Finally, the recognized conductive particles are marked and identified in the image as circles.

[0032] Referring to Figure 3 , the parameter mapping mechanism of the present application converts the intermediate features of the above-mentioned deep learning detection model into the parameter template of the traditional detection algorithm. The recommended operator scale parameter n recommend in the traditional method is mapped by the scale control parameter scale_n in the deep learning model, using the mean value taking and the nearest odd number operation:

[0033]

[0034] In the formula, N is the total number of the size control parameter set in the network model, and p represents each size control parameter.

[0035] The recommended binarization threshold parameter s1 in the traditional method recommend And the recommended area threshold parameter s2 recommend First, the original image I(x, y) is subjected to a convolution operation, and the convolution kernel uses the recommended value n of the operator scale parameter given by the neural network, which is mapped from the prediction mask X recommend The gray gradient image obtained after convolution The prediction mask X is added, and the gray gradient of the foreground area (conductive particle area) and the background area in the gray gradient image of the prediction mask has obvious difference, the gray gradient histogram of the prediction mask is drawn, and the gray gradient distribution of the foreground area and the background area is counted respectively, and there will be a clear dividing line between the two distribution peaks, which can be used as the binarization threshold s1 in the traditional method recommend :

[0036]

[0037] In the formula, OTSU is an algorithm for finding the dividing point by maximizing the inter-class (foreground particles and background panels) variance.

[0038] The area distribution characteristics of the particles in the mask image are statistically analyzed, and the average area of the particle area is calculated, that is, the area threshold s2 in the traditional method can be mapped and derived recommend :

[0039]

[0040] In the formula, n is the number of conductive particles in the mask image, and A i is the area of each conductive particle.

[0041] The operation of the embodiment is divided into two modes: debugging and verification mode and mass production and stable mode. In the debugging and verification mode, the deep learning detection model is used to directly detect the conductive particles and output the mapping parameters.

[0042] In the mass production mode, the system skips the deep learning detection model and directly uses the existing parameter template to call the traditional algorithm for rapid detection, thereby greatly reducing the inference calculation amount and hardware load. The switching between the two modes is automatically completed by the detection system state control module according to the detection result stability and image distribution.

[0043] The conductive particle hybrid detection method proposed in the application fuses the adaptability of the deep learning model and the high efficiency of the traditional image processing algorithm, and realizes the migration of model knowledge to the rule method through the parameter mapping mechanism, and is suitable for industrial detection tasks with certain morphological stability and image structure regularity, and has good application prospect in the fields of display, semiconductor, PCB, etc.

[0044] The technical means disclosed in the present application are not limited to the technical means disclosed in the above-mentioned embodiments, and include technical solutions composed of any combination of the above technical features.

Claims

1. A method for detecting conductive particles based on a hybrid detection strategy, characterized in that, Includes the following steps: S1. Image Acquisition and Input: Acquire images of the conductive particle detection area and input them into the deep learning detection module; S2. Model Prediction and Parameter Extraction: A particle mask map is generated through the deep segmentation model, and the intermediate feature parameters of the model are extracted at the same time. S3. Parameter mapping: Based on the intermediate features and the output mask, the convolution kernel scale, binarization threshold and area threshold required by traditional image processing algorithms are generated using the parameter mapping function. S4. Detection mode switching: During the testing phase, a deep learning detection model is used to output detection results; during the mass production phase, the parameters generated in step S3 are used to call the traditional image processing module to detect conductive particles. S5. Anomaly Feedback and Automatic Rollback: When an anomaly is detected in traditional detection, the system automatically switches back to the deep learning model and regenerates the parameter template.

2. The conductive particle detection method based on a hybrid detection strategy according to claim 1, characterized in that, The depth segmentation model in S2 employs a U-shaped network that incorporates circular convolution and channel attention mechanisms to enhance the modeling ability of edge details and structural textures.

3. The conductive particle detection method based on a hybrid detection strategy according to claim 1, characterized in that, The S3 parameter mapping process includes the following sub-steps: S31: Calculate the average area and average contour size of the target based on the model prediction mask; S32: Extract size control factors and confidence response values ​​from the intermediate layer of the model; S33: Based on a set of preset functions, the model prediction mask and the information extracted from the intermediate layers of the model are converted into three sets of configuration parameters required by traditional algorithms.

4. The conductive particle detection method based on a hybrid detection strategy according to claim 1, characterized in that, The traditional image processing algorithm in S3 includes image preprocessing, edge enhancement, gradient thresholding segmentation, and connected component analysis.

5. The conductive particle detection method based on a hybrid detection strategy according to claim 1, characterized in that, The S4 detection mode switching is divided into debugging and verification mode and mass production operation mode, and has an automatic switching and parameter refresh mechanism.

6. An application of a conductive particle detection method based on a hybrid detection strategy, characterized in that, It is suitable for industrial vision inspection tasks involving various geometrically stable conductive particles, and is especially suitable for online inspection of conductive particles in panel lamination processes.

Images