Anisotropic conductive film conductive particle detection method based on SegNet architecture

Abstract

The invention provides an anisotropic conductive film conductive particle detection method based on SegNet architecture, and aims to solve the problems of low conductive particle region segmentation efficiency and poor precision in industrial production. The method comprises the steps of performing graying, contrast enhancement and noise reduction processing on an input image to generate a preprocessed image; a SegNet network model based on an encoder-decoder structure is constructed, a VGG-16 front 13-layer convolutional network is adopted by an encoder, a multi-scale convolution kernel is introduced into a shallow layer, and cavity convolution is adopted by a deep layer to enlarge a receptive field; the decoder recovers the spatial resolution through deconvolution and jump connection, and introduces a spatial attention module to focus a conductive particle region; a weighted cross entropy and Dice loss function combination optimization model is adopted, and the training stability is improved through dynamic learning rate scheduling. According to the method, the precision and robustness of conductive particle detection are remarkably improved, efficient and automatic industrial-grade detection is achieved, and remarkable economic benefits and social benefits are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of anisotropic conductive film conductive particle detection technology, and particularly to a method for detecting anisotropic conductive film conductive particles based on a SegNet architecture. Background Art

[0002] During the LCM production process, quality monitoring of the conductive microspheres used in LCM anisotropic conductive film is a core component of LCM product reliability and has a significant impact on LCM product reliability. Quality monitoring of LCM anisotropic conductive film conductive microspheres is becoming increasingly significant for the development of the LCM industry, becoming a key support method for ensuring LCM quality, improving production processes, enhancing brand effectiveness, and gaining a larger market share.

[0003] In existing LCM production lines, anisotropic conductive film (ACF) microspheres are often inspected and evaluated offline, using manual visual inspection with the aid of microscopes and other optical instruments. Each piece takes 5-10 minutes to inspect, making it a labor-intensive process. Traditional production efficiency is approximately 1,000 pieces per hour, or one piece every 4 seconds. However, this traditional manual visual inspection method is susceptible to various factors, such as the inspector's condition and experience. Therefore, both speed and reliability make it difficult to meet the quality and production capacity requirements of LCM production.

[0004] Manual microscope imaging cannot meet the requirements of industrial online batch testing. In actual industrial applications, LCM testing capacity is in high demand, and inspections typically need to be completed in around 4 seconds. Traditional analysis using microscope imaging is far from meeting the production capacity requirements of enterprises.

[0005] Pin segmentation in a touchscreen ROI is much more complex than template matching using a marker image. Using the grayscale distribution feature matching method in the marker image template matching method can lead to two problems: First, the effect of uneven local illumination. Because the marker image is large and has distinct features in the touchscreen image, it is less affected by illumination. The pin region image is smaller and more affected by illumination, so the aforementioned method may result in mismatches or missed matches. Second, the touchscreen ROI contains multiple types of pins, and different pin types may have similar grayscale distributions. Therefore, using only a matching method based on grayscale distribution features may result in matching different types of pins. Different pin types may have different numbers of anisotropic conductive film particles, and mismatched pins may fall into the incorrect judgment criteria. Therefore, it is necessary to find a more effective pin segmentation method. To this end, it is necessary to study a method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide an anisotropic conductive film conductive particle detection method based on the SegNet architecture, which effectively solves the problem that pin segmentation is prone to mismatching or missed matching during the existing anisotropic conductive film conductive particle detection process, and has poor segmentation accuracy.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is: a method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture, comprising the following steps: Step 1: pre-process the received input image by gray-scaling, contrast enhancement, and noise reduction in order to eliminate uneven illumination and noise interference, thereby generating a pre-processed image. Step 2: Normalization operation Normalize the image size, calculate the mean and standard deviation of the input image, and then perform normalization; Step 3: Create Segnet network training labels Collect and prepare images and corresponding label data for training. Each label image has a corresponding pixel-level label, and the size of the label image should match the input image size. Step 4: Network Construction A SegNet network model based on an encoder-decoder structure was constructed. The encoder is based on the first 13 convolutional layers of VGG-16 to extract multi-scale features of the image. The decoder gradually restores the spatial resolution through deconvolution and index-guided upsampling, and concatenates the feature maps of the corresponding encoder layer with the upsampled feature maps through skip connections. Step 5: Model training The SegNet network model is trained using a combination of weighted cross entropy loss function and Dice loss function as the optimization target. A dynamic learning rate scheduling strategy and the Adam optimization algorithm are used to update the network weights and save the model weights with the best performance during training. Step 6: Conductive particle area segmentation The anisotropic conductive film image to be segmented is input into the trained SegNet network model, and a segmentation probability map is generated through forward propagation. The segmentation probability map is thresholded to generate a binary segmentation mask; through morphological filtering and connected domain analysis, the final conductive particle area segmentation result is output.

[0008] Furthermore, an image annotation tool is used to annotate the conductive particle area at the pixel level and generate a label image with the same size as the original image.

[0009] Furthermore, convolution kernels of different sizes are introduced in the shallow layer of the encoder to extract multi-scale features in parallel; and dilated convolution is introduced in the deep layer of the encoder to capture multi-scale features.

[0010] Furthermore, in the first and second layers of the encoder, 1×1, 3×3, and 5×5 convolution kernels are used in parallel to reduce the channel dimension through 1×1 convolution, and then multi-scale features are extracted through 3×3 and 5×5 convolutions.

[0011] Furthermore, the deep layers are the third and fourth layers. In the third and fourth layers of the encoder, dilated convolutions with dilation rates of 2 and 4 are used to expand the receptive field to capture multi-scale contextual information.

[0012] Furthermore, a batch normalization layer and a Dropout layer are added after each dilated convolutional layer.

[0013] Furthermore, the pooling index saved by the encoder is used for nonlinear upsampling, and a 5×5 convolution kernel is used in the deconvolution layer of the decoder, combined with feature fusion of jump connections to restore spatial information; residual connections are introduced after each deconvolution block of the decoder to enhance gradient propagation.

[0014] Furthermore, a spatial attention module is introduced after the deconvolution block of the decoder to dynamically adjust the weights of the feature map and focus on the conductive particle area.

[0015] Furthermore, global average pooling and global maximum pooling are performed on the input feature map to generate two 1×1×C feature vectors. The two feature vectors are concatenated and then subjected to a 7×7 convolution to generate a spatial weight map. The weight map is multiplied element-by-element with the input feature map to focus on the conductive particle area.

[0016] Furthermore, a combination of weighted cross entropy loss function and Dice loss function is adopted, and the formula is: ; Among them, α is the balance factor; ; Where N is the number of samples, C is the number of categories, Wc is the weight of category C, y i,c is the true label, p i,c is the predicted probability; ; Among them, p i is the predicted probability, y i is the true label, and ε is the smoothing term.

[0017] The beneficial effects of the above technical solution are as follows: the present invention adopts the SegNet architecture based on deep encoders and decoders to realize the detection of anisotropic conductive film interconnect packaging areas in LCM DIC (differential interference contrast) images. SegNet extracts features and generates category probabilities through multi-layer convolution and pooling operations, and has the advantages of being simple to understand and highly robust.

[0018] At the same time, when constructing the model of the present invention, by using convolution kernels of different sizes in parallel at the shallow level of the encoder, combined with channel dimensionality reduction and multi-scale feature fusion, it is possible to simultaneously capture the local edge details and regional texture features of the conductive particles, so that the model's ability to detect tiny particles is significantly improved, avoiding the problem of missed detection due to small particle size or blurred edges. Introducing dilated convolution at the deep level of the encoder, by expanding the receptive field to capture the global contextual information of the conductive particle area, such as particle distribution density and aggregation pattern, effectively solves the problem of particle adhesion or missing misjudgment caused by insufficient receptive field in traditional methods, and improves the integrity and accuracy of the segmentation results. By generating a spatial weight map through global pooling and convolution, the model can adaptively focus on the conductive particle area while suppressing background noise. The dynamic weighting mechanism is used to significantly enhance the characteristic response of the particle area and reduce the interference of background interference on the detection results.

[0019] The present invention replaces manual inspection with automated inspection. The system significantly improves production efficiency and reduces labor costs, realizes high-precision, strong robustness, and high-efficiency conductive particle detection, and provides an intelligent and sustainable quality monitoring solution for industrial production, with significant economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural block diagram of the implementation of the present invention; Figure 2 This is the image processing effect diagram; Figure 3 This is a label diagram; Figure 4 Schematic diagram of the segmentation processing structure. DETAILED DESCRIPTION

[0021] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1. This embodiment aims to provide a method for detecting conductive particles in anisotropic conductive films based on the SegNet architecture, which is mainly used for detecting conductive particles in anisotropic conductive films. This embodiment adopts the SegNet architecture based on a deep encoder and decoder to realize the detection of anisotropic conductive film interconnect packaging areas in the DIC (differential interference contrast) image of LCM. SegNet extracts features and generates category probabilities through multi-layer convolution and pooling operations, and has the advantages of being simple to understand and highly robust.

[0022] Before detecting conductive particles, the location information of the conductive particle regions must be known. This requires first constructing a standard file from an image of a standard anisotropic conductive film product and recording the location information of the conductive particle regions in the standard file. Subsequent algorithms read the location information stored in the standard file to detect the conductive particles and calculate the detection results for each region. An anisotropic conductive film product contains numerous conductive particle regions, and selecting them one by one is extremely time-consuming and does not meet actual production needs. Therefore, automatic segmentation of the conductive particle regions in anisotropic conductive film is essential.

[0023] This example utilizes the SegNet network to extract conductive particle regions, completing semantic segmentation of images containing target objects. Compared to previously proposed region extraction methods, this method is more streamlined. First, it eliminates the need for texture analysis or feature transformation, enabling direct localization. Second, it eliminates the tedious and lengthy process of splitting and then synthesizing a complete image, reducing errors during the synthesis process and significantly improving recognition efficiency, accuracy, and robustness.

[0024] When implementing it specifically, Figure 1 The paper presents a method for detecting conductive particles in anisotropic conductive films based on the SegNet architecture, aiming to address the problems of low efficiency and poor accuracy in segmenting conductive particle regions in industrial production. The method includes the following steps: grayscale, contrast enhancement, and noise reduction processing of the input image to generate a preprocessed image; constructing a SegNet network model based on an encoder-decoder structure, where the encoder uses the first 13 layers of the VGG-16 convolutional network, introduces multi-scale convolution kernels in the shallow layers, and uses dilated convolutions in the deep layers to expand the receptive field; the decoder restores spatial resolution through deconvolution and skip connections, and introduces a spatial attention module to focus on the conductive particle region; a combined optimization model using weighted cross entropy and Dice loss functions is used, and training stability is improved through dynamic learning rate scheduling. The method of the present invention significantly improves the accuracy and robustness of conductive particle detection, achieving efficient and automated industrial-grade detection, with significant economic and social benefits.

[0025] When implementing this embodiment, the specific steps are as follows: Step 1: pre-process the received input image Receive the input image, perform grayscale conversion, contrast enhancement and noise reduction processing in sequence to eliminate uneven illumination and noise interference, and generate a pre-processed image; in implementation, obtain high-resolution original images through industrial cameras, perform grayscale conversion, contrast enhancement and noise reduction processing in sequence to eliminate uneven illumination and noise interference, and generate a pre-processed image. The specific processing effects are as follows: Figure 2 As shown in .

[0026] Step 2: Normalization operation Normalize the image size, calculate the mean and standard deviation of the input image, and then perform normalization. Calculate the mean and standard deviation of the preprocessed image and normalize the pixel values to the range [0, 1]. At the same time, standardize the image size (for example, scale it to 256×256 pixels). Step 3: Create Segnet network training labels Collect and prepare training images and corresponding label data. Each label image has a corresponding pixel-level label, and the size of the label image should match the input image size. Use the image annotation tool to perform pixel-level annotation of the conductive particle region, generating a binary label image with the same size as the original image. Select the LabelImg annotation tool to create the label and save the label as a PNG image file. Ensure that the label image corresponds to the original image and has the same size.

[0027] Step 4: Network Construction A SegNet network model based on an encoder-decoder structure was constructed. The encoder is based on the first 13 convolutional layers of VGG-16 to extract multi-scale features of the image. The decoder gradually restores the spatial resolution through deconvolution and index-guided upsampling, and concatenates the feature maps of the corresponding encoder layer with the upsampled feature maps through skip connections. Convolution kernels of different sizes are introduced in the shallow layer of the encoder to extract multi-scale features in parallel; void convolution is introduced in the deep layer of the encoder to capture multi-scale features. In specific implementation, the shallow layer is the first and second layers of the encoder. Specifically, in the first and second layers of the encoder, 1×1, 3×3, and 5×5 convolution kernels are used in parallel, the channel dimension is reduced by 1×1 convolution, and then multi-scale features are extracted by 3×3 and 5×5 convolutions.

[0028] In this embodiment, the deep layers are the third and fourth layers. In the third and fourth layers of the encoder, dilated convolutions with dilation rates of 2 and 4 are used to expand the receptive field to capture multi-scale contextual information. Batch normalization and dropout layers are added after each dilated convolution layer. Specifically, this embodiment uses the pooling index saved by the encoder for nonlinear upsampling, uses a 5×5 convolution kernel in the deconvolution layer of the decoder, and combines feature fusion with jump connections to restore spatial information. Residual connections are introduced after each deconvolution block of the decoder to enhance gradient propagation. At the same time, a spatial attention module is introduced after the deconvolution block of the decoder to dynamically adjust the weights of the feature map and focus on the conductive particle area.

[0029] This embodiment generates two 1×1×C feature vectors by performing global average pooling and global maximum pooling on the input feature map. The two feature vectors are concatenated and then subjected to a 7×7 convolution to generate a spatial weight map. The weight map is multiplied element-by-element with the input feature map to focus on the conductive particle area, significantly improving the detection accuracy of the segmentation model for conductive particles.

[0030] Step 5: Model training The SegNet network model is trained using a combination of weighted cross entropy loss function and Dice loss function as the optimization target. A dynamic learning rate scheduling strategy and the Adam optimization algorithm are used to update the network weights and save the model weights with the best performance during training. A combination of weighted cross entropy loss function and Dice loss function is used, and the formula is: ; Among them, α is the balance factor; : Where N is the number of samples, C is the number of categories, Wc is the weight of category C, and y i,c is the true label, p i,c is the predicted probability; :where p i is the predicted probability, y i is the true label, and ε is the smoothing term.

[0031] Step 6: Conductive particle area segmentation The anisotropic conductive film image to be segmented is input into the trained SegNet network model, and a segmentation probability map is generated through forward propagation. The segmentation probability map is thresholded to generate a binary segmentation mask; through morphological filtering and connected domain analysis, the final conductive particle area segmentation result is output.

[0032] This embodiment provides a method for detecting conductive particles in anisotropic conductive films based on the SegNet architecture, and proposes an automated and highly robust solution to the problems of low efficiency and poor accuracy in conductive particle region segmentation in industrial production.

[0033] In practice, this embodiment uses an industrial camera to capture high-resolution raw images. These images are then grayscaled, contrast enhanced (e.g., using the CLAHE algorithm), and noise reduced (e.g., using non-local means filtering) to eliminate uneven illumination and noise. Subsequently, the images are normalized (e.g., scaled to 256×256 pixels), and the global mean and standard deviation are calculated. Pixel values are then normalized to the range [0, 1] to ensure consistency in the input data.

[0034] Image annotation tools (such as LabelImg) are used to annotate the conductive particle regions pixel-wise, generating binary labeled images with the same size as the original images. A SegNet network based on an encoder-decoder architecture is constructed. The encoder is based on the first 13 convolutional layers of VGG-16. In the shallow layers, multi-scale convolution kernels of 1×1, 3×3, and 5×5 are introduced to extract features in parallel. In the deep layers, dilated convolutions with dilation rates of 2 and 4 are used to expand the receptive field. The decoder restores spatial resolution through deconvolution and index-guided upsampling, and combines skip connections to fuse shallow details with deep semantic information. Furthermore, a residual connection and spatial attention module are introduced in the decoder. Global pooling and 7×7 convolution generate a spatial weight map, dynamically focusing on the conductive particle region and suppressing background noise.

[0035] In terms of model training and optimization, this embodiment uses a combination of a weighted cross-entropy loss function and a Dice loss function as the optimization objective. A balancing factor, α, adjusts the weights of the two loss types to address class imbalance. During training, dynamic learning rate scheduling (such as cosine annealing) and the Adam optimization algorithm are combined to improve convergence speed and stability, while preserving the model weights with optimal performance.

[0036] During conductive particle segmentation and post-processing, this example feeds the image to be detected into a trained SegNet model, generates a segmentation probability map, and then performs thresholding to produce a binary segmentation mask. Morphological filtering (such as opening operations) and connected domain analysis are used to remove noise and small areas, ultimately outputting accurate conductive particle segmentation results.

[0037] Therefore, this embodiment significantly improves the accuracy and robustness of conductive particle detection through multi-scale feature extraction, spatial attention mechanism and optimized loss function, while simplifying the complex texture analysis and image synthesis processes in traditional methods, and realizing efficient and automated industrial-grade detection.

[0038] The embodiments of the present invention described above do not limit its scope. The fundamental concept of this invention is to implement LCM DIC (differential interference contrast) image detection of anisotropic conductive film interconnects using a SegNet architecture based on a deep encoder and decoder. Any modifications, equivalent substitutions, and improvements within the spirit and principles of this invention are intended to be included within the scope of protection of the claims.

Claims

1. A method for detecting conductive particles in anisotropic conductive films based on a SegNet architecture, characterized by: The steps include: Step 1: pre-process the received input image Receive the input image, perform grayscale conversion, contrast enhancement and noise reduction in sequence to eliminate uneven illumination and noise interference, and generate a preprocessed image; Step 2: Normalization operation Normalize the image size, calculate the mean and standard deviation of the input image, and then perform normalization; Step 3: Create Segnet network training labels Collect and prepare images and corresponding label data for training. Each label image has a corresponding pixel-level label, and the size of the label image should match the input image size. Step 4: Network Construction Build a SegNet network model based on an encoder-decoder structure, where the encoder is based on the first 13 layers of the VGG-16 convolutional network to extract multi-scale features of the image; The decoder gradually restores the spatial resolution through deconvolution and index-guided upsampling, and concatenates the feature map of the corresponding layer of the encoder with the upsampled feature map through skip connections; Step 5: Model training The SegNet network model is trained using a combination of weighted cross entropy loss function and Dice loss function as the optimization target. A dynamic learning rate scheduling strategy and the Adam optimization algorithm are used to update the network weights and save the model weights with the best performance during training. Step 6: Conductive particle area segmentation The anisotropic conductive film image to be segmented is input into the trained SegNet network model, and a segmentation probability map is generated through forward propagation. The segmentation probability map is thresholded to generate a binary segmentation mask; through morphological filtering and connected domain analysis, the final conductive particle area segmentation result is output.

2. The method for detecting conductive particles in an anisotropic conductive film based on the SegNet architecture according to claim 1, characterized in that: Use the image annotation tool to annotate the conductive particle area at the pixel level and generate a labeled image with the same size as the original image.

3. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 1, characterized in that: Convolution kernels of different sizes are introduced in the shallow layer of the encoder to extract multi-scale features in parallel; and dilated convolution is introduced in the deep layer of the encoder to capture multi-scale features.

4. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 3, wherein: In the first and second layers of the encoder, 1×1, 3×3, and 5×5 convolution kernels are used in parallel. The channel dimension is reduced by 1×1 convolution, and then multi-scale features are extracted by 3×3 and 5×5 convolution.

5. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 3, characterized in that: The deep layers are the third and fourth layers. In the third and fourth layers of the encoder, dilated convolutions with dilation rates of 2 and 4 are used to expand the receptive field to capture multi-scale contextual information. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 5 is characterized in that a batch normalization layer and a Dropout layer are added after each void convolution layer.

6. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 1, characterized in that: The pooling index saved by the encoder is used for nonlinear upsampling, and a 5×5 convolution kernel is used in the deconvolution layer of the decoder, combined with feature fusion of jump connections to restore spatial information; a residual connection is introduced after each deconvolution block of the decoder to enhance gradient propagation.

7. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 7, wherein: A spatial attention module is introduced after the deconvolution block of the decoder to dynamically adjust the weights of the feature map and focus on the conductive particle area.

8. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 7, characterized in that: Perform global average pooling and global maximum pooling on the input feature map to generate two 1×1×C feature vectors. After splicing the two feature vectors, a 7×7 convolution is performed to generate a spatial weight map. The weight map is multiplied element-by-element with the input feature map to focus on the conductive particle area.

9. The method for detecting conductive particles in anisotropic conductive film based on the SegNet architecture according to claim 1, wherein: A combination of weighted cross entropy loss function and Dice loss function is used, and the formula is: ; Among them, α is the balance factor; : Where N is the number of samples, C is the number of categories, Wc is the weight of category C, and y i,c is the true label, p i,c is the predicted probability; :where p i is the predicted probability, y i is the true label, and ε is the smoothing term.

Images