IC board crack defect detection method and system

Abstract

The invention discloses an IC board crack defect detection method and system, and the method comprises the following steps: A1, constructing a U-Net + + deep learning network model which comprises an encoder module, a decoder module and a multi-hop connection structure, the encoder module is composed of five convolution blocks, each convolution block comprises two 3 * 3 convolution layers, a ReLU activation function and a maximum pooling layer with the step length of 2, and the decoder module is connected with the multi-hop connection structure; the decoder module is in jump connection with a feature map of a corresponding level of the encoder through an up-sampling layer, and the multi-jump connection structure establishes dense cross-layer connection between sub-networks of the decoder. According to the method, through the improved U-Net + + network structure, the edge details and semantic information of the crack are extracted at the same time, the detection rate of the micro crack is remarkably increased, a depth supervision mechanism and a weighted loss function are introduced, the learning weight of the crack area is optimized in a targeted mode, the problem of class imbalance is effectively relieved, and the false detection rate can be remarkably reduced under the complex background.

Description

Technical Field

[0001] The present invention relates to the field of optical detection technology, and in particular to an IC board crack defect detection method and system. Background Art

[0002] In the IC board (cell) production process, crack defect detection is the core link to ensure product quality.

[0003] Traditional inspection methods rely primarily on manual visual inspection or rule-based traditional image processing algorithms (such as edge detection and threshold segmentation). These methods have significant limitations: manual visual inspection is inefficient and susceptible to fatigue, making it difficult to adapt to large-scale production needs. However, traditional algorithms are sensitive to noise and lighting changes, rely on manually set threshold parameters, and cannot effectively handle tiny cracks or defects with variable shapes in complex backgrounds.

[0004] To address the above issues, we have introduced an IC board crack defect detection method and system. Summary of the Invention

[0005] The present invention discloses an IC board crack defect detection method and a system thereof, aiming to solve the technical problems in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A method and system for detecting crack defects in an IC board, comprising the following steps: A1. Build a U-Net++ deep learning network model, which includes an encoder module, a decoder module, and a multi-hop connection structure. The encoder module consists of five convolutional blocks, each of which includes two 3×3 convolutional layers, a ReLU activation function, and a maximum pooling layer with a stride of 2. The decoder module is jump-connected to the feature map of the corresponding level of the encoder through the upsampling layer. The multi-hop connection structure establishes dense cross-layer connections between each sub-network of the decoder. A2. Using a deep supervision mechanism, an auxiliary loss function is added to the output layer of each decoding subnetwork of the U-Net++ network. The auxiliary loss function is a weighted cross entropy loss with a weight coefficient α=0.8; A3, dynamically fuses the encoder's low-level edge features with the decoder's high-level semantic features through a multi-scale feature fusion module, with the fusion weights adaptively adjusted by a 1×1 convolution kernel; A4. The network model is trained using the IC board crack image after data enhancement. The data enhancement includes random rotation of ±15°, translation of ±10% of the image size, scaling by 0.8-1.2 times, adding Gaussian noise of σ=0.01, and adjusting the brightness by ±20%. A5. Input the image of the IC board to be inspected into the trained model, output a binary segmentation map of the crack area, and calculate the crack length, width, and location coordinates.

[0007] In a preferred embodiment, the multi-hop connection structure of the U-Net++ network satisfies the following conditions: The connection between the nth layer of the decoder and the mth layer of the encoder satisfies |nm|≤2, and n≥1, m≤5, where n and m are both integers.

[0008] In a preferred embodiment, the weighted cross entropy loss function is defined as:

[0009] in , , Obtained by calculating the class imbalance ratio.

[0010] In a preferred solution, the multi-scale feature fusion module is implemented by the following steps: B1. Perform 3×3 convolution on the feature map output by the encoder k-th layer to reduce the dimension to 64 channels; B2. Perform channel splicing on the feature map after dimensionality reduction and the feature map output by the decoder layer k+1; B3. Apply the spatial attention mechanism to the concatenated feature map, calculate the spatial weight matrix and weight the output.

[0011] In a preferred embodiment, the model is trained using the Adam optimizer with an initial learning rate of 1×10⁻ 4 , β1=0.9, β2=0.999, the batch size is 32, the training cycle is 100 times, and the training is terminated early when the F1 score of the validation set does not improve for 5 consecutive times.

[0012] In a preferred embodiment, the method further comprises a real-time detection optimization step: C1: Deploy the trained model on an embedded hardware platform; C2: Use the TensorRT engine to perform INT8 quantization on the model, compressing the model computation to 40% of its original size; C3: Real-time detection at 30 frames per second is achieved through multi-threaded pipeline processing.

[0013] An IC board crack defect detection system, comprising: Image acquisition module: includes an industrial camera and a ring LED light source. The industrial camera has a resolution of 2048×2048 and a frame rate of 60fps. The U-Net++ detection model according to any one of claims 1 to 6, running on a GPU-accelerated server; Result output module: Visually mark the crack coordinates, size and confidence level on the IC board image and generate a test report.

[0014] In a preferred solution, the image acquisition module is provided with a light source control system, and the light source control system includes: Light intensity adaptive adjustment unit: dynamically adjusts the PWM duty cycle of the LED light source according to the ambient brightness; Polarizing filter: Installed in front of the camera lens to filter out reflection interference from the IC board surface.

[0015] The IC board crack defect detection method and system provided by the present invention have the following advantages: The present invention uses an improved U-Net++ network structure to simultaneously extract edge details and semantic information of cracks, significantly improving the detection rate of tiny cracks. It introduces a deep supervision mechanism and a weighted loss function to specifically optimize the learning weights of the crack area, effectively alleviating the problem of category imbalance, and significantly reducing the false detection rate in complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the overall steps of the defect detection method for an IC board crack defect detection method proposed by the present invention.

[0017] Figure 2 This is a schematic diagram of the steps of the multi-scale feature fusion module of the IC board crack defect detection method proposed by the present invention.

[0018] Figure 3 This is a schematic diagram of the real-time detection optimization steps of an IC board crack defect detection method proposed by the present invention.

[0019] Figure 4 This is a structural diagram of an IC board crack defect detection system proposed by the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and marked in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0021] The present invention discloses an IC board crack defect detection method and system.

[0022] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, a method for detecting crack defects in an IC board includes the following steps: A1. Build a U-Net++ deep learning network model, which includes an encoder module, a decoder module, and a multi-hop connection structure. The encoder module consists of five convolutional blocks, each of which includes two 3×3 convolutional layers, a ReLU activation function, and a maximum pooling layer with a stride of 2. The decoder module is jump-connected to the feature map of the corresponding level of the encoder through the upsampling layer. The multi-hop connection structure establishes dense cross-layer connections between each sub-network of the decoder. A2. Using a deep supervision mechanism, an auxiliary loss function is added to the output layer of each decoding subnetwork of the U-Net++ network. The auxiliary loss function is a weighted cross entropy loss with a weight coefficient α=0.8; A3, dynamically fuses the encoder's low-level edge features with the decoder's high-level semantic features through a multi-scale feature fusion module, with the fusion weights adaptively adjusted by a 1×1 convolution kernel; A4. The network model is trained using the IC board crack image after data enhancement. The data enhancement includes random rotation of ±15°, translation of ±10% of the image size, scaling by 0.8-1.2 times, adding Gaussian noise of σ=0.01, and adjusting the brightness by ±20%. A5. Input the image of the IC board to be tested into the trained model, output a binary segmentation map of the crack area, and calculate the crack length, width, and location coordinates; The U-Net++ network model constructed in this embodiment gradually extracts the abstract features of the IC board image through the encoder module. Each layer of the encoder reduces the size of the feature map and increases the number of channels through convolution and pooling operations, capturing multi-level information from edges to semantics. The decoder module restores the spatial resolution through upsampling and performs jump connections with the feature maps of the corresponding level of the encoder to make up for the loss of details in the downsampling process. The multi-hop connection structure establishes a cross-layer path between the sub-networks of the decoder, so that low-level features (such as crack edges) and high-level features (such as the overall morphology of cracks) can fully interact, thereby enhancing the model's sensitivity to tiny cracks. Data enhancement technology is used in the training phase to simulate image interference (such as lighting changes, mechanical vibration noise) in actual production environments to improve the robustness of the model. The final model outputs a binary segmentation map, and the crack geometric parameters are calculated through connected domain analysis.

[0023] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, in a preferred embodiment, the multi-hop connection structure of the U-Net++ network meets the following conditions: The connection between the nth layer of the decoder and the mth layer of the encoder satisfies |nm|≤2, and n≥1, m≤5, where n and m are integers; The multi-hop connection design in this embodiment restricts the decoder layer n to only connect with encoder features from adjacent layers (within n±2), avoiding feature semantic gaps caused by long-range cross-layer connections. For example, the decoder layer 3 can fuse features from encoder layers 1, 2, and 3, ensuring that the fused features contain both local details (low-level layers) and global context (mid-level layers), thereby accurately locating discontinuous or curved crack paths while suppressing false detections of background textures.

[0024] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, in a preferred embodiment, the weighted cross entropy loss function is defined as:

[0025] in, , , Obtained by calculating the class imbalance ratio; This example uses a weighted cross-entropy loss function to address the extremely low proportion of crack pixels in IC board images. By increasing the loss weight for crack regions (α > 0.5), the model is forced to focus on learning minority class samples. The weight coefficient α is dynamically calculated based on the ratio of crack pixels to background pixels in the training data, mitigating model bias caused by class imbalance. For example, if crack pixels account for 5% of the dataset, α can be set to 0.8, causing the model to prioritize prediction errors in crack regions during backpropagation.

[0026] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, in a preferred embodiment, the multi-scale feature fusion module is implemented by the following steps: B1. Perform 3×3 convolution on the feature map output by the encoder k-th layer to reduce the dimension to 64 channels; B2. Perform channel splicing on the feature map after dimensionality reduction and the feature map output by the decoder layer k+1; B3. Apply the spatial attention mechanism to the concatenated feature map, calculate the spatial weight matrix and weight the output; In this embodiment, the multi-scale feature fusion module first performs convolution on the feature map of the encoder's kth layer to reduce the number of channels and lower the computational effort. The reduced features are then concatenated with the upsampled features output by the decoder's k+1th layer, fusing feature information at different resolutions. Finally, a spatial attention mechanism is used to calculate a weight for each pixel, focusing the model on areas where cracks are likely to be present (e.g., areas with high gradient changes) while suppressing interference from irrelevant background. A spatial weight matrix is ​​generated through convolutional layers and a sigmoid function, enabling adaptive weighting of the feature map.

[0027] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, in a preferred embodiment, the training process of the model uses the Adam optimizer with an initial learning rate of 1×10⁻ 4 , β1=0.9, β2=0.999, batch size is 32, training epochs are 100, and training is terminated early when the F1 score on the validation set does not improve for 5 consecutive times; In this embodiment, model training utilizes an adaptive optimization algorithm. The initial learning rate is preset based on the network depth and parameter magnitude, and the learning rate is dynamically adjusted during training based on changes in validation set accuracy. If validation set performance does not improve over multiple training cycles, an early stopping mechanism is triggered to prevent overfitting. The optimization algorithm combines first-order and second-order momentum estimation to accelerate model convergence and stabilize the training process. The batch size is set based on the GPU memory capacity to ensure efficient use of hardware resources.

[0028] Reference Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, in a preferred embodiment, the method further comprises a real-time detection optimization step: C1: Deploy the trained model on an embedded hardware platform; C2: Use the TensorRT engine to perform INT8 quantization on the model, compressing the model computation to 40% of its original size; C3: Real-time detection at 30 frames per second through multi-threaded pipeline processing; To meet the real-time detection requirements of the production line, the trained model is compressed and optimized before being deployed on an embedded hardware platform. By reducing the precision of the model weights (e.g., INT8 quantization), memory usage and computational latency are reduced. A multi-threaded pipeline is employed to parallelize image acquisition, inference calculations, and output, ensuring system throughput meets the real-time requirements of the production line.

[0029] An IC board crack defect detection system, comprising: Image acquisition module: includes an industrial camera and a ring LED light source. The industrial camera has a resolution of 2048×2048 and a frame rate of 60fps. The U-Net++ detection model according to any one of claims 1 to 6, running on a GPU-accelerated server; Result output module: Visually mark the crack coordinates, size and confidence level on the IC board image and generate a test report; The inspection system consists of a high-resolution industrial camera, a ring light source, a GPU server, and a display terminal. The industrial camera captures images of the IC board surface at a high frame rate, while the ring light source provides uniform illumination to reduce shadow interference. The GPU server runs a trained U-Net++ model to perform crack segmentation and quantitative analysis on the input image. The results are visualized and crack locations are annotated, generating a structured inspection report.

[0030] The image acquisition module is provided with a light source control system, which includes: Light intensity adaptive adjustment unit: dynamically adjusts the PWM duty cycle of the LED light source according to the ambient brightness; Polarizing filter: Installed in front of the camera lens to filter out reflection interference from the IC board surface; The light control system integrates an adaptive light intensity module, dynamically adjusting LED brightness based on ambient light intensity to ensure consistent image quality under varying lighting conditions. For example, when ambient light is strong, the LED's PWM duty cycle is reduced to prevent overexposure. A polarizing filter, combined with the camera lens, removes metallic reflections from the IC board surface, enhancing contrast in crack areas and reducing false detections caused by reflections.

[0031] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. The replacement may be a replacement of a portion of a structure, device, or method step, or it may be a complete technical solution. Any equivalent replacement or modification based on the technical solution and inventive concept of the present invention shall be covered by the scope of protection of the present invention.

Claims

1. A method for detecting crack defects in an IC board, characterized in that: The following steps are involved: A1. Build a U-Net++ deep learning network model, which includes an encoder module, a decoder module, and a multi-hop connection structure. The encoder module consists of five convolutional blocks, each of which includes two 3×3 convolutional layers, a ReLU activation function, and a maximum pooling layer with a stride of 2. The decoder module is jump-connected to the feature map of the corresponding level of the encoder through the upsampling layer. The multi-hop connection structure establishes dense cross-layer connections between each sub-network of the decoder. A2. Using a deep supervision mechanism, an auxiliary loss function is added to the output layer of each decoding subnetwork of the U-Net++ network. The auxiliary loss function is a weighted cross entropy loss with a weight coefficient α=0.8; A3, dynamically fuses the encoder's low-level edge features with the decoder's high-level semantic features through a multi-scale feature fusion module, with the fusion weights adaptively adjusted by a 1×1 convolution kernel; A4. Train the network model using the IC board crack image after data enhancement, wherein the data enhancement includes random rotation of ±15°, translation of ±10% of the image size, scaling by 0.8-1.2 times, adding Gaussian noise of σ=0.01, and adjusting the brightness by ±20%; A5. Input the image of the IC board to be inspected into the trained model, output a binary segmentation map of the crack area, and calculate the crack length, width, and location coordinates.

2. The IC board crack defect detection method according to claim 1, characterized in that: The multi-hop connection structure of the U-Net++ network meets the following conditions: The connection between the nth layer of the decoder and the mth layer of the encoder satisfies |nm|≤2, and n≥1, m≤5, where n and m are both integers.

3. The IC board crack defect detection method according to claim 1, characterized in that: The weighted cross entropy loss function is defined as: Among them, y i is the true label of the pixel, p i is the predicted probability, and α = 0.8 is obtained by calculating the class imbalance ratio.

4. The IC board crack defect detection method according to claim 1, characterized in that: The multi-scale feature fusion module is implemented by the following steps: B1. Perform 3×3 convolution on the feature map output by the encoder k-th layer to reduce the dimension to 64 channels; B2. Perform channel splicing on the feature map after dimensionality reduction and the feature map output by the decoder layer k+1; B3. Apply the spatial attention mechanism to the concatenated feature map, calculate the spatial weight matrix and weight the output.

5. The IC board crack defect detection method according to claim 1, characterized in that: The model is trained using the Adam optimizer with an initial learning rate of 1×10 -4 , β1 = 0.9, β2 = 0.999, the batch size is 32, the training cycle is 100 times, and the training is terminated early when the F1 score of the validation set does not improve for 5 consecutive times.

6. The IC board crack defect detection method according to claim 1, characterized in that: The method further comprises a real-time detection optimization step: C1: Deploy the trained model on an embedded hardware platform; C2: Uses the TensorRT engine to perform INT8 quantization on the model, compressing the model computation to 40% of its original size; C3: Real-time detection at 30 frames per second is achieved through multi-threaded pipeline processing.

7. An IC board crack defect detection system, characterized in that: include: Image acquisition module: includes an industrial camera and a ring LED light source. The industrial camera has a resolution of 2048×2048 and a frame rate of 60fps. The U-Net++ detection model according to any one of claims 1 to 6, running on a GPU-accelerated server; Result output module: Visually mark the crack coordinates, size and confidence level on the IC board image and generate a test report.

8. The IC board crack defect detection system according to claim 7, characterized in that: The image acquisition module is provided with a light source control system, and the light source control system includes: Light intensity adaptive adjustment unit: dynamically adjusts the PWM duty cycle of the LED light source according to the ambient brightness; Polarizing filter: Installed in front of the camera lens to filter out reflection interference from the IC board surface.

Images