Integrated circuit defect image recognition method based on transfer learning

Abstract

The invention discloses an integrated circuit defect image recognition method based on transfer learning, which comprises the following steps: firstly, analyzing the feature distribution difference of a source domain defect image and a target domain defect image to obtain a self-adaptive weight adjustment parameter; then optimizing the pre-training model by utilizing transfer learning to obtain improved model parameters; a layered defect image recognition framework is constructed based on the parameters, and defect severity is evaluated; when the severity exceeds a threshold value, positioning key connection points and pin positions and extracting defect feature vectors of key areas; multiple defect types such as scratches and short circuits are processed through a multi-branch neural network, and multi-variety classification is achieved; extracting a special defect image for the data communication interface, and obtaining an accurate identification index; finally, visible light and thermal imaging temperature distribution characteristics are fused, a correlation model of temperature difference and defect types is established, and defect detection is completed. According to the invention, the accuracy and efficiency of integrated circuit defect identification are improved.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, and in particular relates to a method for image recognition of integrated circuit defects based on transfer learning. Background Technology

[0002] As the core of modern electronic devices, integrated circuits (ICs) directly determine product quality and reliability through defect detection during their manufacturing process. Therefore, researching efficient and accurate defect image recognition technology is crucial for enhancing industrial competitiveness. Traditional defect detection methods largely rely on manual experience or image processing based on fixed rules, making it difficult to adapt to the increasing complexity and diversity of ICs. Especially when facing dynamic changes in manufacturing processes, equipment conditions, and defect types, existing methods have significant shortcomings in feature extraction and adaptive adjustment, hindering efficient automated detection. The core challenge lies in how to address the differences in defect characteristics across different complexity and functional regions within ICs, and how to achieve accurate identification through multimodal information fusion.

[0003] The integration density and functional module complexity of integrated circuits vary significantly. For example, the defect feature distributions of high-density memory chips and low-density logic chips are drastically different. Traditional methods struggle to adaptively adjust identification strategies for different complexity levels, leading to decreased detection accuracy. Furthermore, defect detection requires combining multiple information sources, such as visible light images and thermal imaging data. However, current technologies, when fusing multimodal features, lack in-depth analysis of the correlation between temperature distribution and defect type, making it difficult to effectively locate subtle defects in critical functional areas. For instance, in the inspection of data communication interfaces, minute breaks in serial interfaces may be overlooked due to insignificant temperature differences, affecting the stability of data transmission. Therefore, designing adaptive defect identification strategies for integrated circuits of varying complexity and fusing multimodal image information to achieve accurate defect localization in critical areas has become a key issue in improving the efficiency and reliability of integrated circuit defect detection. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for integrated circuit defect image recognition based on transfer learning, comprising: By analyzing the feature distribution differences between the source and target domain defect images, adaptive weight adjustment parameters are obtained. Based on the adaptive weights, the parameters are adjusted, and the pre-trained model is processed using transfer learning to obtain the optimized model parameters. Using the optimized model parameters, a hierarchical defect image recognition framework is constructed for different integration densities and complexities of integrated circuits; The severity level of the defect is determined based on the hierarchical defect image recognition framework. If the severity level of the defect is higher than a preset threshold, the key connection points and pin positions are located to obtain the defect feature vector of the key area. Based on the defect feature vectors, a multi-branch neural network structure is used to process various defect types and determine the multi-variety classification results. Based on the multi-variety classification results, specific defect images of the data communication interface are extracted to obtain accurate identification indicators for communication interface defects. Based on the precise identification indicators, the temperature distribution characteristics of visible light images and thermal imaging images are fused to establish a correlation model between temperature difference and defect type; Based on the aforementioned correlation model, integrated circuit defect image detection is performed to obtain detection results.

[0005] Preferably, the process of obtaining adaptive weight adjustment parameters by analyzing the feature distribution differences between the source and target domain defect images includes: Defect image features are extracted from the source and target domains to obtain pixel-level feature distributions; If the pixel-level feature distribution difference exceeds a preset threshold, the features are standardized to obtain a normalized feature distribution. A multi-level hierarchical model is constructed based on the normalized feature distribution, and multi-level feature vectors are extracted; Based on the feature difference vectors between the source and target domains, optimize the feature mapping and calculate the transfer learning success rate evaluation index. Multi-level features are integrated to generate a comprehensive evaluation score, and weights are extracted from the comprehensive evaluation score to adjust the initial parameters. If the variance of the initial weight adjustment parameter is less than a preset threshold, the parameter is adjusted by a regularization method to obtain the adaptive weight adjustment parameter.

[0006] Preferably, the process of adjusting the parameters according to the adaptive weights and using transfer learning to perform feature representation processing on the pre-trained model to obtain the optimized model parameters includes: Load the pre-trained model and training data to obtain the initial model parameters; Based on the feature extraction results of the training data, calculate the adaptive weights and determine the weight allocation scheme. If the weight allocation scheme meets the preset threshold, the gradient descent algorithm is used to update the model parameters to obtain the optimized model parameters. If the feature representation does not converge, the learning rate is adjusted and the adaptive weights are recalculated. The model parameters are iteratively updated until the feature representation converges, and the final optimized model parameters are obtained.

[0007] Preferably, the process of constructing a hierarchical defect image recognition framework for different integration densities and complexities of integrated circuits using the optimized model parameters includes: The input integrated circuit defect image is preprocessed to obtain the first image; Extract the geometric and texture features of the defects from the first image to obtain a feature set; If the geometric features of the feature set meet a preset threshold, a convolutional neural network is used to classify the feature set to obtain a preliminary defect category. Based on the preliminary defect categories and integration density data, the classification weights of the hierarchical framework are adjusted to obtain an optimized classification model; The severity level of the defect is determined by performing a secondary analysis on the feature set using the optimized classification model. If the severity level is higher than a preset threshold, the first image is magnified locally to obtain a second image, and a support vector machine is used to verify the local features of the second image to determine the final severity level.

[0008] Preferably, the process of locating key connection points and pin positions to obtain defect feature vectors for key areas includes: If the severity level of the defect is higher than the preset threshold, the input image is processed by the joint detection algorithm to locate the key connection points and pin positions and obtain the detection results. Based on the detection results, key regions are divided, and defect feature vectors of the key regions are extracted to obtain a feature dataset; If the defect feature vectors in the feature dataset meet the preset conditions, then the principal component analysis algorithm is used to reduce the dimensionality of the feature dataset to obtain a dimensionality-reduced feature set. Based on the reduced feature set, the defect features are classified using the support vector machine algorithm to determine the defect category; If the defect category belongs to the high-risk category, then image enhancement is performed on the key area to obtain the enhanced image, and the refined feature vector is extracted to obtain the refined feature set; Based on the refined feature set, a clustering analysis algorithm is used to analyze the defect distribution and determine the defect distribution pattern.

[0009] Preferably, the process of determining the multi-variety classification result by using a multi-branch neural network structure to process multiple defect types based on the defect feature vector includes: The original image data of the key region is acquired and preprocessed to obtain the first image data. Defect feature vectors are extracted from the first image data, and multi-scale feature extraction is performed through a convolutional neural network to obtain a set of feature vectors. If the feature vector set includes scratch or short-circuit features, a multi-branch neural network structure is used to process each branch independently to determine the preliminary defect type; The output features of a multi-branch neural network are fused and classified through a fully connected layer to obtain a set of classification results. Extract the defect type with the highest confidence from the classification result set, filter it using a preset threshold, and determine the final defect type; Based on the final defect type and key area location information, a defect distribution map is generated to obtain defect distribution data; statistical analysis is performed on the defect distribution data to determine the multi-variety classification results.

[0010] Preferably, the process of extracting specific defect images of the data communication interface portion based on the multi-variety classification results to obtain accurate identification indicators for communication interface defects includes: The input image is classified using a multi-class classification model to obtain an initial classification result that includes communication interface defects; A convolutional neural network is used to extract specific defect images from the initial classification results to obtain feature representations of the defect regions. If the confidence level of the feature representation of the defect region is higher than a preset threshold, then the specific defect image is segmented a second time to determine the specific location of the defect. Image processing techniques are used to enhance the identified defect locations, resulting in enhanced defect images. Based on the enhanced defect images, a support vector machine model is used to classify the defect types and obtain the probability distribution of the defect categories; Based on the probability distribution of defect categories, calculate the precise identification index and determine the precise identification index for communication interface defects.

[0011] Preferably, the process of establishing a correlation model between temperature difference and defect type by fusing the temperature distribution features of visible light images and thermal imaging images based on the accurate identification indicators includes: Acquire visible light images and thermal imaging images, and align the two images using image registration technology to obtain registered image data; Temperature distribution characteristics are extracted from the registered image data, and dimensionality reduction is performed using principal component analysis to obtain the dimensionality-reduced temperature distribution vector. Based on the reduced-dimensional temperature distribution vector, the temperature difference value of each region is calculated to obtain the temperature difference feature set; If the temperature difference value in the temperature difference feature set exceeds a preset threshold, the defect type is classified by the support vector machine algorithm to obtain a preliminary defect classification result. Based on the preliminary defect classification results, a convolutional neural network is used to extract deep features from the registered image data to obtain a deep feature set. Based on the depth feature set and temperature difference feature set, a correlation model of the random forest algorithm is constructed to establish the mapping relationship between temperature difference and defect type.

[0012] Preferably, if the temperature difference value in the temperature difference feature set exceeds a preset threshold, the process of classifying the defect type using a support vector machine algorithm to obtain a preliminary defect classification result includes: Temperature distribution feature data, including pixel temperature values ​​and spatial coordinates, are obtained from visible light images and thermal imaging images. Infrared imaging technology is used to extract temperature gradient features from thermal imaging images to obtain a temperature gradient distribution map; The temperature gradient distribution map and the texture features of the visible light image are fused to generate a fused feature matrix; Based on the fused feature matrix, feature vectors of the defect region are extracted using a convolutional neural network to determine the boundary of the defect region. Based on the boundary of the defect area, the deviation between the temperature difference value and the preset threshold is calculated to obtain the initial classification result of the defect type; If the initial defect type classification result matches the historical defect database, the final defect type judgment result will be output.

[0013] Compared with the prior art, the present invention has the following advantages and technical effects: This invention discloses an automated detection technology for integrated circuit defect image recognition, solving the problem of accurate identification and severity assessment of various defect types in high-integration-density circuits. By analyzing the differences in feature distribution between source and target domain defect images, a multi-level transfer learning evaluation system is constructed to obtain adaptive weight adjustment parameters, optimize the feature representation of the pre-trained model, and generate model parameters suitable for different integration densities and complexities. This invention also designs a hierarchical defect image recognition framework to determine the severity level of defects and locate key connection points and pins in high-severity defects through joint point detection, extracting defect feature vectors. A multi-branch neural network is used to process various defect types such as scratches and short circuits, combined with data communication interface-specific defect image extraction, and the fusion of visible light and thermal imaging features to establish a correlation model between temperature difference and defect type. This is integrated into a hardware and software co-optimization scheme for an image recognition tester, achieving automated defect detection. This invention significantly improves the accuracy and efficiency of integrated circuit defect detection, and is particularly suitable for real-time quality control of highly complex circuits. Attached Figure Description

[0014] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0017] like Figure 1 As shown, this embodiment provides a method for integrated circuit defect image recognition based on transfer learning, including: By analyzing the feature distribution differences between the source and target domain defect images, adaptive weight adjustment parameters are obtained. The parameters are adjusted according to adaptive weights, and the pre-trained model is processed for feature representation using transfer learning to obtain the optimized model parameters. Using optimized model parameters, a hierarchical defect image recognition framework is constructed for different integration densities and complexities of integrated circuits. The severity level of the defect is determined based on the hierarchical defect image recognition framework. If the severity level of the defect is higher than the preset threshold, the key connection points and pin positions are located to obtain the defect feature vector of the key area. Based on the defect feature vectors, a multi-branch neural network structure is used to process various defect types and determine the multi-variety classification results. Based on the multi-variety classification results, specific defect images of the data communication interface are extracted to obtain accurate identification indicators for communication interface defects. Based on accurate identification indicators, temperature distribution characteristics of visible light images and thermal imaging images are integrated to establish a correlation model between temperature difference and defect type; Integrated circuit defect image detection is performed based on the correlation model to obtain detection results.

[0018] Furthermore, the process of obtaining adaptive weight adjustment parameters by analyzing the feature distribution differences between the source and target domain defect images includes: Defect image features are extracted from the source and target domains to obtain pixel-level feature distributions; If the difference in pixel-level feature distribution exceeds a preset threshold, the features are standardized to obtain a normalized feature distribution. A multi-level hierarchical model is constructed based on normalized feature distribution, and multi-level feature vectors are extracted; Based on the feature difference vectors between the source and target domains, optimize the feature mapping and calculate the transfer learning success rate evaluation index. Multi-level features are integrated to generate a comprehensive evaluation score, and weights are extracted from the comprehensive evaluation score to adjust the initial parameters. If the variance of the initial weight adjustment parameters is less than a preset threshold, the parameters are adjusted using a regularization method to obtain adaptive weight adjustment parameters.

[0019] In one possible implementation, for pixel-level feature distribution extraction of defect images, this embodiment extracts features from defect images in the source and target domains using a convolutional neural network.

[0020] For example, the source domain might be images of metal parts on a production line, and the target domain might be images of the same type of parts from different batches. Convolutional layers extract low-level features such as edges and textures, while pooling layers obtain high-level semantic features, forming a feature distribution. Assuming the source domain features have a mean of 0.8 and a variance of 0.2, while the target domain features have a mean of 0.6 and a variance of 0.3, and the difference exceeds a preset threshold of 0.1, standardization is required. After standardization, the feature distribution is normalized to a mean of 0 and a variance of 1, which helps to unify the scale and reduce the impact of cross-domain differences on model performance.

[0021] For example, when constructing a hierarchical model of a multi-level system, feature extraction modules at three levels—shallow, intermediate, and deep—can be designed. The shallow layer focuses on local defects such as scratches, the intermediate layer on regional defects such as dents, and the deep layer extracts global semantics such as overall deformation of the part. The hierarchical model improves the detection accuracy of complex defects through multi-scale feature fusion. When using convolutional neural networks for deep feature extraction, a residual network structure can be used to generate multi-level feature vectors. Assuming the source domain vector has a magnitude of 1.5 and the target domain vector has a magnitude of 1.2, the difference vector magnitude of 0.3 exceeds the threshold of 0.2, requiring optimization of the feature mapping. The gradient descent algorithm adjusts the mapping matrix; after 10 iterations, the magnitude decreases to 0.1, and the optimized feature mapping improves transfer learning performance.

[0022] Specifically, this embodiment uses classification accuracy and F1 score to evaluate the success rate of transfer learning. For example, the optimized feature map achieves a classification accuracy of 95% and an F1 score of 0.92 on the validation set, a significant improvement compared to the unoptimized 90% and 0.85, indicating that transfer learning effectively adapts to the target domain. When fusing multi-level features, a weighted average method can be used, assigning weights of 0.2 to the shallow layer, 0.3 to the middle layer, and 0.5 to the deep layer to generate a comprehensive evaluation score. Assuming a score of 0.88, the initial parameter variance is 0.05, which is less than the threshold of 0.1, requiring regularization adjustment. After regularization, the parameter variance increases to 0.08, generating stable weight parameters and improving the model's robustness.

[0023] In one embodiment, the adaptive weight adjustment parameters are fine-tuned through an iterative update mechanism.

[0024] For example, the weights are adjusted according to the loss function in each iteration, and the parameters converge after 5 iterations, generating the final adaptive parameters. Test set validation shows that the defect detection accuracy improved from 93% to 96%, indicating the effectiveness of the parameters. This method, through steps such as feature standardization, hierarchical modeling, and optimized mapping, forms a closed-loop optimization system, significantly improving the cross-domain adaptability and accuracy of defect detection, reducing false detection rates in production, and ensuring product quality.

[0025] Furthermore, the process of adjusting parameters based on adaptive weights and using transfer learning to perform feature representation processing on the pre-trained model to obtain optimized model parameters includes: Load the pre-trained model and training data to obtain the initial model parameters; Based on the feature extraction results of the training data, calculate the adaptive weights and determine the weight allocation scheme. If the weight allocation scheme meets the preset threshold, the gradient descent algorithm is used to update the model parameters to obtain the optimized model parameters. If the feature representation does not converge, the learning rate is adjusted and the adaptive weights are recalculated. The model parameters are iteratively updated until the feature representation converges, and the final optimized model parameters are obtained.

[0026] For example, in this embodiment, when acquiring the pre-trained model and the training data of the training subject, a convolutional neural network model pre-trained on a large image dataset, such as ResNet or VGG, is selected.

[0027] Taking ResNet-50 as a pre-trained model as an example, it is trained on the ImageNet dataset and can recognize a variety of image features. After loading the pre-trained model, the model parameters are initialized to form the initial model.

[0028] For example, in wafer defect detection scenarios, the initial parameters of ResNet-50 can capture basic features such as wafer surface texture and color, laying the foundation for subsequent transfer learning.

[0029] Specifically, feature extraction from the training data involves inputting wafer defect images into the model to obtain low-level edge features and high-level semantic features. Assuming the training data contains 1000 wafer images, each with different types of scratches or blemishes, the feature extraction results might show that the pixel values ​​of scratches are concentrated in high-contrast areas. When calculating adaptive weights, the differences in the distribution of scratch and blemish features are analyzed to determine the weight allocation scheme.

[0030] For example, scratch features may be given higher weights due to their high contrast, thus enhancing the model's sensitivity to critical defects. If the weighted sum of the weight allocation scheme meets a preset threshold, such as 0.8, it indicates that the allocation is reasonable, and gradient descent optimization can be performed directly.

[0031] In one embodiment, the gradient descent algorithm updates model parameters by adjusting the learning rate.

[0032] For example, the initial learning rate is 0.001, which is gradually reduced to 0.0001 to avoid parameter oscillations, given the complex features of the wafer image. The updated model parameters generate optimized feature representations, such as clearer scratch edges or blemish areas. If the feature representations do not converge, for example, if scratch edges remain blurry, the learning rate is adjusted to 0.0005, and the adaptive weights are recalculated.

[0033] For example, increasing the weight of high-contrast features to 0.6 and decreasing the weight of background noise to 0.2 creates a new allocation scheme. After repeating gradient descent, the final model parameters can generate stable feature representations, such as accurately distinguishing the boundaries between scratches and blemishes.

[0034] For example, the feature representation output by the final model can be used for defect classification tasks. Suppose the model output classifies wafer images into "acceptable" and "unacceptable," its feature representation can clearly separate different defect types; for instance, images with scratches wider than 0.1 mm are marked as unacceptable. This clear feature representation improves classification accuracy and contributes to quality control on automated production lines.

[0035] Preferably, the final model parameters are tested on a validation set, for example, achieving a classification accuracy of 95% on 500 new images, to prove the effectiveness of the parameters. This method significantly improves the reliability of defect detection and reduces the cost of manual inspection.

[0036] Understandably, the repeated adjustment of adaptive weights ensures that the model adapts to specific defect features.

[0037] For example, to address minute scratches on the wafer surface, the model can dynamically increase the edge feature weight to 0.7 and decrease the color feature weight to 0.1, thereby focusing on critical defects. This flexibility allows the model to remain robust across different production batches.

[0038] For example, in high-noise environments, the adjusted weighting scheme can filter background interference and highlight defect features, thereby improving detection stability.

[0039] In one possible implementation, the feature representation output by the final model can also be used for defect localization.

[0040] For example, the model generates heat maps that pinpoint the exact location of scratches, with an error range controlled within 0.05 millimeters. This precise positioning supports the accurate implementation of subsequent repair processes, reducing resource waste.

[0041] It should be noted that the optimized feature representation can also be integrated with other detection systems, such as combining it with spectral analysis, to further verify the defect type and improve the overall detection efficiency.

[0042] Furthermore, the process of constructing a hierarchical defect image recognition framework for different integration densities and complexities of integrated circuits using optimized model parameters includes: The input integrated circuit defect image is preprocessed to obtain the first image; Extract the geometric and texture features of the defects from the first image to obtain a feature set; If the geometric features of the feature set meet the preset threshold, then a convolutional neural network is used to classify the feature set to obtain a preliminary defect category. Based on the preliminary defect categories and integration density data, the classification weights of the hierarchical framework are adjusted to obtain the optimized classification model; The severity level of the defect is determined by performing a secondary analysis on the feature set using the optimized classification model. If the severity level is higher than a preset threshold, the first image is magnified locally to obtain a second image, and a support vector machine is used to verify the local features of the second image to determine the final severity level.

[0043] For example, this embodiment combines image processing and feature extraction techniques with machine learning methods to efficiently identify and classify defects, thereby improving production quality. The following analysis focuses on key technical topics, using specific examples to maintain logical rigor and highlight the application of the technology.

[0044] For example, preprocessing an input image of an integrated circuit defect aims to enhance image quality for subsequent analysis. One possible implementation involves using grayscale conversion and Gaussian blurring to remove noise. Assuming the input image is a 1024x1024 pixel image of a wafer surface, grayscale conversion converts the color image to a single-channel grayscale image, followed by 3x3 Gaussian kernel smoothing to obtain the first image. This preprocessing effectively reduces interference from uneven illumination or background noise, providing a clear base image for subsequent feature extraction. For feature extraction algorithms, extracting the geometric and textural features of the defects is a core step.

[0045] For example, this embodiment uses an edge detection algorithm such as the Canny operator to extract geometric features and calculate the boundary length and area of ​​the defect.

[0046] For example, if a defect region is detected with a boundary length of 50 pixels and an area of ​​200 pixels, it indicates that the defect is a thin, elongated crack. Texture features can be calculated using the gray-level co-occurrence matrix to extract contrast and entropy values, quantifying the roughness of the defect region. These features form a feature set, providing data support for subsequent classification. When the geometric features of the feature set meet a preset threshold, such as a boundary length greater than 40 pixels and an area less than 300 pixels, a convolutional neural network can be triggered for classification.

[0047] In one embodiment, a pre-trained ResNet-18 network is used. After inputting a feature set, it outputs a preliminary defect category, such as crack, particle, or scratch. Assuming a defect is classified as a crack with 90% accuracy, it indicates that the model effectively captures the geometric characteristics of the defect. Based on the preliminary defect category, the classification weights of the hierarchical framework are adjusted using ensemble density data.

[0048] For example, if the wafer's integration density is 1000 transistors per square millimeter, defects in high-density areas have a greater impact on product quality. Therefore, the weight of the crack category can be increased to 0.7, while the weight of the particle category can be set to 0.3. This adjustment makes the classification model focus more on high-risk defects, improving the targeting of detection. A secondary analysis using the optimized classification model yields the severity level of the defects.

[0049] For example, a crack defect is rated as high severity because its length exceeds 50 pixels and it is located in a high-density area. If the severity level is higher than a preset threshold, such as level 3, the first image can be locally magnified. In one possible implementation, the central region of the defect is selected and magnified by a factor of 2 to generate a second image of 256x256 pixels, highlighting the crack details. Finally, a support vector machine is used to verify the local features of the second image to determine the final severity level.

[0050] For example, texture features of the second image are extracted, such as an entropy value of 2.5 and a contrast value of 0.8, and the defect is confirmed as a high-severity crack by support vector machine classification. This verification ensures the reliability of the classification results and reduces misclassification.

[0051] It should be noted that the above method employs a multi-stage process, from image preprocessing to feature extraction, classification, and verification, progressing step by step with rigorous logic to ensure the accuracy and robustness of defect detection. The implementation of each step is closely aligned with the business needs of integrated circuit defect detection, optimizing quality control in the production process.

[0052] Furthermore, the process of locating key connection points and pin positions to obtain defect feature vectors for critical regions includes: If the severity level of the defect is higher than the preset threshold, the input image is processed by the joint detection algorithm to locate the key connection points and pin positions and obtain the detection results. Based on the detection results, key regions are divided, and defect feature vectors of the key regions are extracted to obtain a feature dataset; If the defect feature vectors in the feature dataset meet the preset conditions, then the principal component analysis algorithm is used to reduce the dimensionality of the feature dataset to obtain a dimensionality-reduced feature set. Based on the dimensionality reduction feature set, the support vector machine algorithm is used to classify the defect features and determine the defect category; If the defect category belongs to the high-risk category, image enhancement is performed on the key area to obtain the enhanced image, and the refined feature vector is extracted to obtain the refined feature set; Based on the refined feature set, a clustering analysis algorithm is used to analyze the defect distribution and determine the defect distribution pattern.

[0053] In one possible implementation, this embodiment uses a key point detection algorithm to process integrated circuit defect images, aiming to accurately locate key connection points and pin positions.

[0054] For example, in defect detection of a high-density integrated circuit, the image resolution is 1024×768 pixels, containing multiple solder joints and pins. The keypoint detection algorithm first identifies the solder joint contours using edge detection technology, and then, combined with a pre-trained deep learning model, locates the center coordinates of each solder joint, generating detection results containing coordinate data. This method effectively identifies key connection points, facilitating subsequent region segmentation.

[0055] For example, when dividing the inspection results into key regions, a 50×50 pixel area around each solder joint is defined as a key region based on the solder joint coordinates. Assuming a circuit board contains 10 key regions, defect feature vectors are extracted from each region, including the defect's shape, area, and texture information, forming a feature dataset. This feature dataset may contain hundreds of feature vectors, resulting in high data dimensionality.

[0056] It should be noted that feature extraction should be combined with actual business needs, such as prioritizing defect types such as solder joint breakage or pin misalignment, to ensure the analysis is targeted.

[0057] In one possible implementation, this embodiment utilizes principal component analysis (PCA) to reduce the dimensionality of the feature dataset. For example, the feature dataset contains 500 feature vectors, each with 20 dimensions. Through PCA, the dimensionality is reduced from 20 to 5, retaining 95% of the variance information, generating a dimensionality-reduced feature set. After dimensionality reduction, data processing efficiency is improved, while still retaining key defect information, facilitating subsequent classification.

[0058] Specifically, this embodiment uses a support vector machine (SVM) algorithm to classify the dimensionality-reduced feature set and determine the defect category. For example, after the dimensionality-reduced feature set is input into the SVM, the classification model categorizes defects into three types: high-risk, medium-risk, and low-risk, based on the training data. Suppose the feature set of a certain critical area shows an abnormally reduced solder joint area; the classification result would be high-risk. This classification method achieves accurate judgment through quantified features.

[0059] In one possible implementation, if the defect category is high-risk, the key region is enhanced using an image processing algorithm.

[0060] For example, images of high-risk areas can be contrast-enhanced and sharpened to generate enhanced images. In the enhanced images, defect edges are clearer, and the area measurement error is reduced from 5% to 2%, facilitating subsequent fine-grained feature extraction.

[0061] For example, when extracting refined feature vectors, image enhancement can further analyze the microscopic texture and boundary details of defects, generating a refined feature set. Assuming the refined feature set contains information about the crack length and depth of the defects, these feature vectors have lower dimensionality but more concentrated information.

[0062] It should be noted that when refining feature extraction, attention should be paid to the physical characteristics of defects, such as the crack depth, which may directly affect the circuit conductivity.

[0063] In one possible implementation, this embodiment utilizes clustering analysis algorithms to analyze the refined feature set and determine the defect distribution pattern. For example, the K-means clustering algorithm is used to divide the refined feature set into three categories, corresponding to uniform distribution, clustered distribution, and random distribution, respectively. Suppose the analysis results show that defects on a circuit board are clustered, concentrated in several key areas, suggesting a possible manufacturing process problem. This distribution pattern analysis helps identify the causes of defects.

[0064] Understandably, each step of the above method closely aligns with the business needs of integrated circuit defect detection, forming a complete technology chain from localization to classification and distribution analysis. Each step, through feature extraction and processing, progressively focuses on high-risk defects, ensuring the accuracy and practicality of the detection results.

[0065] Furthermore, based on the defect feature vectors, the process of using a multi-branch neural network structure to process multiple defect types and determine the multi-variety classification results includes: The original image data of the key region is acquired and preprocessed to obtain the first image data. Defect feature vectors are extracted from the first image data, and multi-scale feature extraction is performed through a convolutional neural network to obtain a set of feature vectors; If the feature vector set includes scratch or short-circuit features, a multi-branch neural network structure is used to process each branch independently to determine the preliminary defect type; The output features of a multi-branch neural network are fused and classified through a fully connected layer to obtain a set of classification results. Extract the defect type with the highest confidence from the classification result set, filter it using a preset threshold, and determine the final defect type; Based on the final defect type and key area location information, a defect distribution map is generated to obtain defect distribution data; statistical analysis is performed on the defect distribution data to determine the multi-variety classification results.

[0066] For example, in the business scenario of electronic circuit board defect inspection, acquiring raw image data of key areas is the core starting point of the entire process. Raw image data typically comes from high-resolution industrial cameras, which capture the circuit board surface to identify subtle defect features. Assuming the captured image resolution is 1920x1080 pixels, including connection points and pin areas, preprocessing techniques for image denoising and enhancement employ Gaussian filtering to remove noise and ensure clear defect edges; simultaneously, contrast stretching enhances image details to obtain the first image data.

[0067] For example, for a pin area on a circuit board, the grayscale value range of the preprocessed image is expanded from [50, 200] to [0, 255] to highlight the feature areas of scratches or short circuits. When extracting defect feature vectors from the first image data, the defect contour can be identified and feature vectors generated by edge detection algorithms such as the Canny algorithm.

[0068] For example, if a scratch with a length of 20 pixels is detected in a pin area, its feature vector contains information such as the scratch's length, width, and grayscale difference. When performing multi-scale feature extraction, convolutional neural networks can be designed with a three-layer convolutional structure, using 3x3, 5x5, and 7x7 convolutional kernels respectively, to capture defect features at different scales.

[0069] For example, a 3x3 convolution kernel extracts local features from small scratches, while a 7x7 convolution kernel captures features from larger short-circuit regions, generating a set of feature vectors containing multi-scale information.

[0070] In one possible implementation, if the feature vector set contains significant scratch or short-circuit features, it is processed using a multi-branch neural network. Each branch is optimized for a specific defect type; for example, one branch focuses on scratch detection, and another on short-circuit detection. Suppose the feature vector of a pin region shows a grayscale abrupt change exceeding 80, indicating a potential short circuit. The short-circuit branch network will analyze this independently and output a preliminary defect type. When fusing the multi-branch output features, a weighted average can be used to integrate the results from each branch before inputting them into a fully connected layer for classification.

[0071] For example, the scratch branch outputs a confidence level of 0.7, the short circuit branch outputs a confidence level of 0.9, and the short circuit has the highest confidence level in the final classification result set. The screening threshold is set to 0.85, and the final defect type is determined to be short circuit.

[0072] Specifically, when generating defect distribution maps for the final defect type, post-processing algorithms such as morphological operations are used to smooth the boundaries of defect regions. For example, for short-circuit defects, a two-dimensional heatmap containing the defect location and area is generated, showing that the short-circuit region is concentrated near the pin connection point. After the defect distribution data is generated, statistical analysis methods can be used for correlation analysis.

[0073] For example, statistics show that short-circuit defects are more common in areas where the pin spacing is less than 0.5 mm, indicating that excessively small spacing may increase the risk of short circuits. This correlation analysis helps optimize circuit board design.

[0074] In one embodiment, the above process can combine multi-scale feature extraction and multi-branch networks to ensure accurate classification of different types of defects.

[0075] For example, for scratch defects, feature extraction focuses on edge sharpness, while for short-circuit defects, it focuses on regional connectivity. This multi-faceted analysis supports each other, forming a unified defect detection scheme and improving the comprehensiveness and reliability of detection.

[0076] Furthermore, based on the multi-variety classification results, the process of extracting specific defect images of the data communication interface to obtain accurate identification indicators for communication interface defects includes: The input image is classified using a multi-class classification model to obtain an initial classification result that includes communication interface defects; A convolutional neural network is used to extract specific defect images from the initial classification results to obtain feature representations of the defect regions. If the confidence level of the feature representation of the defect region is higher than the preset threshold, then the specific defect image is segmented again to determine the specific location of the defect. Image processing techniques are used to enhance the identified defect locations, resulting in enhanced defect images. Based on the enhanced defect images, a support vector machine model is used to classify the defect types and obtain the probability distribution of the defect categories; Based on the probability distribution of defect categories, calculate the precise identification index and determine the precise identification index for communication interface defects.

[0077] For example, this embodiment addresses the task of classifying defects in communication interfaces by using a multi-class classification model to process the input image. Assume the input is a high-resolution microscopic image of a communication interface, containing potential defects such as cracks, oxidation, or poor contact. The model employs a deep learning framework, such as a ResNet-based convolutional network, and is pre-trained to classify the interface image. The initial classification results may include categories such as "crack," "oxidation," and "normal," each with a probability value, such as 0.7 for crack, 0.2 for oxidation, and 0.1 for normal. This classification result provides the basis for subsequent processing.

[0078] In one possible implementation, extracting specific defect images from the initial classification results can be accomplished using a convolutional neural network. Assuming the initial classification identifies cracks with the highest probability, the network further focuses on suspected crack areas in the image, extracting features such as texture and edge sharpness to form a feature representation vector. If the confidence level of the feature representation exceeds a preset threshold, such as 0.65, secondary segmentation is triggered. Secondary segmentation employs a region growing algorithm, accurately locating defect areas based on the grayscale differences of the cracks, such as determining that the crack is located at a specific position on an interface pin.

[0079] Specifically, image processing techniques to enhance defect locations are achieved through histogram equalization. Assuming the crack region has low grayscale values, histogram equalization improves contrast, making crack edges clearer and facilitating subsequent analysis. The enhanced defect image can clearly display features such as crack length and width, providing reliable input for type classification.

[0080] For example, when using a support vector machine (SVM) model for defect type classification, it is trained based on the feature vectors of the enhanced image. Assuming the training data contains various defect samples such as cracks and oxidation, the model outputs a probability distribution, such as 0.8 for cracks, 0.15 for oxidation, and 0.05 for other defects. This probability distribution reflects the likelihood of the defect type; a high probability for cracks indicates strong classification reliability.

[0081] In one possible implementation, the accuracy identification metric is calculated using a confusion matrix. Assuming 100 images are tested, 90 cracks are correctly identified, 5 are falsely identified, and 5 are missed, resulting in an accuracy of 90%. Key statistics such as precision and recall can generate the final analysis results, clearly identifying the main types and distribution of communication interface defects.

[0082] It should be noted that the defect distribution data can be visualized using heatmaps. Assuming cracks are concentrated at pin connections, the heatmap uses color intensity to represent defect density, allowing analysts to quickly locate problem areas. This method improves the efficiency and accuracy of defect detection, providing a basis for subsequent repairs or design optimization.

[0083] Preferably, the final analysis results can be combined with the defect type and location to generate a detailed report.

[0084] For example, the report indicated that cracks mainly appeared at the root of the leads, possibly due to improper soldering processes, and recommended optimizing soldering parameters. This analysis provides direct guidance for production process improvements, thereby enhancing product quality.

[0085] Furthermore, based on precise identification indicators, the process of fusing temperature distribution characteristics from visible light images and thermal imaging images to establish a correlation model between temperature difference and defect type includes: Acquire visible light images and thermal imaging images, and align the two images using image registration technology to obtain registered image data; Temperature distribution characteristics are extracted from the registered image data, and dimensionality reduction is performed using principal component analysis to obtain the dimensionality-reduced temperature distribution vector. Based on the reduced-dimensional temperature distribution vector, the temperature difference value of each region is calculated to obtain the temperature difference feature set; If the temperature difference value in the temperature difference feature set exceeds the preset threshold, the defect type is classified by the support vector machine algorithm to obtain the preliminary defect classification result. Based on the preliminary defect classification results, a convolutional neural network was used to extract deep features from the registered image data to obtain a deep feature set. Based on the depth feature set and the temperature difference feature set, a correlation model of the random forest algorithm is constructed to establish the mapping relationship between temperature difference and defect type.

[0086] For example, in communication interface defect detection, acquiring visible light and thermal images is a crucial first step. Visible light images can clearly capture surface details of the communication interface, such as scratches and stains, while thermal images reflect the temperature distribution of the interface during operation, potentially revealing abnormal heating caused by poor contact or short circuits. Suppose that on a communication interface circuit board, a visible light image shows slight corrosion at a metal contact, while a thermal image shows that the temperature in that area is 10 degrees Celsius higher than the surrounding area. This difference suggests a potential defect.

[0087] In one possible implementation, image registration is crucial. Because visible light cameras and thermal imaging cameras operate on different imaging principles, their images may exhibit deviations in viewpoint or resolution. Registration can be achieved through feature point matching, such as selecting screw holes or contacts on a circuit board as reference points, and using affine transformations to align the thermal imaging image to the coordinate system of the visible light image. The registered image data forms a unified spatial reference, ensuring the accuracy of subsequent analysis.

[0088] For example, after registration, it can be found that the corroded area and the high-temperature area completely overlap, indicating that the physical location of the defect is the same.

[0089] Specifically, principal component analysis (PCA) is used for dimensionality reduction when extracting temperature distribution characteristics. Assuming the registered image contains 1000 pixels, each with temperature and color information, directly processing high-dimensional data can lead to computational complexity. PCA uses linear transformation to compress the temperature distribution data into several key feature vectors.

[0090] For example, after dimensionality reduction, three principal component vectors are obtained, representing the overall temperature level, local temperature difference, and edge temperature change, respectively, retaining 90% of the original information. This dimensionality reduction process simplifies subsequent calculations while preserving key temperature features.

[0091] For example, when calculating the temperature difference feature set, a threshold of 5 degrees Celsius is set. Suppose the temperature difference in a certain interface area reaches 8 degrees Celsius, exceeding the threshold, indicating a potential defect. The support vector machine algorithm then performs a preliminary classification of these areas, categorizing defects into three types based on temperature difference values ​​and texture features: poor contact, short circuit, or material degradation.

[0092] For example, poor contact may manifest as localized high temperatures without obvious surface damage, while a short circuit may be accompanied by obvious burn marks.

[0093] In one possible implementation, convolutional neural networks are used for deep feature extraction. The network extracts edge, texture, and shape features from the registered image through multiple layers of convolution and pooling.

[0094] For example, the network might identify irregular edges in a corroded area and, combined with high-temperature information, infer that it is a material degradation defect. After combining the deep feature set and the temperature difference feature set, the random forest algorithm constructs a mapping relationship. The random forest analyzes the correlation between temperature difference and defect type through the ensemble of multiple decision trees.

[0095] For example, training data shows that when the temperature difference exceeds 7 degrees Celsius and is accompanied by irregular edges, there is a 90% probability of material degradation.

[0096] Specifically, defect identification of new input images relies on the aforementioned mapping relationship. Assuming a newly acquired image of a communication interface shows a temperature difference of 9 degrees Celsius and irregular edge features, a random forest model can quickly identify it as a material degradation defect. This method, through multimodal image fusion and multi-algorithm collaboration, ensures the comprehensiveness and reliability of defect identification, making it suitable for real-time detection scenarios of communication interfaces.

[0097] Furthermore, if the temperature difference value in the temperature difference feature set exceeds a preset threshold, the defect type is classified using a support vector machine algorithm to obtain a preliminary defect classification result. This process includes: Temperature distribution feature data, including pixel temperature values ​​and spatial coordinates, are obtained from visible light images and thermal imaging images. Infrared imaging technology is used to extract temperature gradient features from thermal imaging images to obtain a temperature gradient distribution map; By fusing the temperature gradient distribution map and the texture features of the visible light image, a fused feature matrix is ​​generated; Based on the fused feature matrix, the feature vector of the defect region is extracted by a convolutional neural network to determine the boundary of the defect region; Based on the boundary of the defect area, the deviation between the temperature difference value and the preset threshold is calculated to obtain the initial classification result of the defect type; If the initial defect type classification result matches the historical defect database, the final defect type judgment result will be output.

[0098] For example, when acquiring temperature distribution feature data from visible light images and thermal imaging images, an infrared thermal imager can capture the thermal radiation information of the target object and generate pixel temperature values ​​and spatial coordinates.

[0099] Specifically, assuming the object being detected is a metal plate, the visible light image provides details of the surface texture, such as scratches or rust, while the thermal image records the temperature value of each pixel, for example, 50°C in the center and 45°C at the edges, with spatial coordinates represented by a two-dimensional grid. This data provides the foundation for subsequent feature extraction.

[0100] In one embodiment, when using infrared imaging technology to extract temperature gradient features, a temperature gradient distribution map is generated by calculating the temperature difference between adjacent pixels.

[0101] For example, if the temperature in a certain area of ​​a metal plate drops sharply from 50°C to 40°C, it indicates that a defect may be causing abnormal heat conduction. The gradient distribution map visually displays the rate of temperature change through color coding, and high gradient areas usually correspond to the location of potential defects.

[0102] Specifically, when fusing texture features and temperature gradient distribution maps from visible light images, edge information from the visible light images is extracted, such as crack textures on the surface of a metal plate. The fusion process is achieved by superimposing feature matrices to generate a fused feature matrix containing both texture and temperature information.

[0103] For example, if the temperature gradient in the crack region is 2℃ / mm and the texture feature shows a crack with a width of 0.5mm, this fusion matrix can more comprehensively describe the defect characteristics.

[0104] For example, when extracting feature vectors of defect regions using a convolutional neural network, the fused feature matrix is ​​input into the network to extract high-dimensional features.

[0105] For example, the network identifies feature vectors in cracked regions that contain information about temperature abrupt changes and texture discontinuities, thereby determining the boundary of the defect region, such as the boundary of a crack that is 5 mm long and 0.3 mm wide. This boundary information provides a precise basis for subsequent analysis.

[0106] In one possible implementation, when calculating the deviation of the temperature difference value from a preset threshold, the normal area temperature difference threshold is set to 3℃. If the temperature difference in a certain area reaches 5℃, it is judged as abnormal and preliminarily classified as a crack or void type. This method can quickly screen potential defect areas.

[0107] It should be noted that when matching the initial classification results with the historical defect database, the defect type is confirmed by comparing records with similar temperature distribution and texture features in the historical data.

[0108] For example, database records show that crack defects are often accompanied by high gradient temperature differences and linear textures. Matching these can confirm that the current defect is a crack.

[0109] Specifically, when generating a sequence of feature vectors and grouping them using a clustering algorithm, the K-means algorithm can be used to divide the feature vectors into several defect feature clusters.

[0110] For example, the eigenvectors of crack clusters show high temperature gradients and linear textures, while void clusters show uniform temperature gradients and circular textures. This grouping helps to distinguish different defect types.

[0111] For example, when calculating the statistical characteristics of temperature distribution within a cluster, the average temperature difference and standard deviation of each cluster are statistically analyzed.

[0112] For example, if the average temperature difference of a crack cluster is 4℃ and the standard deviation is 0.5℃, a temperature distribution statistical vector is generated. This vector can quantify the temperature characteristics of the defect.

[0113] In one embodiment, when matching a preset defect type template, the temperature distribution statistical vector can be compared with templates such as cracks and voids in the template library.

[0114] For example, the crack template requires a temperature difference greater than 4°C and a linear texture. After matching, a refined defect type is output, which improves the classification accuracy.

[0115] This invention significantly improves the accuracy and efficiency of integrated circuit defect detection, and is particularly suitable for real-time quality control of highly complex circuits.

[0116] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for integrated circuit defect image recognition based on transfer learning, characterized in that, include: By analyzing the feature distribution differences between the source and target domain defect images, adaptive weight adjustment parameters are obtained. Based on the adaptive weights, the parameters are adjusted, and the pre-trained model is processed using transfer learning to obtain the optimized model parameters. Using the optimized model parameters, a hierarchical defect image recognition framework is constructed for different integration densities and complexities of integrated circuits; The severity level of the defect is determined based on the hierarchical defect image recognition framework. If the severity level of the defect is higher than a preset threshold, the key connection points and pin positions are located to obtain the defect feature vector of the key area. Based on the defect feature vectors, a multi-branch neural network structure is used to process various defect types and determine the multi-variety classification results. Based on the multi-variety classification results, specific defect images of the data communication interface are extracted to obtain accurate identification indicators for communication interface defects. Based on the precise identification indicators, the temperature distribution characteristics of visible light images and thermal imaging images are fused to establish a correlation model between temperature difference and defect type; Based on the aforementioned correlation model, integrated circuit defect image detection is performed to obtain detection results.

2. The method according to claim 1, characterized in that, The process of obtaining adaptive weight adjustment parameters by analyzing the feature distribution differences between the source and target domain defect images includes: Defect image features are extracted from the source and target domains to obtain pixel-level feature distributions; If the pixel-level feature distribution difference exceeds a preset threshold, the features are standardized to obtain a normalized feature distribution. A multi-level hierarchical model is constructed based on the normalized feature distribution, and multi-level feature vectors are extracted; Based on the feature difference vectors between the source and target domains, optimize the feature mapping and calculate the transfer learning success rate evaluation index. Multi-level features are integrated to generate a comprehensive evaluation score, and weights are extracted from the comprehensive evaluation score to adjust the initial parameters. If the variance of the initial weight adjustment parameter is less than a preset threshold, the parameter is adjusted by a regularization method to obtain the adaptive weight adjustment parameter.

3. The method according to claim 1, characterized in that, The process of adjusting parameters according to the adaptive weights and performing feature representation processing on the pre-trained model using transfer learning to obtain optimized model parameters includes: Load the pre-trained model and training data to obtain the initial model parameters; Based on the feature extraction results of the training data, calculate the adaptive weights and determine the weight allocation scheme. If the weight allocation scheme meets the preset threshold, the gradient descent algorithm is used to update the model parameters to obtain the optimized model parameters. If the feature representation does not converge, the learning rate is adjusted and the adaptive weights are recalculated. The model parameters are iteratively updated until the feature representation converges, and the final optimized model parameters are obtained.

4. The method according to claim 1, characterized in that, The process of constructing a hierarchical defect image recognition framework for different integration densities and complexities of integrated circuits using the optimized model parameters includes: The input integrated circuit defect image is preprocessed to obtain the first image; Extract the geometric and texture features of the defects from the first image to obtain a feature set; If the geometric features of the feature set meet a preset threshold, a convolutional neural network is used to classify the feature set to obtain a preliminary defect category. Based on the preliminary defect categories and integration density data, the classification weights of the hierarchical framework are adjusted to obtain an optimized classification model; The severity level of the defect is determined by performing a secondary analysis on the feature set using the optimized classification model. If the severity level is higher than a preset threshold, the first image is magnified locally to obtain a second image, and a support vector machine is used to verify the local features of the second image to determine the final severity level.

5. The method according to claim 1, characterized in that, The process of locating key connection points and pin positions to obtain defect feature vectors for critical areas includes: If the severity level of the defect is higher than the preset threshold, the input image is processed by the joint detection algorithm to locate the key connection points and pin positions and obtain the detection results. Based on the detection results, key regions are divided, and defect feature vectors of the key regions are extracted to obtain a feature dataset; If the defect feature vectors in the feature dataset meet the preset conditions, then the principal component analysis algorithm is used to reduce the dimensionality of the feature dataset to obtain a dimensionality-reduced feature set. Based on the reduced feature set, the defect features are classified using the support vector machine algorithm to determine the defect category; If the defect category belongs to the high-risk category, then image enhancement is performed on the key area to obtain the enhanced image, and the refined feature vector is extracted to obtain the refined feature set; Based on the refined feature set, a clustering analysis algorithm is used to analyze the defect distribution and determine the defect distribution pattern.

6. The method according to claim 1, characterized in that, The process of determining the multi-variety classification results by using a multi-branch neural network structure to process multiple defect types based on the defect feature vectors includes: The original image data of the key region is acquired and preprocessed to obtain the first image data. Defect feature vectors are extracted from the first image data, and multi-scale feature extraction is performed through a convolutional neural network to obtain a set of feature vectors. If the feature vector set includes scratch or short-circuit features, a multi-branch neural network structure is used to process each branch independently to determine the preliminary defect type; The output features of a multi-branch neural network are fused and classified through a fully connected layer to obtain a set of classification results. Extract the defect type with the highest confidence from the classification result set, filter it using a preset threshold, and determine the final defect type; Based on the final defect type and key area location information, a defect distribution map is generated to obtain defect distribution data; statistical analysis is performed on the defect distribution data to determine the multi-variety classification results.

7. The method according to claim 1, characterized in that, Based on the multi-variety classification results, the process of extracting specific defect images of the data communication interface portion and obtaining accurate identification indicators for communication interface defects includes: The input image is classified using a multi-class classification model to obtain an initial classification result that includes communication interface defects; A convolutional neural network is used to extract specific defect images from the initial classification results to obtain feature representations of the defect regions. If the confidence level of the feature representation of the defect region is higher than a preset threshold, then the specific defect image is segmented a second time to determine the specific location of the defect. Image processing techniques are used to enhance the identified defect locations, resulting in enhanced defect images. Based on the enhanced defect images, a support vector machine model is used to classify the defect types and obtain the probability distribution of the defect categories; Based on the probability distribution of defect categories, calculate the precise identification index and determine the precise identification index for communication interface defects.

8. The method according to claim 1, characterized in that, Based on the aforementioned accurate identification indicators, the process of establishing a correlation model between temperature difference and defect type by fusing the temperature distribution characteristics of visible light images and thermal imaging images includes: Acquire visible light images and thermal imaging images, and align the two images using image registration technology to obtain registered image data; Temperature distribution characteristics are extracted from the registered image data, and dimensionality reduction is performed using principal component analysis to obtain the dimensionality-reduced temperature distribution vector. Based on the reduced-dimensional temperature distribution vector, the temperature difference value of each region is calculated to obtain the temperature difference feature set; If the temperature difference value in the temperature difference feature set exceeds a preset threshold, the defect type is classified by the support vector machine algorithm to obtain a preliminary defect classification result. Based on the preliminary defect classification results, a convolutional neural network is used to extract deep features from the registered image data to obtain a deep feature set. Based on the depth feature set and temperature difference feature set, a correlation model of the random forest algorithm is constructed to establish the mapping relationship between temperature difference and defect type.

9. The method according to claim 1, characterized in that, If the temperature difference value in the temperature difference feature set exceeds a preset threshold, the process of classifying the defect type using a support vector machine algorithm to obtain a preliminary defect classification result includes: Temperature distribution feature data, including pixel temperature values ​​and spatial coordinates, are obtained from visible light images and thermal imaging images. Infrared imaging technology is used to extract temperature gradient features from thermal imaging images to obtain a temperature gradient distribution map; The temperature gradient distribution map and the texture features of the visible light image are fused to generate a fused feature matrix; Based on the fused feature matrix, feature vectors of the defect region are extracted using a convolutional neural network to determine the boundary of the defect region. Based on the boundary of the defect area, the deviation between the temperature difference value and the preset threshold is calculated to obtain the initial classification result of the defect type; If the initial defect type classification result matches the historical defect database, the final defect type judgment result will be output.

Images