Surface mounting detection system in semiconductor packaging

Abstract

The invention discloses a surface mounting detection system in semiconductor packaging, which relates to the field of automatic detection, and comprises a data acquisition module, a preprocessing module, a data processing module, a display module, a display module, a display module, a display module, a display module and a display module, and is characterized in that the data acquisition module acquires temperature information, a morphology image and a reflected light image of the surface of the semiconductor packaging in real time and simultaneously collects historical data of the semiconductor packaging; the system comprises a data preprocessing module for preprocessing collected temperature information, morphology images and reflected light images, a data fusion module for fusing the preprocessed information and images to form a multi-modal feature vector, a three-dimensional reconstruction module for generating a three-dimensional image of semiconductor packaging through a three-dimensional reconstruction algorithm and a multi-modal data set, a defect identification module for identifying defects of the semiconductor packaging, and a detection module for detecting the defects of the semiconductor packaging. And establishing a defect identification model based on the multi-modal feature vector, and outputting a defect identification result. Through CNN multi-scale analysis, defects under different scales can be captured, the weight of each mode is dynamically adjusted through a self-attention mechanism, it is ensured that a detection strategy can be flexibly adjusted according to the actual situation, and the accuracy and flexibility of defect recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of automated detection, and in particular to a surface mount detection system in semiconductor packaging. Background Art

[0002] With the rapid development of the semiconductor industry, integrated circuit packaging technology has been continuously advancing. Especially in the field of surface mount technology, the package size has been continuously shrinking, and the packaging density and complexity have been gradually increasing. Advanced packaging technologies such as complex packages in the forms of BGA, CSP, and Flip-Chip have become the core components in modern electronic devices.

[0003] Existing surface mount detection technologies have several deficiencies. First, single-modal detection technologies can only obtain features of a single aspect of the package surface and are difficult to meet multi-dimensional detection requirements. In addition, in existing defect recognition methods, fixed image processing algorithms are usually used for defect extraction and classification, lacking in-depth analysis and utilization of historical data and unable to fully explore the implicit patterns in normal samples. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a surface mount detection system in semiconductor packaging to solve the problem that single-modal detection technologies can only obtain features of a single aspect of the package surface and are difficult to meet multi-dimensional detection requirements. In addition, in existing defect recognition methods, fixed image processing algorithms are usually used for defect extraction and classification, lacking in-depth analysis and utilization of historical data and unable to fully explore the implicit patterns in normal samples.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a surface mount detection system in semiconductor packaging, which includes

[0008] a data acquisition module that real-time collects temperature information, topography images, and reflected light images on the surface of the semiconductor package, and simultaneously collects historical data of the semiconductor package;

[0009] a preprocessing module that preprocesses the collected temperature information, topography images, and reflected light images;

[0010] a multi-modal data fusion module that fuses the preprocessed information and images to form a multi-modal feature vector;

[0011] a three-dimensional reconstruction module that generates a three-dimensional image of the semiconductor package through a three-dimensional reconstruction algorithm and the multi-modal feature vector;

[0012] The defect recognition module establishes a defect recognition model based on the multi-modal feature vector set and outputs the defect recognition result;

[0013] The defect classification module classifies the defect recognition results output by the defect recognition model to generate defect classification results;

[0014] The defect localization module marks the classified defect positions on the encapsulated three-dimensional image and generates a preliminary detection report;

[0015] The adaptive adjustment module automatically adjusts the parameters of the detection device according to the preliminary detection report, performs secondary detection on key areas, and generates secondary detection results;

[0016] The feedback analysis module inputs the detection data after adaptive feedback adjustment into the defect recognition model again for analysis and confirmation, and finally generates a complete detection report.

[0017] As a preferred solution of the surface mount detection system in the semiconductor package described in the present invention, wherein: the information on the surface of the semiconductor package and the reflected light images of the surface of the semiconductor package at different polarization angles are collected in real time, and the specific steps are as follows,

[0018] The temperature change on the surface of the semiconductor package is recorded in real time through a thermal imaging device, the high-definition image of the package surface is collected in real time through an optical detection device, and the reflected light images at different polarization angles are collected through a polarization detection device.

[0019] As a preferred solution of the surface mount detection system in the semiconductor package described in the present invention, wherein: the collected information and images are preprocessed, and the specific steps are as follows,

[0020] Denoise the collected temperature change information, the high-definition image of the package surface, and the reflected light images;

[0021] Align, register, and enhance the pictures from different devices;

[0022] Convert the format of the data and images.

[0023] As a preferred solution of the surface mount detection system in the semiconductor package described in the present invention, wherein: the preprocessed information and images are fused to form a multi-modal feature vector, and the specific steps are as follows,

[0024] Use the Sobel operator to extract the edge features of the optical image;

[0025] Use the Gabor filter to extract the texture features related to the polarization angle in the polarization image;

[0026] Extract the temperature gradient features of the temperature on the surface of the semiconductor package;

[0027] Fuse the edge features, texture features, and temperature gradient features to form a multi-modal feature vector.

[0028] As a preferred solution of the surface mount detection system in the semiconductor package of the present invention, wherein: a three-dimensional image of the semiconductor package is generated through a three-dimensional reconstruction algorithm and a multi-modal data set, and the specific steps are as follows.

[0029] Map the fused multi-modal feature vector into a three-dimensional voxel grid, where the multi-modal feature vector is a voxel, and each voxel represents a small cube in three-dimensional space.

[0030] Data of different modalities are divided into individual voxels in three-dimensional space, and each voxel contains comprehensive physical information about its coordinate position in three-dimensional space.

[0031] Apply volume rendering technology, and assign colors and transparencies to each voxel according to the attribute values of the voxels to generate a three-dimensional visualization image of the package.

[0032] As a preferred solution of the surface mount detection system in the semiconductor package of the present invention, wherein: a defect recognition model is established based on the multi-modal feature vector, and a defect recognition result is output. The specific steps are as follows.

[0033] Extract the multi-modal data of normal sample data from historical data, and use a convolutional neural network to extract features from the data of each normal sample modality.

[0034] In the feature extraction stage of the convolutional neural network, add multi-scale convolution, capture features at different scales through convolutional kernels of different scales, and fuse the features at different scales.

[0035] Aggregate the multi-modal features of all normal samples to obtain a set of reference feature vectors.

[0036] Perform statistical analysis on each dimension of the multi-modal feature vector of normal samples, and calculate the mean and standard deviation of each dimension.

[0037] Real-time data is extracted in the same way as normal samples in historical data. During the feature extraction process, multi-scale convolution is used, and a self-attention mechanism is used during the fusion process to dynamically adjust the weights according to the importance of each modality to generate a final weighted fusion feature vector.

[0038] Compare the weighted fusion feature vector with the reference feature vector to calculate the standardized deviation of the real-time data.

[0039] Aggregate the standard deviations of each real-time data through the sum of absolute deviations to generate the overall deviation value of this point.

[0040] Based on historical data, set the deviation threshold using cross-validation;

[0041] When the overall deviation value is greater than the deviation threshold, it is determined that the real-time data at this position has anomalies and defects.

[0042] As a preferred solution of the surface mount detection system in the semiconductor package described in the present invention, wherein: classify the recognition results output by the defect recognition model to generate a defect classification result, which specifically includes the following steps,

[0043] Form a defect feature library by aggregating the feature sets of all known defect types;

[0044] Extract the features of the anomalies and defects output by the defect recognition model to form a feature vector of the anomalies and defects;

[0045] Calculate the similarity between the feature vector of the anomalies and defects and each known defect in the defect feature library through the Euclidean distance, and its expression is:

[0046]

[0047] where, dist(F,F a ) represents the similarity between the new defect F and the known defect F a , F represents the feature vector of the new defect, and F a represents the feature vector of the a-th known defect in the defect feature library, n represents the total dimension of the feature vector, F k represents the k-th feature component of the new defect feature vector F, and F a,k represents the k-th feature component of the a-th known defect F a in the defect feature library, and k represents the summation index;

[0048] According to the similarity between the new defect F and the known defect F a , use the nearest neighbor method to determine the category of the new defect;

[0049] After determining the category of the new defect, output the defect classification result.

[0050] As a preferred solution of the surface mount detection system in the semiconductor package described in the present invention, wherein: mark the classified defect positions on the three-dimensional image of the package and generate a preliminary detection report, which specifically includes the following steps,

[0051] Mark the positions of the defects on the three-dimensional image in a visual way, and highlight the defect positions in different colors, shapes and marks in the three-dimensional package image;

[0052] And generate a preliminary detection report including package identification information, defect statistics, defect details, and images and annotations.

[0053] As a preferred solution of the surface mount detection system in the semiconductor package of the present invention, wherein: automatically adjust the parameters of the detection device according to the preliminary detection report, and perform secondary detection on the key areas to generate a secondary detection result, specifically including the following steps:

[0054] Automatically adjust the resolution, detection frequency, light source and angle, and detection method of the detection device according to the preliminary detection report;

[0055] Define the positions of the defects marked on the three-dimensional image as the key areas;

[0056] Perform secondary detection on the key areas with the device after adjusting the parameters, and generate a secondary detection result.

[0057] As a preferred solution of the surface mount detection system in the semiconductor package of the present invention, wherein: input the detection data after adaptive feedback adjustment into the defect recognition model again for analysis and confirmation, and finally generate a complete detection report, specifically including the following steps:

[0058] Input the automatically adjusted secondary detection result into the defect recognition model, and perform final confirmation on all detected defects, and update the defect classification result at the same time;

[0059] After feedback analysis and confirmation, generate a complete detection report including all detected defect information, detailed analysis of the defects, and defect classification results.

[0060] The beneficial effects of the present invention are as follows: By fusing the preprocessed data to form a multi-modal feature vector, the extraction of multiple features on the package surface is realized. The fused feature vector can more comprehensively reflect the state of the package. This fusion process enhances the analysis ability, enabling it to make comprehensive judgments based on multiple data sources, ensuring that potential defects are not missed, and improving the comprehensiveness and accuracy of detection. In addition, by establishing a defect recognition model based on the multi-modal feature vector, using CNN to extract features, and combining multi-scale convolution and self-attention mechanism for analysis, the defect recognition result is output. At the same time, by comparing the benchmark vector of historical data with real-time data, anomalies and defects are identified in a timely manner. Through the multi-scale analysis of CNN, defects at different scales can be captured, and the weights of each modality can be dynamically adjusted through the self-attention mechanism to ensure that the detection strategy can be flexibly adjusted according to the actual situation, improving the accuracy and flexibility of defect recognition. Description of the Drawings

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0062] Figure 1 It is a system diagram of the surface mount detection system in the semiconductor package in Embodiment 1.

[0063] Figure 2 It is a schematic diagram of the surface mount detection in the semiconductor package in Embodiment 1. Specific Embodiments

[0064] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention in conjunction with the drawings of the specification.

[0065] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0066] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not all refer to the same embodiment, nor is it a separate or selectively exclusive embodiment from other embodiments.

[0067] Embodiment 1, referring to Figure 1 and Figure 2 , is the first embodiment of the present invention. This embodiment provides a surface mount detection system in a semiconductor package, including the following steps:

[0068] A data acquisition module that real-time collects the temperature information, topography image, and reflected light image on the surface of the semiconductor package, and simultaneously collects the historical data of the semiconductor package;

[0069] The function of the thermal imaging device is to detect the temperature distribution of different regions on the surface of the semiconductor package and real-time monitor the possible local overheating or uneven heat dissipation phenomena during the packaging process. The thermal imaging device scans the surface of the package through an infrared sensor, captures its thermal radiation information, and converts it into temperature data;

[0070] Optical inspection equipment is mainly used to obtain high-resolution images of the semiconductor package surface, capturing the appearance features, surface quality, and physical structure of the package. The optical inspection equipment can scan in the visible light band and generate high-definition images to help detect surface defects such as cracks, scratches, foreign objects, etc.

[0071] The polarized light detection equipment controls the polarization angle of the light source and collects the reflected light images of the package surface at different polarized light angles. Different polarized light angles can reveal the texture, stress distribution, and details of the material structure on the package surface. Polarized light imaging technology can be particularly effective in revealing micro stress concentrations, bubbles, or foreign objects that are difficult to detect in ordinary optical images.

[0072] Furthermore, by combining the thermal imaging equipment, optical inspection equipment, and polarized light detection equipment, the system can obtain comprehensive information about the package surface from three different dimensions: temperature, appearance, and microstructure. The unique data provided by each device complements each other.

[0073] The preprocessing module preprocesses the collected temperature information, topography images, and reflected light images;

[0074] The data and images are processed through a denoising algorithm to eliminate or reduce the influence of noise.

[0075] Images from different sources are aligned to the same coordinate system so that physical points at the same position can be corresponding, and enhancement processing is performed, such as contrast enhancement, edge detection enhancement, etc., to highlight the key information in the images.

[0076] The content of format conversion includes standardization of image formats (such as converting from different image formats such as JPEG, PNG to a unified format), adjustment of data resolution, and conversion of color spaces (such as converting RGB images to grayscale images or other color spaces).

[0077] Furthermore, preprocessing reduces false detections and missed detections, improves the reliability of detection, and enhances the compatibility and efficiency of processing data, reducing the processing bottlenecks caused by format incompatibility.

[0078] The data fusion module fuses the preprocessed information and images to form a multi-modal feature vector;

[0079] The Sobel operator is a filter widely used in edge detection. By calculating the gradients of the image in the horizontal and vertical directions, it can effectively extract the edge information in the image. The operator is particularly sensitive to regions with significant brightness changes in the image (i.e., the edge part) and is often used to detect surface defects such as object contours, cracks, scratches, etc.

[0080] Apply the Sobel operator to the high-definition images collected by the optical detection device. The Sobel operator calculates the gradients of the image in the horizontal and vertical directions respectively, and generates an edge intensity image by combining the gradient information in these two directions.

[0081] The Gabor filter is a frequency-domain filter commonly used for texture feature extraction of images. It can capture both the spatial information and frequency information of the image, and is particularly suitable for extracting directional textures and periodic structures in the image. For polarized images, the light reflection characteristics at different polarization angles will reveal the texture features of the packaging surface material and the internal stress distribution.

[0082] Apply the Gabor filter to the reflected light images collected by the polarization detection device. The core of the Gabor filter is to decompose the image in terms of direction and frequency, so as to effectively capture the texture features in the polarized image.

[0083] The temperature gradient on the semiconductor packaging surface refers to the rate of temperature change between different regions on the packaging surface. The temperature gradient information can reveal the thermal distribution of the packaging surface under working conditions, and helps to discover potential problems caused by thermal stress, poor heat dissipation or excessive heat concentration.

[0084] Obtain the temperature distribution map of the packaging surface through the thermal imaging device and calculate the temperature gradient.

[0085] After calculating the edge features, texture features and temperature gradient features respectively, combine them into a multi-modal feature vector through the feature fusion algorithm. This feature vector contains key information from different modalities and can provide more accurate data support for subsequent defect identification and 3D reconstruction.

[0086] Furthermore, multi-modal feature fusion can integrate the data advantages of different modalities, thus forming a more comprehensive and detailed description of the packaging surface. The edge features, texture features and temperature gradient features respectively provide key information in terms of physical appearance, material structure and thermal performance. Their fusion has the ability to accurately detect defects from multiple angles. Through this feature fusion, the accuracy and coverage of defect detection can be greatly improved, and the limitations of single-modal data can be effectively avoided.

[0087] The 3D reconstruction module generates a 3D image of the semiconductor package through the 3D reconstruction algorithm and the multi-modal feature vector;

[0088] Perform spatial partitioning on the packaging surface, and divide the packaging surface and the internal space into several small cubes (i.e., voxels).

[0089] Map the fused multi-modal feature vector into these voxels.

[0090] Each voxel contains coordinate information in the three-dimensional space it occupies, as well as multimodal feature information at that location (such as edge, texture, and temperature data).

[0091] Based on the spatial coordinates of each voxel, the eigenvalue of the multimodal data (such as edge intensity, texture direction, and temperature value) is associated with the voxel.

[0092] In three-dimensional space, voxels at different positions contain different physical information.

[0093] For example, a voxel may contain a high temperature gradient and distinct edge features, indicating that there may be thermal stress concentration and surface cracks at that location.

[0094] Based on the attribute value of each voxel (such as edge intensity, texture feature, or temperature gradient), a color and transparency are assigned to it.

[0095] For example, voxels with high temperature can be rendered red, voxels with high edge intensity can be rendered bright white, and voxels with distinct texture features can be given a specific color or direction indication. The setting of transparency can help inspectors view the internal structure of the package in the three-dimensional image without being blocked by surface information.

[0096] Furthermore, through volume rendering technology, the three-dimensional voxel grid is transformed into an intuitive three-dimensional image. Through color and transparency, the physical properties of the package surface can be intuitively understood. This three-dimensional visualization image enhances the efficiency and effectiveness of defect recognition. Especially in the detection of complex package structures and subtle defects, the three-dimensional image can provide more detailed information.

[0097] The defect recognition module establishes a defect recognition model based on the multimodal feature vector and outputs the defect recognition result;

[0098] Extract the multimodal data of normal samples from historical data. Multimodal data usually includes different types of perceptual data, such as optical images, polarized images, temperature data, etc. The data of each modality can provide physical information from different angles, which helps in the comprehensive analysis of the package surface.

[0099] Defects on the package surface may occur at different sizes and scales. Therefore, single-scale feature extraction may not be sufficient to comprehensively capture defects. For this reason, multi-scale convolution is introduced in the feature extraction stage of the CNN. By using convolution kernels of different sizes, features can be extracted from different scales, and its expression is:

[0100]

[0101] where, f i,jdenotes the feature vector of the j-th modality of the i-th sample, where i represents the sample index, the i-th sample, and j represents the modality index, the j-th modality. Concat() represents combining different feature vectors into a larger vector, usually by concatenating columns or rows. CNN represents a convolutional neural network. k1 represents the size of the convolutional kernel, which refers to the size of the first convolutional kernel used in the CNN, and x i,j represents the original input data of the j-th modality of the i-th sample. If the input is image data, x i,j might be an image. k2 represents the size of the convolutional kernel, which represents the size of the second convolutional kernel used in the CNN and is usually different from k1.

[0102] Small-scale convolutional kernels are used to capture fine features on the encapsulated surface, such as tiny cracks or fine scratches.

[0103] Large-scale convolutional kernels are used to capture features in larger regions, such as large temperature gradients or large-area texture changes.

[0104] Fuse the features at different scales to ensure that the data of each modality can fully capture features at different scales.

[0105] Aggregate the features of all normal samples. By aggregating the features of these normal samples, a set of benchmark feature vectors are generated, which represent the typical features of normal samples in different modalities. The expression is:

[0106]

[0107] where, F 基准 denotes the benchmark feature vector obtained by averaging the feature vectors of all normal samples. N represents the total number of normal samples, and f i denotes the final fused feature vector of the i-th sample.

[0108] Conduct statistical analysis on the feature values of each dimension to calculate the mean and standard deviation. The expression for calculating the mean is:

[0109]

[0110] where, μ d denotes the mean on the d-th dimension, representing the average of the feature values of all normal samples on the d-th dimension. μ represents the mean, which is usually used to describe the central tendency of a set of data. d represents the d-th dimension of the feature vector. A feature vector of a sample may have multiple dimensions, and each dimension represents a certain feature, and f i,d denotes the feature value of the d-th dimension of the final fused feature vector f i of.

[0111] The expression for calculating the standard deviation is:

[0112]

[0113] where σ d represents the standard deviation of the d-th dimension.

[0114] For real-time data, multi-modal feature extraction is performed in the same way as for normal samples. During feature fusion, the system introduces a self-attention mechanism to dynamically adjust the weights of each modality. Its expression is:

[0115]

[0116] where f 实时 represents the fused feature vector of real-time data, M represents the number of modalities of this real-time data, α j represents the weight coefficient of the j-th modality for the fused feature vector f 实时 , j represents the index of the modality, the j-th modality, f 实时,j represents the real-time fused feature vector of the j-th modality.

[0117] To measure the difference between real-time data and the benchmark feature vector, the standardized deviation of each feature dimension is calculated, and its expression is:

[0118]

[0119] where z d represents the value after standardizing the real-time feature f 实时,d in the d-th dimension, f 实时,d represents the real-time fused feature f 实时 at the feature value in the d-th dimension.

[0120] The standardized deviation vectors of real-time data are aggregated to generate an overall deviation value, and its expression is:

[0121]

[0122] where D represents the overall deviation value, D 总 represents the total number of dimensions of the feature vector of real-time data, |z d | represents the standardized deviation of the d-th feature dimension.

[0123] Cross-validation is performed using historical data to determine the deviation threshold T. This threshold is used to distinguish normal data from abnormal data.

[0124] During the cross-validation process, different thresholds are used to test the historical data, and finally a threshold that can maximize the distinction between normal and abnormal data is selected.

[0125] Compare the overall deviation value D of the real-time data with the deviation threshold T.

[0126] If the overall deviation value exceeds the threshold, the system determines that there are abnormalities and defects in the real-time data at this position and outputs the detection result.

[0127] Furthermore, through multi-scale convolution, defect information of different sizes and scales can be captured to ensure the comprehensiveness of detection. And by calculating the standardized deviation and the overall deviation value, the degree of abnormality of the real-time data can be quantified to achieve accurate defect recognition.

[0128] The defect classification module classifies the defect recognition results output by the defect recognition model to generate defect classification results;

[0129] Collect a large number of samples of known defects through data analysis, extract the features of each sample, and form a feature library with these features. Each feature vector in the feature library represents a certain known defect.

[0130] Extract the feature vector of the abnormality or defect output by the defect recognition model.

[0131] This feature vector is a certain description of the defect itself and contains multi-dimensional information such as shape, texture, color, position, size, etc.

[0132] Calculate the similarity between the feature vectors of the abnormality and defect and each known defect in the defect feature library through the Euclidean distance. Its expression is:

[0133]

[0134] where dist(F,F a ) represents the similarity between the new defect F and the known defect F a ; F represents the feature vector of the new defect, and F a represents the feature vector of the a-th known defect in the defect feature library. n represents the total dimension of the feature vector, F k represents the k-th feature component of the new defect feature vector F, and F a,k represents the k-th feature component of the a-th known defect F a in the defect feature library. a represents the subscript of the defect feature library, and k represents the summation index;

[0135] Quantify the similarity between the new defect and each known defect by calculating the Euclidean distance between the new defect and each known defect.

[0136] Select the known defect with the smallest distance through the nearest neighbor method.

[0137] Output the classification result of the defect according to the category to which the new defect belongs determined by the nearest neighbor method.

[0138] The defect location module marks the classified defect positions on the encapsulated three-dimensional image and generates a preliminary inspection report;

[0139] Mark the detected defect positions on the three-dimensional model through a coordinate system. Each defect is assigned its exact coordinates in three-dimensional space (such as (x, y, z)).

[0140] By analyzing the defect feature vectors, determine the specific position of the defect in the encapsulated object (such as the encapsulation surface, internal structure, etc.).

[0141] According to the type or severity of the defect, use different colors to highlight the defects. For example, red can indicate severe defects, yellow for minor defects, and green for negligible defects.

[0142] Mark different types of defects by using different geometric shapes (such as circles, squares, triangles, etc.). The choice of shape can be distinguished according to the attributes of the defect (such as cracks, holes, bubbles, etc.).

[0143] Next to each defect, marks or labels can be attached to display information such as the defect number, type, size, location, etc.

[0144] After the marking is completed, generate a preliminary inspection report containing the encapsulation identification information, defect statistics, defect details, and images and markings.

[0145] The encapsulation identification information refers to product information and encapsulation information.

[0146] The defect statistics refer to the total number of defects, type distribution, and severity.

[0147] The defect details refer to the defect number, defect location, defect description, and severity.

[0148] The images and markings refer to three-dimensional image screenshots and defect marking diagrams.

[0149] Furthermore, through the visualization of the three-dimensional image, the spatial position and shape of the defect can be presented more intuitively. Traditional two-dimensional images are difficult to reflect the specific position of the defect inside the object or on a complex surface, while three-dimensional visualization can provide a clearer spatial perception. Through the highlighting in three-dimensional space, defects can be marked more effectively, avoiding omissions or misjudgments. Especially for defects inside the encapsulation, three-dimensional display can better reflect their positions and severities. The use of different colors, shapes, and markings facilitates the distinction of different types and severities of defects, making the detection results more explicit.

[0150] The adaptive adjustment module automatically adjusts the parameters of the detection device according to the preliminary detection report, and performs secondary detection on key areas to generate secondary detection results;

[0151] If the preliminary detection shows that the sizes of some defects are large and obvious, choose to maintain a lower resolution to save detection time and resources. For tiny or difficult-to-identify defects, automatically increase the resolution to obtain higher-quality images in the secondary detection.

[0152] In the preliminary detection, if the defect types in some areas are complex (such as dynamic defects or rapidly changing defects), it is necessary to increase the detection frequency to capture more images for analysis. For areas that do not require high-frequency detection, appropriately reduce the detection frequency to reduce the data volume and save bandwidth and storage resources.

[0153] According to the defect types in the preliminary detection report, automatically adjust parameters such as the intensity, color, and angle of the light source. For example, for fine surface cracks, a side light source may be needed to enhance reflection; for internal bubbles, the position of the light source may need to be adjusted to penetrate the detection. Automatically select the most suitable light source settings in the secondary detection to ensure that defects can be captured more clearly.

[0154] According to the preliminary detection results, different detection methods can be selected according to the actual situation to optimize the secondary detection. For example: optical detection, ultrasonic detection, X-ray detection, and electromagnetic detection.

[0155] The key areas are defined based on the severity of the defects, the density of the defects, and the detection difficulty.

[0156] After parameter adjustment, perform secondary detection on the key areas. At this time, higher resolution, higher frequency, or switching to a more suitable detection method (such as X-ray, ultrasonic, etc.) will be used to obtain more detection data.

[0157] Furthermore, the purpose of the secondary detection is to further confirm the defects found in the preliminary detection and obtain more detailed characteristics of the defects to ensure that there are no omissions or misjudgments.

[0158] Generate secondary detection results according to the secondary detection.

[0159] The secondary detection results include detailed defect descriptions, correction of preliminary detection errors, high-precision positioning, defect classification, and priority adjustment.

[0160] Furthermore, the function of automatically adjusting device parameters can greatly reduce the need for manual intervention. Optimizing the parameters automatically according to the preliminary detection report ensures that the device can operate in the best state during the secondary detection, reducing the time and error of manual adjustment of the device. By automatically focusing on the key areas, it can maintain the overall detection efficiency while focusing on the areas that require the most refined detection, avoiding global redundant detection.

[0161] The feedback analysis module inputs the detection data after adaptive feedback adjustment into the defect recognition model again for analysis and confirmation, and finally generates a complete detection report.

[0162] During the preliminary detection and secondary detection, some pseudo-defects or noises may be detected. These pseudo-defects may be caused by factors such as device errors and changes in lighting conditions. The defect recognition model can remove these false alarms by further analyzing the detection results, ensuring that the finally confirmed defects actually exist. At the same time, some defects may not be detected during the preliminary detection and are identified after the secondary refined detection. The defect recognition model will confirm these newly added defects to ensure that no important defects are missed.

[0163] The model may re-evaluate the severity of some defects based on the new detection results and parameter adjustments. For example, some parts that were considered minor defects during the preliminary detection may be re-evaluated as serious defects after the secondary detection.

[0164] Generate a complete detection report including an overview of defects, a detailed analysis of defects, and updated classification results.

[0165] The overview of defects includes the total number of defects, defect classification statistics, and the distribution of defect severity.

[0166] The detailed analysis of defects includes the defect location, a description of defect characteristics, and an analysis of the impact of defects.

[0167] The updated classification results include a description of classification updates.

[0168] Furthermore, through the analysis of the secondary detection results by the defect recognition model, defects can be identified and confirmed more accurately, avoiding false alarms and missed detections. This greatly improves the reliability of the detection, making the finally confirmed defects more real and accurate.

[0169] In summary, the present invention: by fusing the preprocessed data to form a multi-modal feature vector, realizes the extraction of multiple features on the encapsulation surface. The fused feature vector can more comprehensively reflect the state of the encapsulation. This fusion process enhances the analysis ability, enabling comprehensive judgment based on multiple data sources, ensuring that potential defects are not missed, and improving the comprehensiveness and accuracy of detection. In addition, by establishing a defect recognition model based on the multi-modal feature vector, using CNN to extract features, and combining multi-scale convolution and self-attention mechanism for analysis, the defect recognition result is output. At the same time, by comparing the benchmark vector of historical data with real-time data, anomalies and defects are identified in a timely manner. Through the multi-scale analysis of CNN, defects at different scales can be captured, and the weights of each modality are dynamically adjusted through the self-attention mechanism, ensuring that the detection strategy can be flexibly adjusted according to the actual situation, and improving the precision and flexibility of defect recognition.

[0170] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A surface mount detection system in a semiconductor package, characterized in that: including, a data acquisition module that real - time collects temperature information, topography images, and reflected light images on the surface of semiconductor packages, and simultaneously collects historical data of semiconductor packages; a pre - processing module that pre - processes the collected temperature information, topography images, and reflected light images; a data fusion module that fuses the pre - processed information and images to form a multi - modal feature vector; a 3D reconstruction module that generates a 3D image of the semiconductor package through a 3D reconstruction algorithm and the multi - modal feature vector; a defect recognition module that establishes a defect recognition model based on the multi - modal feature vector and outputs a defect recognition result; a defect classification module that classifies the defect recognition results output by the defect recognition model to generate defect classification results; a defect localization module that marks the classified defect positions on the 3D image of the package and generates a preliminary detection report; an adaptive adjustment module that automatically adjusts the parameters of the detection device according to the preliminary detection report, and performs secondary detection on key areas to generate secondary detection results; a feedback analysis module that re - inputs the detection data after adaptive feedback adjustment into the defect recognition model for analysis and confirmation, and finally generates a complete detection report.

2. The surface mount detection system in the semiconductor package according to claim 1, wherein: Real - time collect the information on the surface of the semiconductor package and the reflected light images of the semiconductor package surface at different polarization angles. The specific steps are as follows: Real - time record the temperature change on the surface of the semiconductor package through a thermal imaging device, real - time collect the high - definition image of the package surface through an optical detection device, and collect the reflected light images at different polarization angles through a polarization detection device.

3. The surface mount detection system in the semiconductor package according to claim 2, wherein: Pre - process the collected information and images. The specific steps are as follows: Denoise the collected temperature change information, high - definition image of the package surface, and reflected light images; Align, register, and enhance the pictures from different devices; Convert the format of the data and images.

4. The surface mount detection system in the semiconductor package according to claim 3, wherein: Fuse the pre - processed information and images to form a multi - modal feature vector. The specific steps are as follows: Use the Sobel operator to extract the edge features of the optical image; Use the Gabor filter to extract the texture features related to the polarization angle in the polarization image; Extract the temperature gradient features of the temperature on the surface of the semiconductor package; Fuse the edge features, texture features, and temperature gradient features to form a multi - modal feature vector.

5. The surface mount detection system in the semiconductor package according to claim 4, characterized in that: Generate a 3D image of the semiconductor package through a 3D reconstruction algorithm and the multi - modal data set. The specific steps are as follows: Map the fused multi - modal feature vector into a 3D voxel grid. The multi - modal feature vector is a voxel, and each voxel represents a small cube in 3D space; Data of different modalities are divided into individual voxels in 3D space, and each voxel contains comprehensive physical information about its coordinate position in 3D space; Apply volume rendering technology to assign color and transparency to each voxel according to the attribute value of the voxel to generate a 3D visualization image of the package.

6. The surface mount detection system in the semiconductor package according to claim 5, wherein: Establish a defect recognition model based on the multi - modal feature vector and output a defect recognition result. The specific steps are as follows: Extract the multi - modal data of normal sample data from the historical data, and use a convolutional neural network to extract features from the data of each normal sample modality; In the feature extraction stage of the convolutional neural network, multi-scale convolution is added to capture features at different scales through convolutional kernels of different scales, and the features at different scales are fused; Aggregate the multi-modal features of all normal samples to obtain a set of benchmark feature vectors; Perform statistical analysis on each dimension of the multi-modal feature vectors of normal samples to calculate the mean and standard deviation of each dimension; The real-time data is extracted using the same method as the normal samples in the historical data. During the feature extraction process, multi-scale convolution is used, and the self-attention mechanism is used during the fusion process to dynamically adjust the weights according to the importance of each modality to generate the final weighted fusion feature vector; Calculate the standardized deviation of the real-time data by comparing the weighted fusion feature vector with the benchmark feature vector; Aggregate the standard deviations of each real-time data through the sum of absolute deviations to generate the overall deviation value at this point; Based on the historical data, use cross-validation to set the deviation threshold; When the overall deviation value is greater than the deviation threshold, it is determined that the real-time data at this position has anomalies and defects.

7. The surface mount detection system in the semiconductor package according to claim 6, wherein: Classify the recognition results output by the defect recognition model to generate defect classification results, which specifically include the following steps. Form a defect feature library by aggregating the feature sets of all known defect types; Extract the features of the anomalies and defects output by the defect recognition model to form the feature vector of this anomaly and defect; Calculate the similarity between the feature vector of the anomaly and defect and each known defect in the defect feature library through the Euclidean distance. Its expression is: Among them, dist(F, F a ) represents the similarity between the new defect F and the known defect F a . F represents the feature vector of the new defect, and F a represents the feature vector of the a-th known defect in the defect feature library. n represents the total dimension of the feature vector, F k represents the k-th feature component of the new defect feature vector F, and F a,k represents the k-th feature component of the a-th known defect F a in the defect feature library. k represents the summation index; According to the similarity between the new defect F and the known defect F a the category of the new defect is determined using the nearest neighbor method; Output the defect classification result after determining the category of the new defect.

8. The surface mount detection system in the semiconductor package according to claim 7, wherein: Mark the classified defect positions on the encapsulated three-dimensional image and generate a preliminary detection report, which specifically includes the following steps. Mark the positions of the defects on the three-dimensional image in a visual way, and highlight the defect positions in different colors, shapes and marks in the three-dimensional encapsulated image; And generate a preliminary detection report including package identification information, defect statistics, defect details and images and annotations.

9. The surface mount detection system in the semiconductor package according to claim 8, characterized in that: Automatically adjust the parameters of the detection device according to the preliminary detection report, and perform secondary detection on the key areas to generate secondary detection results, which specifically include the following steps. Automatically adjust the resolution, detection frequency, light source and angle, and detection method of the detection device according to the preliminary detection report; Define the positions of the defects marked on the three-dimensional image as key areas; Perform secondary detection on the key areas according to the device with adjusted parameters and generate secondary detection results.

10. The surface mount detection system in the semiconductor package as described in claim 9, characterized in that: Input the detection data after adaptive feedback adjustment into the machine learning model for analysis and confirmation again, and finally generate a complete detection report, which specifically includes the following steps. Input the automatically adjusted secondary detection results into the defect recognition model, and finally confirm all detected defects, and update the defect classification results at the same time; After feedback analysis and confirmation, generate a complete detection report including all detected defect information, detailed analysis of the defects, and defect classification results.

Images