Chip internal solder joint defect intelligent identification method based on X-ray image

Abstract

The application discloses a kind of chip internal solder joint defect intelligent identification method based on X-ray image, it is related to X-ray image processing and intelligent defect identification technical field, for solving the problem that chip internal solder joint defect is difficult to accurately locate, classification and credible determination;Including obtaining the X-ray image of the chip solder joint area to be detected, the image is corrected, normalized, noise suppression and gray scale enhancement;According to the arrangement information of solder joint, the position of solder joint area is determined and corrected, and the solder joint sample image is cut out;Through solder joint defect identification network, generate defect probability graph and solder joint category probability, and combine defect candidate area, boundary refinement, contour merging and confidence value calculation, output solder joint category, defect position, defect parameter and identification confidence degree, realize the intelligent identification of chip internal solder joint defect.

Description

Technical Field

[0001] This invention relates to the field of X-ray image processing and intelligent defect recognition technology, and more specifically, to a method for intelligent recognition of internal solder joint defects in chips based on X-ray images. Background Technology

[0002] As chip packaging density continues to increase, solder joints are typically located inside the package, making it difficult for visual inspection to directly detect defects such as voids, bridging, missing solder, and cold solder joints. In existing inspection methods, X-ray imaging can obtain grayscale images of internal solder joints without damaging the chip structure, thus becoming an important technological starting point for internal chip solder joint inspection.

[0003] However, the projections of solder joints, substrates, electrical connection structures, and packaging materials in X-ray images are prone to overlapping, and the image grayscale is significantly affected by exposure conditions, detector response, chip placement deviations, and noise. Traditional manual interpretation or single threshold segmentation methods often rely on experience and struggle to reliably distinguish between actual solder joint defects and abnormal grayscale caused by structural projections. This is especially problematic when multiple types of defects coexist, defect boundaries are blurred, or there is connected grayscale between adjacent solder joints, leading to missed detections, false detections, or inaccurate defect locations.

[0004] To address the above problems, this invention proposes a solution. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an intelligent identification method for internal solder joint defects of chips based on X-ray images, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: In a preferred embodiment, it includes: The original X-ray image of the solder joint area of ​​the chip to be inspected is acquired. The target X-ray image is constructed based on the dark field image and the flat field image. The actual location of the solder joint area is determined based on the solder joint layout information. The corrected position of the solder joint area is determined based on the target X-ray image and the actual position of the solder joint area, and a sample image of the solder joint is obtained according to the corrected position of the solder joint area. Based on the solder joint sample image, a defect candidate region is constructed, and the solder joint sample image is input into the solder joint defect recognition network to obtain a defect probability map, solder joint category probability, and initial solder joint defect recognition result. Based on the initial solder joint defect identification results, the effective defect area is determined, and a co-location reference area is established based on the solder joint layout information. The co-location reference area, common grayscale reference map, and individual residual map are combined to distinguish the common grayscale components corresponding to the internal structure projection and the individual abnormal components corresponding to the current effective defect area. Then, the defect confidence value is determined based on the defect probability map, solder joint category probability, internal structure projection repetition state value, individual abnormal retention area, and multiple state values ​​to generate the solder joint defect identification result.

[0007] In a preferred embodiment, the original X-ray image of the solder joint area of ​​the chip under inspection is acquired, and the dark field image and flat field image are read. The three images are matched pixel by pixel according to the same pixel coordinates. The target X-ray image is constructed based on the dark field subtraction and flat field normalization relationship. Then, the solder joint layout information in the packaging design file, inspection process file, or solder joint array coordinate table is read to extract the solder joint arrangement direction, row and column number, spacing between adjacent solder joints, design size of a single solder joint, and design coordinates of the solder joint center. The theoretical center position and theoretical boundary range of each solder joint area in the target X-ray image are calculated by combining the imaging magnification. Then, based on the actual center position and direction of the outer edge of the chip under inspection or the outer edge of the solder joint array, the lateral position correction, longitudinal position correction, and angle correction are calculated. The coordinate transformation of the theoretical solder joint area is performed to determine the actual position of the solder joint area.

[0008] In a preferred embodiment, a local image is cropped at the actual location of the solder joint area, and solder grayscale regions that are distinct from the substrate background and continuously distributed are identified. The solder grayscale regions are then correlated with the design coverage area, internal abnormal grayscale, boundary continuity, connectivity between adjacent solder joints, and connection position relationship. Normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects are labeled. Among them, void defects correspond to local abnormal grayscale regions inside the solder joint, bridging defects correspond to continuous solder grayscale structures between adjacent solder joints, missing solder defects correspond to abnormal regions within the design coverage area that are not covered by solder grayscale regions, and cold solder joint defects correspond to discontinuous regions between solder grayscale regions and connection positions. For the above defect categories, defect outlines are drawn along the boundaries of the corresponding abnormal regions, and the target X-ray image, the actual location of the solder joint area, the solder joint category, and the defect location are written into the solder joint defect sample set.

[0009] In a preferred embodiment, the target X-ray image and the actual location of the solder joint area are read, and the pixel cropping range of the solder joint area is determined by combining the solder joint design size and imaging magnification. Then, noise suppression, local contrast enhancement, and edge grayscale enhancement are performed on the target X-ray image to form an enhanced target X-ray image. For each solder joint area, a temporary local image containing the solder joint and the adjacent substrate background is first cropped from the actual location of the solder joint area. Then, the solder grayscale judgment threshold is determined in the temporary local image based on the average grayscale value of the background area, the average grayscale value of the solder joint center area, and the grayscale display method. The solder grayscale area connected to the solder joint center area is extracted. Then, the outer boundary of the solder grayscale area is extracted, and the broken boundary and isolated boundary segments are corrected. The correction center coordinates and correction boundary range are calculated based on the corrected outer boundary to determine the correction position of the solder joint area. Using the correction center coordinates in the solder joint area correction position as the clipping center, and the correction boundary range and sample edge area as the clipping boundary, an initial solder joint sample image containing the complete solder joint area and the local substrate background is clipped. For solder joints that have been marked as bridging defects, the clipping boundary also includes the adjacent solder joint area connected to the current solder joint to form a continuous solder grayscale structure. Then, the initial solder joint sample image is sized and grayscale unified, and the defect contour points under the target X-ray image coordinates are converted into solder joint sample image coordinates according to the clipping boundary, scaling ratio and fill offset. The solder joint sample image, solder joint area correction position, solder joint category and converted defect position are written into the solder joint sample record.

[0010] In a preferred embodiment, the solder joint sample image, the corrected location of the solder joint area, the solder joint category, and the converted defect location are read. First, the solder grayscale area and the substrate background area are divided in the solder joint sample image, and the corresponding grayscale mean and grayscale standard deviation are calculated respectively. Then, defect candidate areas are constructed according to the grayscale and positional relationship of different defects. Among them, voids correspond to continuous pixel areas inside the solder grayscale area that deviate from the solder grayscale mean; bridging corresponds to continuous pixel areas within the interval between adjacent solder joints that simultaneously contact the solder grayscale areas on both sides; missing solder corresponds to pixel areas within the solder joint design coverage area that are not covered by the solder grayscale area and whose area exceeds the allowable deviation; and cold solder corresponds to grayscale discontinuity areas where the width between the solder grayscale area and the connection position exceeds the allowable connection gap. Then, the above abnormal grayscale areas are merged or retained according to the shortest pixel distance, and the region boundary, region area, region center position, and the corresponding solder joint sample image of the defect candidate area are recorded. The solder joint defect recognition network takes solder joint sample images as input and uses convolution processing with different receptive ranges to extract local edges, small-sized abnormal grayscale, solder coverage, connectivity of adjacent solder joints, and overall grayscale distribution features. It then generates a defect probability map corresponding to the pixel positions of the solder joint sample image on the feature map after channel merging, as well as solder joint category probabilities for normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects. During recognition, the defect probability map is converted into a defect recognition region according to the defect probability threshold determined by the verification samples. The recognized defect region is then compared with the defect candidate region for overlap judgment, and either merged or retained as the initial defect region. Simultaneously, the initial solder joint category is determined by combining the solder joint category probability and the relationship between the initial defect region and the solder grayscale region, the interval between adjacent solder joints, the designed coverage of the solder joint, and the connection position. This generates an initial solder joint defect recognition result containing the solder joint sample image, the chip number to be inspected, the corrected position of the solder joint region, the initial solder joint category, the initial defect region, the defect probability map, and the solder joint category probability.

[0011] In a preferred embodiment, the initial solder joint defect identification results are read, including solder joint sample images, solder joint region correction positions, initial solder joint categories, initial defect regions, defect probability maps, and solder joint category probabilities. Subsequently, a boundary search region is set around the outer boundary of the initial defect region, and the grayscale change intensity is calculated along the outer normal direction of the initial defect region. The position with the largest grayscale change intensity is taken as the correction boundary point to form an initial correction contour. For the break positions in the initial correction contour, contour points are supplemented according to the shortest pixel path. For protruding contour segments formed by isolated noise, they are deleted according to the connection length, contour length, and minimum effective defect area to obtain the corrected defect contour. The defect contours are defined according to the location relationship of the defect categories: void defects are limited to the inside of the solder gray area, bridging defects are limited to the outer boundary of the continuous solder gray structure between adjacent solder joints, missing solder defects are limited to the position not covered by the solder gray area within the design coverage of the solder joint, and cold solder joint defects are limited to the discontinuous area between the solder gray area and the connection position.

[0012] In a preferred embodiment, overlapping pixels in the same solder joint sample image, or the shortest pixel distance being less than the boundary breakage distance and corresponding to the same initial solder joint category, are merged. The area enclosed by the merged contours is recorded as the effective defect area. Then, effective defect areas that do not meet the conditions are screened out based on the minimum effective defect area and the degree of grayscale anomaly. The final solder joint category is determined according to the relationship between the effective defect area and the solder grayscale area, the interval between adjacent solder joints, the designed coverage area of ​​the solder joint, and the connection position. The defect area, defect center position, defect outer range, and defect area ratio of the effective defect area are calculated. The defect area ratio is selected as the solder grayscale area, the designed coverage area of ​​the solder joint, the area of ​​the interval between adjacent solder joints, or the area of ​​the designed contact area as the solder joint reference area according to different defect categories.

[0013] In a preferred embodiment, the sources of grayscale anomalies in the target X-ray image are classified into three cases: defects in the solder joint itself, projection of internal structure, and overlap of the two. Grayscale components that appear repeatedly at the same relative position in multiple adjacent solder joint regions are defined as common grayscale components, and grayscale components that appear only in the current solder joint region and correspond to the solder grayscale region, connection position, or adjacent solder joint interval region are defined as individual anomalous components. The system reads information such as the effective defect area, defect probability map, solder joint category probability, solder joint area correction position, solder grayscale area, substrate background area, connection position, and solder joint arrangement. It establishes relative position description data based on the positional offset and circumscribed dimension of the current effective defect area relative to the solder joint area correction center or bridging interval center. Then, it selects reference solder joint areas or reference interval areas along the same solder joint array direction, in the same row or column, with the same design size and within the effective imaging range. The relative position description data is mapped to each reference object to obtain a co-located reference area with the same relative position and size as the current effective defect area. Subsequently, according to the final solder joint category, it extracts the internal solder anomaly area corresponding to voids, the cross-interval connected area corresponding to bridging, the uncovered area within the design coverage range corresponding to missing solder, and the discontinuous connection position area corresponding to cold solder joints from the co-located reference area. It calculates the internal structure projection repetition state value based on the number of co-located grayscale anomaly areas, the average degree of relative position overlap, and the average degree of grayscale similarity, which characterizes the degree to which the current effective defect area recurs along the solder joint array direction.

[0014] In a preferred embodiment, when available co-location reference regions exist, a common grayscale reference map is further established using the median values ​​of the grayscale values ​​of pixels at the same relative positions in multiple co-location reference regions. Then, the current effective defect region is compared pixel-by-pixel with the common grayscale reference map to generate an individual residual map representing the degree of grayscale deviation of the current region relative to the array's repeated projection. The residual judgment threshold is jointly determined by the reference residual of each co-location reference region relative to the common grayscale reference map, the threshold increment corresponding to the internal structure projection repetition state value, and the threshold reduction corresponding to the category position conformity state value. The category position conformity state value is determined based on whether the void is located inside the solder and is not connected. The calculations include: whether the substrate background, whether the bridging spans the interval between adjacent solder joints and connects the solder on both sides, whether the missing solder is within the design coverage area and not covered by solder, and whether the cold solder joint is located in the discontinuous area between the solder grayscale area and the connection position. Then, the continuous pixel area in the individual residual map that is greater than or equal to the residual judgment threshold is determined as the individual abnormality retention area. If there is no individual abnormality retention area and the internal structure projection repetition state value is higher than the projection repetition judgment threshold, the current effective defect area is marked as the projection influence area. If there is an individual abnormality retention area, the area is used as the confidence evaluation area; otherwise, the current effective defect area is used as the confidence evaluation area.

[0015] In a preferred embodiment, local defect probability state values, category separation state values, category location conformity state values, boundary stability state values, and grayscale individual state values ​​are calculated within the confidence evaluation region. The minimum value among these state values ​​is taken as the basic confidence state value. The basic confidence state value is then corrected by combining the internal structure projection repetition state value and the proportion of individual abnormal retention areas. Finally, the defect confidence value is determined. For cases where the final solder joint category is a normal solder joint, the maximum value of the defect probability map, the normal solder joint category probability, the boundary stability state of the solder grayscale region, and the individual residual state within the solder are also read. Continuous abnormal regions within the solder grayscale region that exceed the residual judgment threshold are set as areas to be reviewed. Finally, the effective defect region is reverse-converted from the solder joint sample image coordinates back to the target X-ray image coordinates. The chip number to be inspected, the target X-ray image name, the corrected position of the solder joint region, the final solder joint category, the effective defect region, the defect area, the defect center position, the defect outer range, the defect area proportion, and the defect confidence value are written into the solder joint defect identification result.

[0016] The technical effects and advantages of the intelligent identification method for internal solder joint defects of chips based on X-ray images of this invention are as follows: This invention reduces subjective errors caused by manual interpretation by forming a continuous processing flow for X-ray image acquisition, image correction, solder joint location, sample cropping, defect identification, and result evaluation of internal chip solder joints. Through grayscale reference correction, noise suppression, and local enhancement, the distinguishability between solder joint areas and abnormal grayscale areas can be improved; by combining solder joint layout information and boundary refinement, the accuracy of defect location output can be enhanced; by jointly determining the defect probability map, solder joint category probability, candidate area, and effective defect area, both defect classification and contour localization can be considered; and by calculating defect confidence values, the risk of misjudgment caused by factors such as structural projection and repetitive grayscale anomalies can be reduced, making the detection results more suitable for batch chip quality screening, verification and location, and defect tracing. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the dual-output structure of the solder joint defect identification network in the intelligent identification method for chip internal solder joint defects based on X-ray images according to the present invention.

[0018] Figure 2 This is a schematic diagram of X-ray imaging and grayscale reference correction for an intelligent identification method for internal solder joint defects of a chip based on X-ray images, according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In this embodiment, the present invention discloses an intelligent identification method for internal solder joint defects of a chip based on X-ray images, comprising: Step 1: Acquire the original X-ray image of the solder joint area of ​​the chip to be inspected, construct the target X-ray image based on the dark field image and the flat field image, and determine the actual location of the solder joint area based on the solder joint layout information; Step 2: Determine the corrected position of the solder joint area based on the target X-ray image and the actual position of the solder joint area, and obtain a sample image of the solder joint according to the corrected position of the solder joint area; Step 3: Construct a defect candidate region based on the solder joint sample image, and input the solder joint sample image into the solder joint defect recognition network to obtain a defect probability map, solder joint category probability, and initial solder joint defect recognition result; Step 4: Determine the effective defect area based on the initial solder joint defect identification results, and establish a co-position reference area based on the solder joint layout information. Combine the co-position reference area, common grayscale benchmark map, and individual residual map to distinguish the common grayscale components corresponding to the internal structure projection and the individual abnormal components corresponding to the current effective defect area. Then, determine the defect confidence value based on the defect probability map, solder joint category probability, internal structure projection repetition state value, individual abnormal retention area, and multiple state values ​​to generate the solder joint defect identification result.

[0021] In step one, the chip to be tested is designated as the chip under test, and the area inside the chip under test formed by solder and serving as the electrical connection is designated as the solder joint area. X-ray imaging is performed on the solder joint area of ​​the chip under test to obtain the original X-ray image. Grayscale calibration is then performed on the original X-ray image to obtain the target X-ray image, such as... Figure 2 As shown; the actual location of the solder joint area is determined based on the solder joint layout information of the chip under inspection and the solder joint imaging position in the target X-ray image; the target X-ray image, the actual location of the solder joint area, the solder joint type and the defect location are written into the solder joint defect sample set.

[0022] Place the chip under test on the X-ray imaging stage, ensuring that the chip's package reference edge or package positioning point is aligned with the positioning direction of the imaging stage. The package reference edge is determined by the straight edge of the chip's package outline, while the package positioning point is determined by the chip's package corner, carrier board positioning holes, silkscreen positioning marks, or fixed positioning marks on the inspection fixture. After placement, ensure that the solder joint area of ​​the chip is within the X-ray irradiation range and the detection imaging field of view.

[0023] The chip under test is irradiated using X-ray fluoroscopic imaging, allowing X-rays to pass sequentially through the chip's packaging material, solder, and substrate material before reaching a digital detector. The digital detector receives the intensity distribution of the X-rays after they have passed through the chip and converts this distribution into a two-dimensional grayscale image, obtaining the original X-ray image. The pixel positions in the original X-ray image correspond to the spatial positions of the chip under test within the imaging field of view, and the grayscale value of each pixel in the original X-ray image is determined by the degree of attenuation of the X-rays at that pixel position after passing through the chip.

[0024] When acquiring raw X-ray images, the batch information, chip number, imaging conditions, and image coordinate information are recorded simultaneously. The batch information is generated by the current inspection task; the chip number is obtained from the identification information on the chip surface, the carrier number, or a pre-assigned number in the inspection task; the imaging conditions include the exposure voltage, exposure current, exposure time, magnification, distance from the focal point to the chip, distance from the chip to the digital detector, and imaging angle used during X-ray imaging; the image coordinate information uses the top-left pixel of the raw X-ray image as the origin, the horizontal direction as the horizontal coordinate direction, the vertical direction as the vertical coordinate direction, and pixels as the coordinate unit.

[0025] Before acquiring the original X-ray image of the chip under inspection, or within the same inspection batch, dark-field images and flat-field images are acquired. A dark-field image is a grayscale image obtained when the digital detector maintains the same acquisition settings and receives no effective X-ray irradiation. The pixel grayscale value of the dark-field image represents the background grayscale generated by the digital detector at the corresponding pixel location. A flat-field image is a grayscale image obtained under the same conditions as the original X-ray image, without the chip under inspection being placed, and with the same exposure voltage, exposure current, exposure time, magnification, and imaging angle. The pixel grayscale value of the flat-field image represents the incident X-ray intensity received at the corresponding pixel location and the digital detector response.

[0026] The original X-ray image, dark-field image, and flat-field image are read, and pixel-by-pixel mapping is performed on the three images according to the same pixel coordinates. If the pixel size or image size of the original X-ray image, dark-field image, and flat-field image are different, the three images are first resized according to the pixel arrangement of the digital detector, so that pixels at the same coordinate position in the three images correspond to the same detection position on the digital detector. After pixel mapping is completed, grayscale reference correction is performed on the original X-ray image to obtain the target X-ray image.

[0027] Gray-scale reference correction is performed according to the following formula: ; Where I(x,y) represents the gray value of the target X-ray image at pixel position (x,y); R(x,y) represents the gray value of the original X-ray image at pixel position (x,y); D(x,y) represents the gray value of the dark field image at pixel position (x,y); and F(x,y) represents the gray value of the flat field image at pixel position (x,y). This prevents positive numbers with a denominator of zero. The effective gray level difference is defined as the smallest non-zero gray level change that the digital detector can distinguish, or as a positive number less than the minimum effective gray level difference. The effective gray level difference is the difference between the gray level value of a pixel in a flat-field image and the gray level value of the same pixel location in a dark-field image.

[0028] After grayscale reference correction is completed, the target X-ray image undergoes grayscale range normalization. During grayscale range normalization, the minimum and maximum grayscale values ​​of all effective pixels in the target X-ray image are first calculated, and then the grayscale value of each effective pixel in the target X-ray image is mapped to a uniform grayscale range. Effective pixels are those pixels in the target X-ray image that are located within the effective imaging area of ​​the digital detector and are not overexposed or underexposed; overexposed pixels are determined by pixels reaching the upper grayscale limit of the digital detector, and underexposed pixels are determined by pixels reaching the lower grayscale limit of the digital detector. The normalized image is still recorded as the target X-ray image.

[0029] The solder joint layout information of the chip under inspection is read, and the theoretical position of each solder joint region in the target X-ray image is determined based on this information. The solder joint layout information is obtained from the chip's package design file, inspection process file, or solder joint array coordinate table. This information includes the package reference point position, solder joint arrangement direction, solder joint row and column numbers, spacing between adjacent solder joints, individual solder joint design dimensions, and design coordinates of each solder joint center relative to the package reference point. Based on the imaging magnification, the design coordinates of the solder joint center relative to the package reference point are converted into pixel distances in the target X-ray image. This pixel distance is then superimposed onto the image coordinates of the package reference point in the target X-ray image to obtain the theoretical center position of each solder joint region. Finally, based on the individual solder joint design dimensions and the imaging magnification, the theoretical boundary range of each solder joint region in the target X-ray image is determined.

[0030] If the actual imaging position of the chip under test in the target X-ray image deviates from its theoretical position, the theoretical position of the solder joint area is corrected. During correction, an image region containing the outer edge of the chip under test or the outer edge of the solder joint array is first selected in the target X-ray image. The gray-level gradient of this image region is calculated to obtain the intensity of gray-level change at each pixel location. Then, the average gray-level gradient of the background region and the average gray-level gradient of the solder joint array region are calculated. The background region is determined by the imaging region located outside the chip under test in the target X-ray image, and the solder joint array region is determined by the theoretical solder joint coverage area corresponding to the solder joint arrangement information. The median value between the average gray-level gradient of the background region and the average gray-level gradient of the solder joint array region is used as the edge detection threshold, and consecutive pixels with a gray-level change intensity greater than the edge detection threshold are identified as edge pixels.

[0031] Next, after obtaining the outer edge of the chip under test or the outer edge of the solder joint array from the edge pixels, the center position and direction of the actual outer edge are calculated. The center position of the actual outer edge is obtained from the mean of the horizontal and vertical coordinates of the edge pixels; the direction of the actual outer edge is determined by the straight line direction fitted by the edge pixels. The center position of the actual outer edge is compared with the theoretical center position of the outer edge corresponding to the solder joint layout information to obtain the horizontal and vertical position corrections; the direction of the actual outer edge is compared with the theoretical direction of the outer edge corresponding to the solder joint layout information to obtain the angle correction. Based on the horizontal, vertical, and angle corrections, the theoretical center position and theoretical boundary range of each solder joint area are transformed to obtain the actual position of each solder joint area.

[0032] For the target X-ray images used to establish a solder joint defect sample set, the image region corresponding to the actual location of each solder joint area is labeled with a solder joint category. Solder joint categories include normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects. A normal solder joint indicates that the solder grayscale distribution within the solder joint area is continuous, and the solder grayscale coverage matches the corresponding solder joint design size. A void defect indicates that there is a local abnormal grayscale region formed by internal cavities of the solder within the solder joint area. This local abnormal grayscale region is located within the solder joint boundary and is not connected to the background area outside the solder joint. A bridging defect indicates that there is a continuous solder grayscale structure between adjacent solder joint areas, and this continuous solder grayscale structure spans the interval between adjacent solder joint areas. A missing solder defect indicates that the identifiable solder grayscale coverage within the solder joint area is smaller than the design coverage area corresponding to that solder joint area. A cold solder joint defect indicates that there is a discontinuous region between the solder grayscale distribution and the connection position within the solder joint area, or an abnormal interval appears between the solder joint edge and the connection position.

[0033] When labeling solder joints, a local image is first cropped from the actual location of the solder joint area, and the grayscale region of the solder is identified within this local image. The grayscale region of the solder consists of pixels in the local image whose grayscale values ​​differ from the surrounding substrate background and are continuously distributed. The grayscale region of the solder is compared with the designed coverage area corresponding to the actual location of the solder joint area, and the solder joint category is determined based on the internal abnormal grayscale, boundary continuity, connectivity between adjacent solder joints, and coverage area of ​​the grayscale region. Manual review results, historical inspection conclusions, or cross-sectional observation conclusions are used to confirm the solder joint category. Manual review results are obtained by inspectors interpreting the solder joint area in the target X-ray image; historical inspection conclusions are obtained from existing inspection records of the same chip under inspection or chips of the same type; and cross-sectional observation conclusions are obtained by observing the sample solder joints through cross-section.

[0034] When marking the location of defects in the solder joint area, if the solder joint is classified as a void defect, the defect outline is drawn along the outer boundary of the local abnormal grayscale area inside the solder joint area; if the solder joint is classified as a bridging defect, the defect outline is drawn along the outer boundary of the continuous solder grayscale structure between adjacent solder joints; if the solder joint is classified as a missing solder defect, the defect outline is drawn along the boundary of the abnormal area not covered by the solder grayscale area within the designed coverage area of ​​the solder joint; if the solder joint is classified as a cold solder joint defect, the defect outline is drawn along the boundary of the discontinuous area between the solder grayscale area and the connection position. The defect outline is represented by the pixel coordinates in the target X-ray image, and each outline point on the defect outline has horizontal and vertical coordinates.

[0035] When the defect boundary exhibits a grayscale transition in the target X-ray image, the intensity of the grayscale change is first calculated along the direction perpendicular to the defect boundary, and the location with the largest grayscale change intensity is determined as the defect boundary position. When multiple non-connected abnormal grayscale regions exist within the same solder joint area, corresponding defect contours are generated for each region, and these multiple defect contours are collectively recorded in the defect position record corresponding to that solder joint area. When the solder joint type is a normal solder joint, the defect position record is left blank, and the solder joint type of that solder joint area is recorded as a normal solder joint.

[0036] After completing the labeling of solder joint categories and defect locations, the target X-ray image, the chip number to be inspected, the actual location of the solder joint area, the solder joint category, and the defect location are written into the solder joint defect sample set. The target X-ray image is obtained from the original X-ray image after grayscale reference correction and grayscale range normalization; the chip number to be inspected is obtained from the inspection batch information, the surface identification information of the chip to be inspected, or the carrier number; the actual location of the solder joint area is obtained from the solder joint layout information, the horizontal position correction amount, the vertical position correction amount, and the angle correction amount; the solder joint category is determined by the solder grayscale area in the target X-ray image and combined with the results of manual review, historical inspection conclusions, or cross-sectional observation conclusions; the defect location is determined by the boundary pixel coordinates of the abnormal grayscale area in the target X-ray image.

[0037] When augmenting an annotated target X-ray image, the target X-ray image, the actual location of the weld joint area, and the defect location are treated as a single transformation group. During rotation, the target X-ray image is rotated by a set rotation angle, and the actual location of the weld joint area and the coordinates of the defect contour points are transformed by the same rotation angle. During mirroring, the target X-ray image is flipped horizontally or vertically, and the actual location of the weld joint area and the coordinates of the defect contour points are transformed by the same mirror direction. During translation, the target X-ray image is moved by set horizontal and vertical translation amounts, and the actual location of the weld joint area and the coordinates of the defect contour points are moved by the same translation amount. During scaling, the target X-ray image size is changed according to a set scaling ratio, and the actual location of the weld joint area and the coordinates of the defect contour points are transformed by the same scaling ratio. When performing brightness or contrast perturbation, only the pixel grayscale values ​​of the target X-ray image are changed; the actual location of the weld joint area, the weld joint type, and the defect location remain unchanged.

[0038] The rotation angle, lateral translation, longitudinal translation, and scaling ratio used in image augmentation are determined based on the allowable placement deviation range of the chip under test on the X-ray imaging stage. This allowable placement deviation range is determined by the positioning accuracy of the inspection fixture, the repeatability accuracy of the imaging stage, or the attitude deviation range given in the inspection process document. The image obtained after image augmentation is recorded as the augmented target X-ray image. The augmented target X-ray image, the actual location of the augmented solder joint area, the location of the augmented defect, the original solder joint type, and the augmentation method are written into the solder joint defect sample set.

[0039] In step two, the target X-ray image and the actual location of the solder joint area obtained in step one are read. The target X-ray image is subjected to noise suppression and grayscale enhancement processing to obtain an enhanced target X-ray image. The actual location of the solder joint area in the enhanced target X-ray image is refined to obtain the corrected location of the solder joint area. A local image containing a single solder joint area is cropped according to the corrected location of the solder joint area, and the cropped local image is subjected to size unification and grayscale unification processing to obtain a solder joint sample image.

[0040] When reading the target X-ray image, the actual position of the solder joint area, the designed size of the solder joint, and the imaging magnification are simultaneously read. The target X-ray image is obtained from the original X-ray image in step one after grayscale reference correction and grayscale range normalization. The actual position of the solder joint area is obtained from the solder joint layout information, lateral position correction, longitudinal position correction, and angle correction in step one. The designed size of the solder joint is obtained from the chip packaging design file, inspection process file, or solder joint array coordinate table. The imaging magnification is obtained from the imaging conditions corresponding to the target X-ray image. Based on the designed size of the solder joint and the imaging magnification, the actual size of the solder joint area in the chip under inspection is converted into the pixel size in the target X-ray image, and this pixel size is used as the basis for determining the trimming range of the solder joint area.

[0041] When performing noise suppression processing on a target X-ray image, the type of noise in the target X-ray image is first determined. If there are isolated bright spots or isolated dark spots in the target X-ray image, median filtering is used. Median filtering involves selecting neighboring pixels centered on the current pixel, sorting the gray values ​​of the neighboring pixels by size, and replacing the current pixel's gray value with the median gray value of the sorted pixels. The size of the neighborhood used for median filtering is determined based on the pixel range of isolated noise points in the target X-ray image, which is determined by pixel regions in the target X-ray image that clearly deviate from the continuous gray-level variation relationship between adjacent pixels. If there is continuous granular noise in the target X-ray image, mean smoothing is used. Mean smoothing involves selecting neighboring pixels centered on the current pixel, calculating the average gray value of the neighboring pixels, and replacing the current pixel's gray value with this average gray value. The size of the neighborhood used for mean smoothing is determined based on the distribution density of granular noise in the target X-ray image, which is determined by the number of pixels within a unit image region whose gray-level fluctuations exceed the background gray-level fluctuation range.

[0042] After noise suppression, the target X-ray image undergoes grayscale enhancement processing to obtain an enhanced target X-ray image. Grayscale enhancement processing includes local contrast enhancement and edge grayscale enhancement. Local contrast enhancement involves dividing the target X-ray image into multiple adjacent image sub-regions, statistically analyzing the pixel grayscale distribution within each sub-region, and redistributing pixel grayscale values ​​based on this distribution. This increases the difference between the grayscale values ​​of holes, solder, and substrate background within the solder joint area. The size of the image sub-region is determined based on the pixel size calculated from the solder joint design size, ensuring that each sub-region covers at least one solder joint area or the main grayscale variation range of a solder joint area. If the local grayscale enhancement amplitude within a certain image sub-region exceeds a preset enhancement limit, the excess grayscale allocation is redistributed to other grayscale ranges within that sub-region. The preset enhancement limit is determined by the grayscale fluctuation range of the background region in the target X-ray image, which is obtained from the imaging area of ​​the target X-ray image that does not contain the chip under inspection.

[0043] Edge grayscale enhancement processing refers to calculating the pixel grayscale gradient in an image after local contrast enhancement processing, and increasing the intensity of grayscale changes at weld joint edges and defect edges based on the grayscale gradient. The grayscale gradient represents the magnitude of grayscale change between adjacent pixels, calculated from the grayscale difference between the current pixel and its horizontal and vertical adjacent pixels. For pixel locations where the grayscale gradient is greater than the background grayscale gradient, the grayscale difference between that pixel and its adjacent pixels is enhanced according to the magnitude of the grayscale gradient; for pixel locations where the grayscale gradient is not greater than the background grayscale gradient, its grayscale value is maintained or retained according to the noise-suppressed grayscale value. The background grayscale gradient is determined by the average grayscale gradient of the background region in the target X-ray image. The image after local contrast enhancement processing and edge grayscale enhancement processing is denoted as the enhanced target X-ray image.

[0044] After obtaining the enhanced target X-ray image, a temporary local image containing the corresponding solder joint area and its adjacent background is extracted from the enhanced target X-ray image based on the actual location of the solder joint area obtained in step one. The center position of the temporary local image is determined using the center coordinates of the solder joint in the actual location of the solder joint area, and the size of the temporary local image is determined based on the pixel size converted from the solder joint design size. The boundary of the temporary local image extends outward from the solder joint area by an edge margin, which is determined based on the spacing between adjacent solder joints and the solder joint design size, and the edge margin does not exceed half the interval between two adjacent solder joint areas. If the solder joint to be inspected is located at the edge of the image, causing the temporary local image to exceed the range of the enhanced target X-ray image, the excess part is filled with the average gray value of the background area in the target X-ray image.

[0045] In the temporary local image, the boundary of the solder joint area is refined. During boundary refinement, the solder grayscale region is first determined in the temporary local image. The solder grayscale region consists of pixels whose grayscale values ​​are distinct from the surrounding substrate background and are spatially continuous. To determine the solder grayscale region, the average grayscale value of the background region and the average grayscale value of the solder joint center region in the temporary local image are calculated first. The background region is obtained from the edge image region of the temporary local image that is far from the center of the solder joint and does not contain adjacent solder joints. The solder joint center region is obtained from the coordinates of the solder joint center in the actual location of the solder joint and its surrounding pixel region. Then, the grayscale boundary value between the average grayscale value of the background region and the average grayscale value of the solder joint center region is taken as the solder grayscale determination threshold. Pixels that meet the solder grayscale determination threshold and are connected to the solder joint center region are determined as the solder grayscale region. If the solder appears as high grayscale in the target X-ray image, pixels with grayscale values ​​greater than the solder grayscale threshold and connected to the center region of the solder joint are defined as solder grayscale regions. If the solder appears as low grayscale in the target X-ray image, pixels with grayscale values ​​less than the solder grayscale threshold and connected to the center region of the solder joint are defined as solder grayscale regions. Whether the solder appears as high or low grayscale in the target X-ray image is determined by the grayscale display mode of the target X-ray image; the grayscale display mode is determined by the output settings of the imaging device or the grayscale inversion marker in the image recording.

[0046] After determining the solder grayscale region, the outer boundary of the solder grayscale region is extracted. The outer boundary consists of pixels in the solder grayscale region that are adjacent to non-solder grayscale regions. When performing continuity correction on the outer boundary, if there is a break in the outer boundary caused by noise suppression or local grayscale changes, the boundary pixels are filled in between the two ends of the break using the shortest pixel path; if there is an isolated boundary segment that significantly protrudes beyond the solder joint design size range, the isolated boundary segment is deleted from the outer boundary. The isolated boundary segment refers to a boundary segment that is not continuously connected to the main body of the solder grayscale region and has an area smaller than the main body area of ​​the solder grayscale region. The main body of the solder grayscale region is determined by the largest continuous solder grayscale region in the temporary local image.

[0047] The corrected position of the solder joint area is calculated based on the corrected outer boundary. The corrected position includes the corrected center coordinates and the corrected boundary range. The corrected center coordinates are obtained from the mean values ​​of the horizontal and vertical coordinates of the area enclosed by the corrected outer boundary; the corrected boundary range is obtained from the minimum and maximum coordinates of the corrected outer boundary in the horizontal direction and the minimum and maximum coordinates in the vertical direction. If the difference between the corrected boundary range and the pixel size converted from the solder joint design size exceeds the allowable dimensional deviation range in the inspection process document, the corrected position of the solder joint area is determined using the center coordinates of the solder joint in its actual location and the pixel size converted from the solder joint design size. If the difference between the corrected boundary range and the pixel size converted from the solder joint design size does not exceed the allowable dimensional deviation range in the inspection process document, the corrected center coordinates and the corrected boundary range are used as the corrected position of the solder joint area.

[0048] After obtaining the corrected location of the solder joint area, a sample image of the solder joint is cropped from the enhanced target X-ray image. During cropping, the correction center coordinates in the corrected location of the solder joint area are used as the cropping center, and the sample edge area is expanded outward based on the correction boundary range to obtain the cropping boundary. The sample edge area is determined according to the solder joint design size and the spacing between adjacent solder joints, so that the cropping boundary includes the complete solder joint area and the local substrate background around the solder joint area. If the labeling result of the solder joint type as a bridging defect already exists in the solder joint defect sample set, the cropping boundary also includes the adjacent solder joint areas that are connected to the solder joint area by a continuous solder grayscale structure. The cropped local image is recorded as the initial solder joint sample image.

[0049] The initial solder joint sample image is resized to obtain a resized solder joint sample image. During resizing, the horizontal and vertical pixel counts of the initial solder joint sample image are first read, and then the initial solder joint sample image is scaled or padded according to a preset sample size. The preset sample size is determined based on the pixel size converted from the solder joint design size, the sample edge region, and the input size of the recognition algorithm. If both the horizontal and vertical pixel counts of the initial solder joint sample image are greater than the preset sample size, the initial solder joint sample image is scaled while maintaining the solder joint shape proportions. If the scaled image is still smaller than the preset sample size in either the horizontal or vertical direction, the insufficient portion is padded with the average gray value of the background region in the target X-ray image. During padded, the corrected position of the solder joint region is kept in the center region of the solder joint sample image.

[0050] Grayscale unification processing is performed on the solder joint sample images after size unification. During grayscale unification, the minimum and maximum grayscale values ​​of the effective pixels in the solder joint sample image are first counted, and then the grayscale values ​​of the effective pixels are mapped to a unified grayscale range. Effective pixels are pixels in the solder joint sample image originating from the target X-ray image; fill pixels are not included in the minimum and maximum grayscale value statistics. If overexposed or underexposed pixels exist in the solder joint sample image, they are excluded from the grayscale statistics; overexposed pixels are determined by pixels reaching the upper grayscale limit of the imaging device, and underexposed pixels are determined by pixels reaching the lower grayscale limit of the imaging device. After grayscale unification, the solder grayscale region, substrate background region, and abnormal grayscale region in the solder joint sample image are all at the same grayscale scale.

[0051] If a defect location exists within the weld joint defect sample set, the defect location is transformed from the target X-ray image coordinates to the weld joint sample image coordinates. During coordinate transformation, based on the coordinates of the upper left corner of the clipping boundary, the horizontal and vertical coordinates of the defect contour point in the target X-ray image are subtracted from the horizontal and vertical coordinates of the upper left corner of the clipping boundary, respectively, to obtain the coordinates of the defect contour point in the initial weld joint sample image. If the initial weld joint sample image has been scaled, the defect contour point coordinates are transformed according to the same scaling ratio. If the initial weld joint sample image has been filled, the defect contour point coordinates are translated according to the horizontal and vertical offsets corresponding to the filling position. The transformed defect location is saved together with the weld joint sample image.

[0052] After completing the above processing, the solder joint sample image, the chip number to be inspected, the corrected position of the solder joint area, the solder joint category, and the converted defect location are written into the solder joint sample record. The solder joint sample image is obtained by cropping the enhanced target X-ray image according to the corrected position of the solder joint area; the chip number to be inspected follows the chip number to be inspected corresponding to the target X-ray image; the corrected position of the solder joint area is calculated from the outer boundary of the solder grayscale area in the temporary local image; the solder joint category follows the solder joint category of the corresponding solder joint area in the solder joint defect sample set; the converted defect location is obtained by cropping, scaling, and filling the coordinates of the target X-ray image.

[0053] In step three, the solder joint sample image obtained in step two is read, and the solder grayscale region, substrate background region and abnormal grayscale region in the solder joint sample image are distinguished; a defect candidate region is generated based on the abnormal grayscale region; the solder joint sample image is input into the solder joint defect recognition network for defect feature learning to obtain a defect probability map and solder joint category probability; an initial solder joint defect recognition result is generated based on the defect probability map, defect candidate region and solder joint category probability.

[0054] The defect candidate region refers to a continuous pixel region in the solder joint sample image that may correspond to a solder joint defect. The defect probability map is a probability image with corresponding pixel positions in the solder joint sample image; the value of each pixel position in the defect probability map represents the probability that the pixel position belongs to a defect region. The solder joint category probability refers to the probability value of the solder joint sample image belonging to a normal solder joint, void defect, bridging defect, missing solder defect, and cold solder joint defect, respectively. The solder joint defect recognition network is an image recognition algorithm trained on a solder joint defect sample set; its input is the solder joint sample image, and its output is the defect probability map and the solder joint category probability, such as... Figure 1 As shown.

[0055] In one specific implementation, the solder joint defect recognition network includes an input layer, a multi-scale convolutional feature extraction layer, a feature channel merging layer, a defect probability map output branch, and a solder joint category probability output branch. The input layer receives solder joint sample images with uniform size and grayscale. The multi-scale convolutional feature extraction layer includes at least two convolutional processing paths with different receptive ranges. One convolutional processing path uses a smaller kernel to extract local edges and small-sized abnormal grayscale values, while the other convolutional processing path uses a larger kernel to extract solder coverage, connectivity between adjacent solder joints, and overall grayscale distribution features. The feature maps obtained from different convolutional processing paths are merged according to the channel direction in the feature channel merging layer to obtain a channel-merged feature map.

[0056] The defect probability map output branch performs pixel-level mapping on the channel-merged feature map, generating a defect probability map with corresponding pixel positions in the solder joint sample image. The value of each pixel in the defect probability map represents the probability that the pixel position belongs to a defect region. The solder joint category probability output branch performs global feature aggregation on the channel-merged feature map, obtaining the solder joint category probabilities corresponding to normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects. This global feature aggregation refers to averaging, maximizing, or weighting the channel-merged feature map within a spatial range to obtain a feature vector representing the defect category of the entire solder joint sample image.

[0057] When reading the solder joint sample image, the corresponding solder joint area correction position, solder joint category, and defect location are also read simultaneously. The solder joint sample image is obtained by cropping the enhanced target X-ray image from step two according to the solder joint area correction position; the solder joint area correction position is calculated from the outer boundary of the solder grayscale area in step two; the solder joint category is determined by the manual review results, historical inspection conclusions, or cross-sectional observation conclusions from step one; the defect location is obtained by cropping, scaling, and filling coordinate transformation of the defect contour from step one in step two.

[0058] When extracting abnormal grayscale regions from solder joint sample images, the solder grayscale region and the substrate background region are first determined in the solder joint sample image. The solder grayscale region is obtained from the pixel region that has a grayscale difference from the surrounding substrate background and is continuously distributed, as determined in step two; the substrate background region is obtained from the image region in the solder joint sample image that is located outside the solder grayscale region and is not a filling pixel. The grayscale mean and grayscale standard deviation of the solder grayscale region are calculated, as are the grayscale mean and grayscale standard deviation of the substrate background region. The grayscale mean represents the average grayscale value of all valid pixels in the corresponding region, and the grayscale standard deviation represents the degree of dispersion of all valid pixel grayscale values ​​in the corresponding region relative to the grayscale mean.

[0059] Based on the mean and standard deviation of the solder grayscale region, internal abnormal grayscale regions are extracted within the solder grayscale region. If the solder in the solder joint sample image exhibits high grayscale, then consecutive pixels within the solder grayscale region whose grayscale values ​​are lower than the mean grayscale value of the solder grayscale region and whose deviation exceeds the limit of the standard deviation of the solder grayscale region are identified as internal abnormal grayscale regions. If the solder in the solder joint sample image exhibits low grayscale, then consecutive pixels within the solder grayscale region whose grayscale values ​​are higher than the mean grayscale value of the solder grayscale region and whose deviation exceeds the limit of the standard deviation of the solder grayscale region are identified as internal abnormal grayscale regions. Whether the solder exhibits high or low grayscale is determined by the grayscale display method read in step two.

[0060] Based on the mean and standard deviation of the grayscale value of the substrate background area, bridging aberration grayscale regions are extracted between adjacent solder joints. A bridging aberration grayscale region refers to a continuous pixel region located between adjacent solder joint regions, with a grayscale value close to the solder grayscale region and connected to at least two solder grayscale regions. When determining bridging aberration grayscale regions, the interval region between adjacent solder joints is first determined in the solder joint sample image; this interval region is jointly determined by the corrected position of the solder joint region and the position of the adjacent solder joint regions. Subsequently, continuous pixels with grayscale values ​​conforming to the solder grayscale range are extracted within the interval region, and it is determined whether these continuous pixels simultaneously contact two adjacent solder grayscale regions; if they do, the continuous pixels are identified as bridging aberration grayscale regions. The solder grayscale range is determined by the mean and standard deviation of the solder grayscale region.

[0061] Based on the solder joint design dimensions and the actual coverage of the solder grayscale area, abnormal grayscale areas with missing solder are extracted within the solder joint area. An abnormal grayscale area with missing solder refers to a pixel area within the solder joint design coverage area that is not covered by the solder grayscale area and whose area exceeds the allowable deviation range in the inspection process document. The solder joint design coverage area is calculated from the solder joint design dimensions and imaging magnification in step two. The solder joint design coverage area is compared with the solder grayscale area to identify pixel areas not covered by the solder grayscale area; if the area of ​​this pixel area exceeds the allowable solder coverage deviation range in the inspection process document, then this pixel area is identified as an abnormal grayscale area with missing solder.

[0062] Based on the continuity between the solder grayscale area and the connection position, abnormal grayscale areas of poor soldering are extracted within the solder joint area. The connection position refers to the location where the solder joint, as described in the chip packaging design document or testing process document, should contact the chip electrical connection structure or the substrate electrical connection structure. When extracting abnormal grayscale areas of poor soldering, the coordinate range of the connection position in the solder joint sample image is first determined based on the solder joint layout information. Then, it is determined whether the solder grayscale area forms a continuous pixel connection with the connection position. If there is a grayscale discontinuity between the solder grayscale area and the connection position, and the width of this discontinuous grayscale area exceeds the connection gap range allowed by the testing process document, then this discontinuous grayscale area is identified as an abnormal grayscale area of ​​poor soldering.

[0063] Internal abnormal grayscale regions, bridging abnormal grayscale regions, solder missing abnormal grayscale regions, and cold solder joint abnormal grayscale regions are merged to obtain defect candidate regions. During merging, if the shortest pixel distance between two abnormal grayscale regions is less than the boundary breakage distance allowed by the detection process document, the two abnormal grayscale regions are merged into one defect candidate region; if the shortest pixel distance between two abnormal grayscale regions is not less than the boundary breakage distance allowed by the detection process document, they are retained as different defect candidate regions. For each defect candidate region, its region boundary, region area, region center position, and the corresponding solder joint sample image are recorded. The region boundary is determined by the outermost pixel of the defect candidate region; the region area is obtained by multiplying the number of pixels in the defect candidate region by the actual area corresponding to a single pixel; the region center position is obtained by the mean of the horizontal coordinates and the mean of the vertical coordinates of all pixels in the defect candidate region; the actual area corresponding to a single pixel is calculated by converting the imaging magnification and the pixel size of the digital detector.

[0064] When building the training data for the solder joint defect recognition network, solder joint sample images, solder joint categories, and defect locations are read from the solder joint defect sample set. For each solder joint sample image, a defect annotation map is generated based on the defect location. The defect annotation map is a binary image with the same size as the solder joint sample image. A binary image is an image where each pixel location has only a defect marker value or a non-defect marker value. If the solder joint category corresponding to the solder joint sample image is a normal solder joint, then all pixels in the defect annotation map are set to non-defect marker values; if the solder joint category corresponding to the solder joint sample image is a void defect, bridging defect, missing solder defect, or cold solder joint defect, then the pixels inside the contour corresponding to the defect location are set to defect marker values, and the pixels outside the contour are set to non-defect marker values.

[0065] Solder joint sample images and corresponding defect annotation maps are input into the solder joint defect recognition network for training. During training, convolution processing is first performed on the solder joint sample images. Convolution processing refers to using a trainable convolution kernel to slide across the solder joint sample image according to pixel positions, and multiplying the pixel grayscale values ​​within the coverage area of ​​the convolution kernel with the convolution kernel parameters and summing the results to obtain a feature map representing local grayscale changes. The convolution kernel parameters are obtained through training on the solder joint defect sample set, and the coverage area of ​​the convolution kernel is determined based on the pixel scale of the solder joint edges, defect edges, and local grayscale changes in the solder joint sample image. The feature map is image-format data obtained from the convolution processing, and its pixel values ​​represent the local grayscale structure response of the solder joint sample image at the corresponding location.

[0066] Nonlinear mapping is performed on the feature map obtained from convolution. This nonlinear mapping suppresses values ​​smaller than a set response boundary in the feature map while retaining or enhancing values ​​larger than the set response boundary, ensuring that solder joint edges, defect edges, and abnormal grayscale regions form distinguishable responses in the feature map. The set response boundary is determined by parameters automatically updated during training based on the solder joint defect sample set. After nonlinear mapping, downsampling is performed on the feature map. This downsampling retains values ​​with strong grayscale structural responses within a local pixel range or calculates the average response value within that local pixel range, resulting in a smaller feature map. The local pixel range used for downsampling is determined based on the minimum identifiable size of the defect candidate region in the solder joint sample image. This minimum identifiable size is determined by the minimum defect size specified in the inspection process document or the minimum size of the labeled defect location in the solder joint defect sample set.

[0067] To simultaneously identify large-scale solder loss, bridging between adjacent solder joints, and small-scale voids, convolution processing with different receptive ranges is performed on the same solder joint sample image. The receptive range refers to the pixel coverage area in the solder joint sample image corresponding to a response position in the feature map. Small receptive range convolution processing extracts local edges and small-sized abnormal grayscale changes in the solder joint sample image; large receptive range convolution processing extracts the solder coverage area, connectivity status of adjacent solder joints, and overall grayscale distribution. The feature maps obtained from different receptive ranges are then adjusted to the same size and merged. Channel merging refers to arranging multiple feature maps according to feature category direction, so that the same pixel position simultaneously contains local edge features, overall coverage features, and connectivity features.

[0068] A defect probability map is generated on the feature map after channel merging. During the generation of the defect probability map, the feature map after channel merging is upsampled to restore its size to the size of the solder joint sample image. This upsampling process involves adding response values ​​to intermediate pixel positions based on the values ​​of adjacent response positions, thereby increasing the number of horizontal and vertical pixels in the feature map. Subsequently, the resized feature map undergoes probability mapping, converting the feature response value of each pixel position into a defect probability value, thus obtaining the defect probability map. The defect probability value ranges from non-defect to defective, with a higher probability value indicating a higher likelihood that the pixel position belongs to a defective region.

[0069] Solder joint category probabilities are generated on the feature map after channel merging. To generate these probabilities, a global statistical processing is first performed on the merged feature map to obtain overall feature values ​​representing the entire solder joint sample image. This global statistical processing involves calculating the average and maximum response values ​​of each feature category across all pixel locations in the feature map. Subsequently, the overall feature values ​​are linearly mapped to obtain category response values ​​for normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects. These category response values ​​are then converted into solder joint category probabilities, ensuring that each solder joint category probability represents the likelihood that the solder joint sample image belongs to its corresponding category.

[0070] When training the solder joint defect recognition network, the training loss is calculated based on the defect probability map, defect annotation map, solder joint category probability, and solder joint category. The training loss is calculated using the following formula: ; Where L represents the training loss; This represents the loss for determining the solder joint category, calculated from the difference between the solder joint category probability and the solder joint category. The defect location determination loss is calculated from the pixel difference between the defect probability map and the defect annotation map; This indicates the weight of the loss in determining the solder joint category; The weight represents the loss in defect location determination. The solder joint category probability is obtained from the output of the solder joint defect identification network; the solder joint category is obtained from the labeled data in the solder joint defect sample set; the defect probability map is also obtained from the output of the solder joint defect identification network; and the defect labeling map is generated from the defect location. and stated The settings are based on the error changes in solder joint category determination and defect location determination during the training process, and are adjusted using validation samples from the solder joint defect sample set. The validation samples are samples separated from the solder joint defect sample set that have not participated in the convolution kernel parameter update and possess both solder joint category and defect location information.

[0071] The convolutional kernel parameters and linear mapping parameters in the solder joint defect recognition network are updated based on the training loss. The linear mapping parameters are trainable parameters used to convert the overall feature values ​​into category response values. During parameter updates, the direction of change of the training loss relative to the convolutional kernel parameters and linear mapping parameters is calculated, and the convolutional kernel parameters and linear mapping parameters are adjusted along the direction that reduces the training loss. The process of inputting solder joint sample images, generating defect probability maps, generating solder joint category probabilities, calculating training loss, and updating parameters is repeated until the solder joint category determination loss and defect location determination loss on the verification samples no longer decrease, or the training stop condition specified in the inspection process document is reached, resulting in a successfully trained solder joint defect recognition network.

[0072] When using the trained solder joint defect recognition network to identify solder joint sample images, the solder joint sample image is input into the network to obtain the corresponding defect probability map and solder joint category probability. A defect probability threshold is determined based on validation samples. To determine the defect probability threshold, validation samples are input into the trained solder joint defect recognition network to obtain the corresponding defect probability map. Under multiple candidate probability thresholds, the defect probability map is converted into a binary defect region, and the binary defect region is compared with the defect annotation map of the validation sample. The candidate probability threshold that results in the highest degree of overlap between the binary defect region and the defect annotation map is selected as the defect probability threshold. The binary defect region refers to the continuous pixel region formed after determining pixels in the defect probability map that are greater than or equal to the defect probability threshold as defect pixels.

[0073] For the thresholds determined by the verification samples in this embodiment, the same threshold selection rule is followed. Specifically, the verification samples are input into the trained solder joint defect recognition network to obtain the defect probability map and solder joint category probability corresponding to the verification samples; then, based on the defect location, solder joint category, solder grayscale area, substrate background area, connection location, and solder joint arrangement information in the verification samples, the recognition results corresponding to the candidate thresholds are calculated respectively; finally, the candidate threshold that preserves the real defect area, suppresses the projected pseudo-anomaly area, and has a high accuracy in solder joint category recognition is selected as the corresponding threshold.

[0074] The projection repetition threshold is used to determine whether the current valid defect area belongs to the projection influence area. To determine the projection repetition threshold, the internal structure projection repetition state values ​​of the real defect area and the projection pseudo-anomaly area are first calculated in the verification sample. Then, a candidate threshold that can distinguish between the real defect area and the projection pseudo-anomaly area is selected as the projection repetition threshold. If a candidate threshold can make the internal structure projection repetition state value of most projection pseudo-anomaly areas higher than the candidate threshold and the internal structure projection repetition state value of most real defect areas lower than the candidate threshold, then that candidate threshold is used as the projection repetition threshold.

[0075] The residual decision threshold is used to determine the individual anomaly retention area from the individual residual map. When determining the residual decision threshold, the baseline residual threshold is first determined based on the degree of grayscale deviation of each co-located reference area relative to the common grayscale baseline map. Then, the threshold increment is determined based on the internal structure projection repetition state value, and the threshold decrement is determined based on the category position conformity state value. The residual decision threshold is obtained by combining the baseline residual threshold, the threshold increment, and the threshold decrement.

[0076] Boundary break distance is used to determine whether additional contour points or merging are needed for correcting defect contours. The boundary break distance is determined based on the solder joint design size, imaging magnification, and the distribution of boundary discontinuities in the defect contours of the verification sample. If the shortest pixel distance between two boundary endpoints is less than the boundary break distance, contour points are added according to the shortest pixel path; if the shortest pixel distance between two corrected defect contours is less than the boundary break distance and the corresponding initial solder joints are of the same type, the two corrected defect contours are merged.

[0077] The allowable connection gap is used to determine cold solder joint defects. The allowable connection gap is calculated from the permissible interval between connection positions specified in the inspection process document; if the inspection process document does not directly specify it, it is determined based on the gap distribution between the solder grayscale area and the connection position in normal solder joint samples. If the width of the discontinuous area between the solder grayscale area and the connection position exceeds the allowable connection gap, this discontinuous area is used to determine the cold solder joint defect.

[0078] The defect probability map corresponding to the solder joint sample image is converted into a binary defect region according to the defect probability threshold, and connected component extraction is performed on the binary defect region. The connected component refers to the set of defect pixels that can be interconnected in a horizontal, vertical, or diagonal adjacent relationship. The area and boundary of each connected component are calculated; if the area of ​​a connected component is less than the minimum effective defect area, the connected component is deleted. The minimum effective defect area is obtained by converting the minimum defect size specified in the inspection process document, or determined by the minimum area of ​​the already marked defect location in the solder joint defect sample set. The remaining connected components after deletion are used as the defect identification region.

[0079] The identification defect region is compared with the aforementioned defect candidate regions. If an identification defect region and a defect candidate region have overlapping pixels, the identification defect region and the defect candidate region are merged into the same initial defect region. If an identification defect region does not overlap with any defect candidate region, but its average defect probability is greater than the defect probability threshold and its area is not less than the minimum effective defect area, the identification defect region is retained as the initial defect region. If a defect candidate region does not overlap with any identification defect region, it is judged based on the degree of grayscale anomaly of the defect candidate region. The degree of grayscale anomaly is determined by the difference between the grayscale mean of the defect candidate region and the grayscale mean of the solder grayscale region. If the degree of grayscale anomaly exceeds the grayscale standard deviation limit of the solder grayscale region, the defect candidate region is retained as the initial defect region; otherwise, the defect candidate region is deleted.

[0080] The initial solder joint category is determined based on the probability of the solder joint category. When determining the initial solder joint category, the category with the highest probability value among the solder joint categories corresponding to normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects is selected as the initial solder joint category. If the initial defect area is empty and the probability of the solder joint category corresponding to the normal solder joint is the highest, then the initial solder joint category of the solder joint sample image is determined as a normal solder joint; if the initial defect area is not empty, the initial solder joint category is determined based on the positional relationship of the initial defect area. When the initial defect area is located inside the solder grayscale area and is not connected to the external background of the solder joint, it corresponds to a void defect; when the initial defect area spans the interval between adjacent solder joints, it corresponds to a bridging defect; when the initial defect area is located within the designed coverage area of ​​the solder joint and corresponds to a position not covered by the solder grayscale area, it corresponds to a missing solder defect; when the initial defect area is located in a discontinuous area between the solder grayscale area and the connection position, it corresponds to a cold solder joint defect.

[0081] When the solder joint category determined by the solder joint category probability is inconsistent with the solder joint category determined by the positional relationship of the initial defect area, the selection is based on the recognition accuracy of the corresponding solder joint category in the verification samples. The recognition accuracy is calculated as the ratio of the number of correctly identified samples of that solder joint category to the total number of samples of that solder joint category. If the recognition accuracy of the solder joint category corresponding to the solder joint category probability is higher than the recognition accuracy of the solder joint category corresponding to the positional relationship of the initial defect area, then the solder joint category determined by the solder joint category probability is used; otherwise, the solder joint category determined by the positional relationship of the initial defect area is used.

[0082] After completing the above processing, the initial solder joint defect identification results are generated. These results include a solder joint sample image, the chip number to be inspected, the corrected position of the solder joint region, the initial solder joint category, the initial defect region, the defect probability map, and the solder joint category probability. The solder joint sample image is obtained from step two; the chip number to be inspected follows the chip number corresponding to the target X-ray image; the corrected position of the solder joint region is obtained from step two; the initial solder joint category is determined by the relationship between the solder joint category probability and the position of the initial defect region; the initial defect region is obtained by merging the defect probability map and the defect candidate region; and the defect probability map and solder joint category probability are obtained from the output of the trained solder joint defect identification network.

[0083] In step four, the initial solder joint defect identification result obtained in step three is read, and the boundary of the initial defect area is refined to obtain the corrected defect contour. The corrected defect contours that overlap or are adjacent to each other in the same solder joint area are merged to obtain the effective defect area. The solder joint defect identification result is generated based on the effective defect area, the corrected position of the solder joint area, and the initial solder joint category.

[0084] When reading the initial solder joint defect identification result, the following are simultaneously read: solder joint sample image, chip number under inspection, corrected position of solder joint area, initial solder joint category, initial defect area, defect probability map, and solder joint category probability. The solder joint sample image is obtained from step two; the corrected position of solder joint area is calculated from the outer boundary of the solder grayscale area in step two; the initial solder joint category, initial defect area, defect probability map, and solder joint category probability are obtained from step three. The corrected defect contour refers to the outer boundary of the defect obtained after grayscale edge correction of the boundary of the initial defect area; the effective defect area refers to the defect area retained after boundary refinement, repeated merging, and validity judgment; the solder joint defect identification result refers to the identification result including the chip number under inspection, corrected position of solder joint area, final solder joint category, effective defect area, defect parameters, and defect confidence value. The defect confidence value represents the reliability of the corresponding solder joint defect identification result and is jointly determined by the defect probability map, solder joint category probability, and grayscale anomaly degree of the effective defect area.

[0085] When refining the boundaries of the initial defect region, the pixel range corresponding to the initial defect region is first read from the solder joint sample image, and then the boundary search region is obtained by expanding outward from this pixel range. The boundary search region is a local image area set around the outer boundary of the initial defect region, and its expansion distance is determined based on the grayscale transition width near the boundary of the initial defect region. The grayscale transition width is determined by the number of pixels through which the grayscale on both sides of the initial defect region boundary in the solder joint sample image changes from defect grayscale to solder grayscale or substrate background grayscale. If the grayscale transition width near the boundary of the initial defect region is difficult to determine directly, the pixel width corresponding to the minimum defect size specified in the inspection process document is used as the expansion distance of the boundary search region.

[0086] Within the boundary search area, the intensity of grayscale changes is calculated along the outward normal direction of the initial defect region. The outward normal direction refers to the direction from the inside of the initial defect region to the outside of the initial defect region and perpendicular to the boundary of the initial defect region. The intensity of grayscale changes is obtained from the grayscale difference between adjacent pixels; when there are multiple grayscale change locations along the outward normal direction, the pixel location with the largest grayscale change intensity is taken as the corresponding boundary point. The above processing is performed on all boundary points on the boundary of the initial defect region to obtain an initial corrected contour composed of multiple boundary points.

[0087] After obtaining the initial corrected contour, a continuity processing step is performed. If a break exists between two adjacent boundary points in the initial corrected contour, contour points are added between the two boundary points using the shortest pixel path. If a protruding contour segment formed by isolated noise exists in the initial corrected contour, that protruding contour segment is deleted. The protruding contour segment is a local contour that extends outward from the defect region, has a connection length with the main body of the defect region that is less than its own contour length, and has a corresponding area that is less than the minimum effective defect area specified in the inspection process document. The contour after the continuity processing step is recorded as the corrected defect contour.

[0088] For the initial defect region corresponding to a void defect, the corrected defect contour is confined within the solder grayscale region. If the corrected defect contour crosses the outer boundary of the solder grayscale region, the protruding portion is cut back into the solder grayscale region along the outer boundary. For the initial defect region corresponding to a bridging defect, the corrected defect contour is confined to the outer boundary of the continuous solder grayscale structure between adjacent solder joint regions; if the corrected defect contour does not cover the continuous solder grayscale structure between adjacent solder joints, pixels in the continuous solder grayscale structure that are connected to both adjacent solder grayscale regions are incorporated into the corrected defect contour. For the initial defect region corresponding to a missing solder defect, the corrected defect contour is confined to the position within the solder joint design coverage area that is not covered by the solder grayscale region; if the corrected defect contour exceeds the solder joint design coverage area, the excess portion is deleted. For the initial defect region corresponding to a cold solder joint defect, the corrected defect contour is confined to the discontinuous region between the solder grayscale region and the connection position; if the corrected defect contour does not cover the discontinuous region, contour points are added along the grayscale boundary of the discontinuous region.

[0089] Multiple corrected defect contours in the same solder joint sample image are merged. If the regions enclosed by two corrected defect contours have overlapping pixels, the two corrected defect contours are merged into one. If the shortest pixel distance between two corrected defect contours is less than the boundary breakage distance allowed by the inspection process document, and the initial solder joint categories corresponding to the two corrected defect contours are the same, the two corrected defect contours are merged into one. If the shortest pixel distance between two corrected defect contours is not less than the boundary breakage distance allowed by the inspection process document, they are retained separately. The region enclosed by the merged corrected defect contours is recorded as the valid defect region.

[0090] The validity of valid defect areas is assessed. If the area of ​​a valid defect area is smaller than the minimum valid defect area, the valid defect area is deleted. The minimum valid defect area is calculated from the minimum defect size specified in the inspection process document, or determined by the minimum area of ​​the marked defect location in the solder joint defect sample set. If the grayscale anomaly of a valid defect area does not exceed the grayscale standard deviation limit of the solder grayscale area, the valid defect area is deleted. The grayscale anomaly is determined by the difference between the grayscale mean of the valid defect area and the grayscale mean of the adjacent solder grayscale area; the grayscale standard deviation of the solder grayscale area is calculated from the grayscale values ​​of all valid pixels within the solder grayscale area in step three.

[0091] The final solder joint category is determined for the retained valid defect areas. If the valid defect area is located inside the solder grayscale area and is not connected to the substrate background area outside the solder joint, the final solder joint category is determined to be a void defect. If the valid defect area spans the gap between adjacent solder joint areas and is connected to both adjacent solder grayscale areas, the final solder joint category is determined to be a bridging defect. If the valid defect area is located within the solder joint design coverage area but is not covered by the solder grayscale area, and the area of ​​the valid defect area exceeds the solder coverage deviation range allowed by the inspection process document, the final solder joint category is determined to be a missing solder defect. If the valid defect area is located between the solder grayscale area and the connection position, and there is a discontinuous area between the solder grayscale area and the connection position with a width exceeding the connection gap range allowed by the inspection process document, the final solder joint category is determined to be a cold solder joint defect. If there are no retained valid defect areas, and the probability value corresponding to the normal solder joint is the highest among the solder joint category probabilities, the final solder joint category is determined to be a normal solder joint.

[0092] When multiple valid defect regions exist in the same solder joint sample image, the defect category corresponding to each valid defect region is determined separately. If multiple valid defect regions correspond to different defect categories, the final solder joint category record for the solder joint sample image is formed by combining the defect categories corresponding to each valid defect region. If multiple valid defect regions correspond to the same defect category, the defect category is retained, and the multiple valid defect regions are jointly recorded in the defect region record under that defect category. If the initial solder joint category is inconsistent with the defect category determined by the positional relationship of the valid defect regions, the final solder joint category is determined by the positional relationship of the valid defect regions; when the positional relationship of the valid defect regions cannot distinguish the defect categories, the final solder joint category is determined by the defect category with the highest probability value among the solder joint category probabilities.

[0093] Defect parameters are calculated based on the effective defect area. These parameters include the defect area, defect center location, defect perimeter, and defect area percentage. The defect area is obtained by multiplying the number of pixels within the effective defect area by the actual area corresponding to a single pixel. The actual area corresponding to a single pixel is calculated using the imaging magnification and digital detector pixel size from step three. The defect center location is obtained by the average of the horizontal and vertical coordinates of all pixels within the effective defect area. The defect perimeter is obtained by the minimum and maximum coordinates of the effective defect area in the horizontal direction and in the vertical direction.

[0094] The defect area ratio is calculated using the following formula: ; Where P represents the percentage of defect area; The defect area is represented by multiplying the number of pixels within the effective defect area by the actual area corresponding to a single pixel. This represents the reference area of ​​the solder joint, determined by the area of ​​the solder joint region corresponding to the corrected location, or calculated from the solder joint design dimensions. If the final solder joint category is a void defect, then... Take the area of ​​the grayscale region of the solder; if the final solder joint category is a solder missing defect, then Take the area covered by the designed solder joint; if the final solder joint category is a bridging defect, then Take the area of ​​the interval between adjacent solder joints; if the final solder joint category is a cold solder joint defect, then Take the area of ​​the designed contact area corresponding to the connection position. The area of ​​the solder grayscale area is calculated from the number of pixels of the solder grayscale area in step two; the area of ​​the solder joint design coverage, the area of ​​the interval between adjacent solder joints, and the area of ​​the designed contact area are obtained from the chip packaging design file or the testing process file and calculated according to the imaging magnification.

[0095] It should be noted that during X-ray imaging of the solder joint area inside the chip under inspection, X-rays simultaneously pass through the chip's packaging material, solder, chip electrical connection structure, and substrate electrical connection structure. Therefore, a certain gray-scale abnormal area in the target X-ray image is not necessarily formed solely by defects in the solder joint itself, but may also be formed by the overlap of the projection of the chip electrical connection structure or substrate electrical connection structure along the X-ray irradiation direction with the solder joint area. The projection refers to the gray-scale representation formed in the target X-ray image after different internal structures are superimposed along the X-ray irradiation direction. After cropping the target X-ray image into solder joint sample images in step two, each solder joint sample image only retains the corresponding solder joint area and its local background. If the projection of a certain internal structure is continuously distributed along the solder joint array direction, the projection of this internal structure may form similar abnormal gray-scale areas at the same relative position in multiple adjacent solder joint sample images. At this point, for a single solder joint sample image, the abnormal grayscale region may have a high probability of defect, or it may be identified as an initial defect region in step three and retained as a valid defect region in step four. However, from the overall distribution of multiple solder joint regions in the target X-ray image, the abnormal grayscale region has repeatability and positional consistency in the array direction, and its cause may not be a defect of a single solder joint itself.

[0096] Furthermore, within the same chip under inspection, the abnormal grayscale areas formed by the projection of the internal structure may overlap with actual solder joint defects. In other words, a valid defect area contains both common grayscale components formed by the projection of the internal structure and individual abnormal components formed by void defects, bridging defects, missing solder defects, or cold solder joint defects. The common grayscale components refer to grayscale components that repeat in multiple adjacent solder joint areas according to their similar relative positions; the individual abnormal components refer to grayscale components that appear only within the current solder joint area and correspond to the solder grayscale area, connection position, or interval area between adjacent solder joints in the current solder joint area.

[0097] In the above situations, relying solely on the average probability value of the defect probability map within the effective defect area might misidentify common grayscale components formed by the projection of internal structures as real defects; relying solely on the probability value of the solder joint category corresponding to the final solder joint category might overlook the repeatability of the effective defect area along the solder joint array direction; and relying solely on the degree of grayscale anomaly in the effective defect area might fail to distinguish between local grayscale anomalies caused by real defects and superimposed grayscale anomalies caused by the projection of internal structures. Therefore, even when the effective defect area and the final solder joint category are determined, the authenticity of the effective defect area remains difficult to ascertain.

[0098] Therefore, in this embodiment, when calculating the defect confidence value, the effective defect area, defect probability map, solder joint category probability, solder joint area correction position, solder grayscale area, substrate background area, connection position, and solder joint layout information are first read. The effective defect area is obtained from the initial defect area through boundary refinement, duplicate result merging, and validity judgment; the defect probability map and solder joint category probability are obtained from the output of the solder joint defect identification network in step three; the solder joint area correction position, solder grayscale area, and substrate background area are obtained from step two; the connection position is obtained from the chip packaging design file or inspection process file; and the solder joint layout information is obtained from the chip packaging design file, inspection process file, or solder joint array coordinate table in step one.

[0099] Within the current solder joint area, using the corrected center coordinates of the solder joint area correction position as a reference, the relative position description data of the effective defect area relative to the current solder joint area is calculated. This relative position description data includes the lateral offset of the effective defect area center relative to the corrected center coordinates, the longitudinal offset of the effective defect area center relative to the corrected center coordinates, and the relative lateral and longitudinal dimensions of the effective defect area's outer perimeter relative to the solder joint area correction position. If the final solder joint category is a bridging defect, then using the center coordinates of the interval between two adjacent solder joint areas as a reference, the relative position description data of the effective defect area relative to the interval between the two adjacent solder joint areas is calculated; the interval center coordinates are obtained by taking the midpoint of the center coordinates of the correction positions of the two adjacent solder joint areas.

[0100] Based on the solder joint layout information, a reference solder joint region is selected in the target X-ray image that is located in the same solder joint array direction as the current solder joint region. The reference solder joint region refers to a solder joint region located in the same row or column as the current solder joint region, having the same solder joint design size, and not exceeding the effective imaging range of the target X-ray image. If the current solder joint region is located at the edge of the solder joint array, a reference solder joint region is selected along the side where adjacent solder joint regions can be obtained; if adjacent solder joint regions exist on both sides of the current solder joint region, reference solder joint regions are selected along both sides respectively. For bridging defects, the interval between adjacent solder joint regions is used as a reference object, and a corresponding reference interval region is selected in the same solder joint array direction. The reference interval region refers to an adjacent solder joint interval region that has the same arrangement relationship as the adjacent solder joint interval where the current bridging defect is located.

[0101] The relative position description data in the current solder joint region is mapped to each reference solder joint region to obtain a co-position reference region. The co-position reference region refers to an image region in the reference solder joint region that has the same relative position and size as the current valid defect region. For bridging defects, the relative position description data in the current adjacent solder joint interval is mapped to each reference interval region to obtain a co-position reference region. Each co-position reference region is represented by pixel coordinates in the target X-ray image and establishes a correspondence with the corresponding reference solder joint region or reference interval region.

[0102] Extract corresponding grayscale anomaly regions within each corresponding reference region. If the final solder joint is classified as a void defect, extract continuous pixel regions within the corresponding solder grayscale region that deviate from the average grayscale value of that corresponding solder grayscale region as corresponding grayscale anomaly regions. If the final solder joint is classified as a bridging defect, extract continuous pixel regions within the corresponding reference region that span the corresponding reference interval region and are connected to the solder grayscale regions on both sides of the reference interval region as corresponding grayscale anomaly regions. If the final solder joint is classified as a missing solder defect, extract continuous pixel regions within the corresponding solder joint design coverage area that are not covered by the corresponding solder grayscale region as corresponding grayscale anomaly regions. If the final solder joint is classified as a cold solder joint defect, extract discontinuous regions within the corresponding solder grayscale region and the corresponding connection position as corresponding grayscale anomaly regions within the corresponding reference region.

[0103] The internal structure projection repetition value is calculated based on the corresponding grayscale anomaly region. The internal structure projection repetition value indicates whether the currently valid defect region exhibits a recurring grayscale appearance along the weld point array direction. The internal structure projection repetition value is calculated using the following formula: ; in, Indicates the repeated state value of the internal structure projection; This indicates the number of available cosine reference regions, which are determined by cosine reference regions in the target X-ray image that do not exceed the effective imaging range and have corresponding solder grayscale regions. This indicates the number of co-located reference regions where co-located grayscale anomalies exist. This indicates the average degree of overlap between the corresponding gray-scale anomaly region and the current valid defect region in relative position; This represents the average similarity in grayscale appearance between the corresponding grayscale anomaly region and the current valid defect region. The average overlap is calculated by averaging the proportion of the overlapping area between each corresponding grayscale anomaly region mapped back to the current solder joint region and the current valid defect region to the area of ​​the current valid defect region. The average similarity is determined by the difference between the mean grayscale value of the corresponding grayscale anomaly region and the mean grayscale value of the current valid defect region; the closer their mean grayscale values ​​are, the higher the average similarity. If no usable corresponding reference region exists, the internal structure projection repetition state value is not calculated, and the defect confidence value is determined based on the local grayscale appearance and positional relationship of the current valid defect region.

[0104] When the current solder joint area is located at the edge of the solder joint array, resulting in an insufficient number of reference solder joint areas in the same row or column direction, a reference solder joint area or reference interval area with the same design size and within the effective imaging range is selected first along another solder joint array direction. If there is still no available co-located reference area in another solder joint array direction, a common grayscale reference map is not established, and the basic reliable state value is determined by the local defect probability state value, category separation state value, category position conformity state value, and boundary stability state value of the current effective defect area.

[0105] When the area of ​​an abnormal grayscale region in a certain reference region exceeds a preset multiple of the area of ​​the current effective defect region, or the probability of the solder joint category corresponding to the reference region does not point to a normal solder joint, or there is a identified effective defect region in the reference region, the reference region is marked as an unusable reference region and is not included in the calculation of the common grayscale reference map and the internal structure projection repetition state value. By excluding unusable reference regions, it is possible to avoid mistaking real defects in the reference solder joint region as common grayscale components.

[0106] When solder joint areas with different design sizes exist within the same solder joint array, only the reference solder joint area with the same design size as the current solder joint area is selected as the co-position reference area. If a bridging defect corresponds to an adjacent solder joint interval, only the reference interval area with the same adjacent solder joint spacing and the same connection direction as the current bridging interval is selected as the co-position reference area. Areas with different design sizes or different adjacent solder joint spacings are not considered as co-position reference areas.

[0107] When a local area in a target X-ray image is overexposed, underexposed, obstructed, or has its imaging field of view truncated, the reference solder joint area or reference interval area located within that local area is not considered a usable co-located reference area. Overexposed areas are defined as continuous pixel areas with grayscale values ​​reaching the upper limit of the imaging device's grayscale; underexposed areas are defined as continuous pixel areas with grayscale values ​​reaching the lower limit of the imaging device's grayscale; and truncated imaging field of view areas are defined as reference solder joint areas or reference interval areas exceeding the effective imaging range of the target X-ray image.

[0108] When a usable co-location reference region exists, a common grayscale reference map is established based on each co-location reference region. The common grayscale reference map is a grayscale image with the same size as the current effective defect region, and the grayscale value of each pixel is obtained by taking the median value of the grayscale values ​​of pixels at the same relative position in multiple co-location reference regions.

[0109] For example, if the current effective defect area has a lateral offset of 'a' pixels and a vertical offset of 'b' pixels relative to the solder joint area correction center, with an outer dimension of 'm' pixels multiplied by 'n' pixels, then reference solder joint areas with the same design size and within the effective imaging range are selected in the same row or column. Within each reference solder joint area, a corresponding reference area is obtained by using the corresponding solder joint area correction center as a reference and applying the same lateral offset 'a', vertical offset 'b', and outer dimension of 'm' pixels multiplied by 'n' pixels. If the current effective defect area corresponds to a bridging defect, then a corresponding reference area is obtained by using the bridging interval center as a reference and applying the same lateral offset, vertical offset, and outer dimension within other reference interval areas with the same adjacent solder joint spacing.

[0110] When generating the individual residual map, the grayscale value of each pixel in the current effective defect area is subtracted from the grayscale value of the same position in the common grayscale reference map, and the absolute value is taken to obtain the grayscale deviation degree of that pixel position. If the grayscale deviation degree is greater than or equal to the residual judgment threshold, the pixel position is retained as an individual abnormal pixel; the continuous pixel region formed by interconnected individual abnormal pixels is determined as the individual abnormal retention region.

[0111] The median value is the gray value located in the middle position after sorting multiple pixel gray values ​​by size; when the number of pixel gray values ​​involved in the sorting is even, the average of the two middle gray values ​​is taken as the median value. Through the above processing, the projected gray components that appear repeatedly in multiple reference solder joint areas are written into the common gray-scale reference map, while the local abnormal gray components that appear only in a few reference solder joint areas do not directly determine the gray values ​​of the common gray-scale reference map.

[0112] The grayscale values ​​of pixels within the current valid defect area are compared with the grayscale values ​​of pixels at the same location in the common grayscale reference image to obtain an individual residual map. The individual residual map is an image with the same size as the current valid defect area, and the value at each pixel position represents the degree of grayscale deviation of the current valid defect area relative to the common grayscale reference image at that pixel position. If the grayscale performance of the current valid defect area is close to that of the common grayscale reference image, the grayscale deviation at the corresponding pixel position is low; if the current valid defect area exhibits grayscale changes at a local location that differ from the array repetition projection, the grayscale deviation at the corresponding pixel position is high.

[0113] The residual judgment threshold is determined based on the individual residual map. The residual judgment threshold is jointly determined by the baseline residual threshold, the internal structure projection repetition state value, and the category position conformity state value. The baseline residual threshold is determined by the degree of grayscale deviation between each co-located reference region and the common grayscale reference map; specifically, the grayscale difference of each pixel in each co-located reference region relative to the pixel at the same position in the common grayscale reference map is first calculated, and then the baseline residual threshold is determined based on the median value and dispersion of all grayscale differences. The category location conformity status value is used to indicate whether the positional relationship of the current effective defect area conforms to the defect performance corresponding to the final solder joint category; the category location conformity status corresponding to the void defect is determined by the degree to which the effective defect area is located inside the solder grayscale area and is not connected to the substrate background area; the category location conformity status corresponding to the bridging defect is determined by the degree to which the effective defect area crosses the interval between adjacent solder joints and simultaneously connects the solder grayscale areas on both sides; the category location conformity status corresponding to the solder missing defect is determined by the degree to which the effective defect area is located within the solder joint design coverage area and is not covered by the solder grayscale area; and the category location conformity status corresponding to the cold solder joint defect is determined by the degree to which the effective defect area is located in the discontinuous area between the solder grayscale area and the connection position.

[0114] When the final solder joint is classified as a void defect, the category location compliance value is determined by the proportion of the effective defect area located within the solder grayscale area and the state in which the effective defect area is not connected to the substrate background area. If the entire effective defect area is located within the solder grayscale area and is not connected to the substrate background area, the category location compliance value is high; if part of the effective defect area extends beyond the solder grayscale area or is connected to the substrate background area, the category location compliance value is low.

[0115] When the final solder joint category is a bridging defect, the category location compliance value is determined by the proportion of the area of ​​the effective defect region that spans the interval between adjacent solder joints and the connectivity between the effective defect region and the grayscale regions of the solder on both sides. If the effective defect region spans the interval between adjacent solder joints and connects to the grayscale regions of the solder on both sides, the category location compliance value is higher; if the effective defect region is located only within a single solder joint region or only connects to the grayscale region of the solder on one side, the category location compliance value is lower.

[0116] When the final solder joint category is a solder missing defect, the category location compliance value is determined by the proportion of the effective defect area that is within the solder joint design coverage area and is not covered by the solder gray area. If the effective defect area is mainly within the solder joint design coverage area and does not belong to the solder gray area, the category location compliance value is higher; if the effective defect area deviates from the solder joint design coverage area, the category location compliance value is lower.

[0117] When the final solder joint is classified as a cold solder joint defect, the category location compliance value is determined by the proportion of the area of ​​the discontinuous region between the solder gray area and the connection position, and whether the width of the discontinuous region exceeds the allowable connection gap. If the effective defect area is located between the solder gray area and the connection position and the width of the discontinuous region exceeds the allowable connection gap, the category location compliance value is higher; if the effective defect area is not located in this positional relationship, the category location compliance value is lower.

[0118] When determining the residual judgment threshold, a threshold increment that changes in the same direction as the internal structure projection repetition state value is introduced on the basis of the baseline residual threshold, and a threshold decrement that changes in the same direction as the category position conformity state value is introduced. A higher internal structure projection repetition state value indicates that the current effective defect area is more likely to contain common grayscale components that recur in the array direction, thus increasing the residual judgment threshold. A higher category position conformity state value indicates that the current effective defect area better conforms to the defect position relationship corresponding to the final solder joint category, thus decreasing the residual judgment threshold. The adjustment ratio of the threshold increment and threshold decrement is determined by the validation samples, which are samples separated from the solder joint defect sample set in step three that have not participated in the convolution kernel parameter update and have solder joint category and defect position information.

[0119] A contiguous pixel region in the individual residual map that is greater than or equal to the residual judgment threshold is defined as an individual anomaly retention region. The individual anomaly retention region represents a local anomaly portion in the current valid defect region that differs from the array repetition projection. If no individual anomaly retention region exists in the current valid defect region, and the internal structure projection repetition state value is higher than the projection repetition judgment threshold determined by the validation samples, then the current valid defect region is marked as a projection influence region. The projection repetition judgment threshold is determined by the distribution of internal structure projection repetition state values ​​in the real defect region and the projection pseudo-anomaly region in the validation samples. If an individual anomaly retention region exists in the current valid defect region, it is used as a confidence evaluation region; if no individual anomaly retention region exists, but the internal structure projection repetition state value is not higher than the projection repetition judgment threshold, then the current valid defect region is used as a confidence evaluation region. The confidence evaluation region is the defect region that participates in the calculation of the defect confidence value.

[0120] The local defect probability state value is calculated within the confidence evaluation region. This value is determined by the pixel probability values ​​located within the confidence evaluation region in the defect probability map. Specifically, the probability value of each pixel within the confidence evaluation region is first read from the defect probability map. Then, extreme probability values ​​formed by isolated pixels are removed. Finally, the average of the remaining pixel probability values ​​is calculated to obtain the local defect probability state value. An isolated pixel refers to a single pixel or a small number of consecutive pixels that are not continuously connected to the main body of the confidence evaluation region. The main body of the confidence evaluation region is determined by the largest continuous pixel area within the confidence evaluation region.

[0121] The class separation state value is calculated from the solder joint category probability. The class separation state value is determined by the difference between the solder joint category probability corresponding to the final solder joint category and the highest solder joint category probability among other solder joint categories. If the solder joint category probability corresponding to the final solder joint category is significantly higher than that of other solder joint categories, the class separation state value is high; if the solder joint category probability corresponding to the final solder joint category is close to that of other solder joint categories, the class separation state value is low. For cases where multiple valid defect regions exist in the same solder joint sample image, the class separation state value corresponding to the final solder joint category for each valid defect region is calculated separately.

[0122] The boundary stability state value is calculated within the confidence evaluation region. The boundary stability state value is determined by the correspondence between the boundary of the confidence evaluation region and the location of the maximum grayscale change intensity in the solder joint sample image. During calculation, the grayscale change intensity is read along the outward normal direction of the confidence evaluation region boundary, and it is determined whether the boundary point of the confidence evaluation region is located at the location of the maximum grayscale change intensity or its adjacent location. The higher the proportion of boundary points located at the location of the maximum grayscale change intensity or its adjacent location to the total number of boundary points, the higher the boundary stability state value. The adjacent location refers to the pixel position that is horizontally, vertically, or diagonally adjacent to the location of the maximum grayscale change intensity.

[0123] The grayscale individual state value is calculated within the confidence evaluation region. The grayscale individual state value is determined by the pixel values ​​of the individual residual map corresponding to the confidence evaluation region. Specifically, the values ​​of all pixels within the confidence evaluation region in the individual residual map are read, and the proportion of pixels with values ​​greater than or equal to the residual judgment threshold is calculated relative to the total number of pixels in the confidence evaluation region. Simultaneously, the average grayscale deviation of these pixels is calculated. The grayscale individual state value is determined jointly by the proportion and the average grayscale deviation. If most pixels in the confidence evaluation region differ from the common grayscale reference map, and the average grayscale deviation is high, the grayscale individual state value is high; if the confidence evaluation region is mainly composed of common grayscale components, the grayscale individual state value is low.

[0124] The basic confidence state value is determined based on the local defect probability state value, category separation state value, category location conformity state value, boundary stability state value, and grayscale individual state value. The basic confidence state value is the minimum value among these state values. If any state value is low, the basic confidence state value is correspondingly limited; if all state values ​​are high, the basic confidence state value remains within a high range. Before comparison, each state value is converted to the same numerical range based on the minimum and maximum valid values ​​of the corresponding state values ​​in the verification samples.

[0125] In one specific implementation, the local defect probability state value, category separation state value, category location conformance state value, boundary stability state value, and grayscale individual state value are all converted to a range of 0 to 1. The local defect probability state value is determined by the average pixel probability of the defect probability map within the confidence evaluation area; the category separation state value is determined by the difference between the probability of the solder joint category corresponding to the final solder joint category and the probability of the largest solder joint category among other solder joint categories; the category location conformance state value is determined by the degree of matching between the effective defect area and the corresponding position of the final solder joint category; the boundary stability state value is determined by the matching ratio between the boundary point of the confidence evaluation area and the position with the maximum grayscale change intensity or its adjacent position; the grayscale individual state value is jointly determined by the proportion of pixels in the individual residual map of the confidence evaluation area that are higher than the residual judgment threshold and the degree of average grayscale deviation.

[0126] By taking the minimum value, when any one of the following factors—defect probability map, solder joint category probability, defect location relationship, boundary stability, or grayscale individuality—does not meet the credibility requirements, the defect confidence value is limited, thereby avoiding misjudgment caused by relying solely on a single probability or a single grayscale feature.

[0127] The basic confidence state value is corrected based on the internal structure projection repetition state value and the proportion of individual anomaly retention areas to obtain the defect confidence value. The proportion of individual anomaly retention areas refers to the ratio between the area of ​​the individual anomaly retention area and the area of ​​the currently valid defect area. If the internal structure projection repetition state value is high and the proportion of individual anomaly retention areas is low, the basic confidence state value is reduced; if the internal structure projection repetition state value is high but the proportion of individual anomaly retention areas is also high, the basic confidence state value corresponding to the individual anomaly retention area is retained; if the internal structure projection repetition state value is low and the category position conformity state value is high, the retention level of the basic confidence state value is increased. The defect confidence value is determined according to the following relationship: ; Where C represents the defect confidence value; Represents the basic trusted state value; Indicates the repeated state value of the internal structure projection; This indicates the percentage of the individual anomaly retention area. The internal structure projection repetition state value is determined by the repetitive grayscale anomalies in the corresponding reference area; the percentage of the individual anomaly retention area is determined by the area of ​​the individual anomaly retention area and the area of ​​the currently effective defect area. If no usable corresponding reference area exists, then... It does not participate in the calculation of defect confidence values; the defect confidence values ​​are determined by the basic confidence state values.

[0128] If the final solder joint category is a normal solder joint, then the maximum defect probability value of the defect probability map, the solder joint category probability value corresponding to the normal solder joint, the boundary stability state of the solder grayscale region, and the individual residual state within the solder grayscale region are read within the solder joint region corresponding to the corrected position in the solder joint area. If the maximum defect probability value in the defect probability map is lower than the normal solder joint defect exclusion threshold determined by the verification samples, and there are no continuous abnormal regions within the solder grayscale region that are higher than the residual judgment threshold, then the solder joint category probability value corresponding to the normal solder joint is used as the defect confidence value of the normal solder joint; if the solder joint category probability points to a normal solder joint, but there are continuous abnormal regions within the solder grayscale region that are higher than the residual judgment threshold, then the defect confidence value of the normal solder joint is reduced, and this continuous abnormal region is written into the solder joint defect identification result as a region to be reviewed. The region to be reviewed refers to an image region that has abnormal grayscale performance but has not been determined as a valid defect region.

[0129] For solder joints ultimately classified as normal, the defect confidence value indicates the degree of confidence in classifying them as normal. If the maximum value of the defect probability map is lower than the normal solder joint defect exclusion threshold, the probability of the normal solder joint category is higher than that of other solder joint categories, the boundary of the solder grayscale area is stable, and the individual residual state within the solder does not show continuous abnormal areas, then the defect confidence value of the normal solder joint is relatively high. If the solder joint category probability points to a normal solder joint, but there are continuous abnormal areas within the solder grayscale area that exceed the residual judgment threshold, then the solder joint is not directly identified as a defective solder joint. Instead, the defect confidence value of the normal solder joint is lowered, and the continuous abnormal area is included as a region to be reviewed in the solder joint defect identification result.

[0130] The final generated defect confidence value, along with the chip number to be inspected, the name of the target X-ray image, the corrected location of the solder joint area, the final solder joint category, the effective defect area, the defect area, the defect center location, the defect circumference range, and the defect area percentage, are written into the solder joint defect identification result.

[0131] Before generating the solder joint defect identification results, the coordinates of the effective defect area are converted from the solder joint sample image coordinates back to the target X-ray image coordinates. During the coordinate conversion, based on the coordinates of the upper left corner of the clipping boundary, the scaling ratio, and the fill position from step two, an inverse coordinate conversion is performed on each contour point in the effective defect area. If the solder joint sample image has been scaled, the contour point coordinates are first restored according to the reciprocal of the scaling ratio; if the solder joint sample image has been filled, the horizontal and vertical offsets caused by the filling are deducted; then, the horizontal and vertical coordinates of the upper left corner of the clipping boundary in the target X-ray image are added to the contour point coordinates to obtain the coordinates of the effective defect area in the target X-ray image. The converted effective defect area is recorded together with the chip number to be inspected and the corrected position of the solder joint area.

[0132] After completing the above processing, solder joint defect identification results are generated. These results include the chip number to be inspected, the target X-ray image name, the corrected location of the solder joint area, the final solder joint category, the effective defect area, the defect area, the defect center location, the defect circumference, the defect area percentage, and the defect confidence value. The chip number to be inspected is obtained from the inspection batch information, the chip surface identification information, or the carrier number in step one; the target X-ray image name is obtained from the target X-ray image record in step one; the corrected location of the solder joint area is obtained from step two; the final solder joint category is determined by the positional relationship of the effective defect area and the solder joint category probability; the effective defect area is obtained from the initial defect area through boundary refinement, merging of duplicate results, and validity judgment; the defect area, defect center location, defect circumference, and defect area percentage are calculated from the effective defect area; and the defect confidence value is calculated from the defect probability map, the solder joint category probability, and the degree of grayscale anomaly.

[0133] In one specific implementation, the solder joint defect identification results are output in the form of a record table, a structured data file, or the results from the inspection system interface. Each solder joint area corresponds to at least one solder joint defect identification result. If multiple valid defect areas exist within the same solder joint area, each valid defect area records its corresponding defect area, defect center location, defect circumference, defect area percentage, and defect confidence value, and collectively corresponds to the same chip number under inspection, target X-ray image name, and corrected position of the solder joint area.

[0134] For example, the solder joint defect identification result includes the following fields: chip number to be inspected, target X-ray image name, solder joint row and column number, solder joint area correction position, final solder joint category, effective defect area contour point coordinates, defect area, defect center position, defect circumference range, defect area percentage, defect confidence value, and area to be reviewed marker. The solder joint row and column number is obtained from the solder joint layout information; the effective defect area contour point coordinates are obtained by inversely converting the solder joint sample image coordinates back to the target X-ray image coordinates; the area to be reviewed marker indicates a situation where the solder joint category probability points to a normal solder joint, but there are continuous abnormal areas within the solder grayscale area that exceed the residual judgment threshold.

[0135] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0136] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0137] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0140] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent identification of internal solder joint defects in chips based on X-ray images, characterized in that, include: Acquire the original X-ray image of the solder joint area of ​​the chip to be inspected, construct the target X-ray image based on the dark field image and the flat field image, and determine the actual location of the solder joint area based on the solder joint layout information; The corrected position of the solder joint area is determined based on the target X-ray image and the actual position of the solder joint area, and a sample image of the solder joint is obtained according to the corrected position of the solder joint area. Based on the solder joint sample image, a defect candidate region is constructed, and the solder joint sample image is input into the solder joint defect recognition network to obtain a defect probability map, solder joint category probability, and initial solder joint defect recognition result. Based on the initial solder joint defect identification results, the effective defect area is determined, and a co-location reference area is established based on the solder joint layout information. The co-location reference area, common grayscale reference map, and individual residual map are combined to distinguish the common grayscale components corresponding to the internal structure projection and the individual abnormal components corresponding to the current effective defect area. Then, the defect confidence value is determined based on the defect probability map, solder joint category probability, internal structure projection repetition state value, individual abnormal retention area, and multiple state values ​​to generate the solder joint defect identification result.

2. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 1, characterized in that: The process involves acquiring the original X-ray image of the solder joint area of ​​the chip under inspection, and reading the dark field image and the flat field image. These three images are then matched pixel-by-pixel according to the same pixel coordinates. Based on the dark field subtraction and flat field normalization relationship, a target X-ray image is constructed. Subsequently, the solder joint layout information is read from the packaging design file, inspection process file, or solder joint array coordinate table. The solder joint arrangement direction, row and column numbers, spacing between adjacent solder joints, individual solder joint design dimensions, and solder joint center design coordinates are extracted. Combined with the imaging magnification, the theoretical center position and theoretical boundary range of each solder joint area in the target X-ray image are calculated. Then, based on the actual center position and direction of the outer edge of the chip under inspection or the outer edge of the solder joint array, the lateral position correction, longitudinal position correction, and angle correction are calculated. The theoretical solder joint area is then subjected to coordinate transformation to determine the actual position of the solder joint area.

3. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 2, characterized in that: A local image is cropped from the actual location of the solder joint area. Solder grayscale regions that are distinct from the substrate background and continuously distributed are identified. The solder grayscale regions are then mapped to the design coverage area, internal abnormal grayscale, boundary continuity, connectivity between adjacent solder joints, and connection position relationship. Normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects are labeled. Among them, void defects correspond to local abnormal grayscale regions inside the solder joint, bridging defects correspond to continuous solder grayscale structures between adjacent solder joints, missing solder defects correspond to abnormal regions within the design coverage area that are not covered by solder grayscale regions, and cold solder joint defects correspond to discontinuous regions between solder grayscale regions and connection positions. For the above defect categories, defect contours are drawn along the boundaries of the corresponding abnormal regions. The target X-ray image, the actual location of the solder joint area, the solder joint category, and the defect location are written into the solder joint defect sample set.

4. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 3, characterized in that: The target X-ray image and the actual location of the solder joint area are read, and the pixel cropping range of the solder joint area is determined by combining the solder joint design size and imaging magnification. Then, noise suppression, local contrast enhancement, and edge grayscale enhancement are performed on the target X-ray image to form an enhanced target X-ray image. For each solder joint area, a temporary local image containing the solder joint and the adjacent substrate background is first cropped from the actual location of the solder joint area. Then, the solder grayscale judgment threshold is determined in the temporary local image based on the average grayscale value of the background area, the average grayscale value of the solder joint center area, and the grayscale display method. The solder grayscale area connected to the solder joint center area is extracted. Then, the outer boundary of the solder grayscale area is extracted, and the broken boundary and isolated boundary segments are corrected. The correction center coordinates and correction boundary range are calculated based on the corrected outer boundary to determine the correction position of the solder joint area. Using the correction center coordinates in the solder joint area correction position as the clipping center, and the correction boundary range and sample edge area as the clipping boundary, an initial solder joint sample image containing the complete solder joint area and the local substrate background is clipped. For solder joints that have been marked as bridging defects, the clipping boundary also includes the adjacent solder joint area connected to the current solder joint to form a continuous solder grayscale structure. Then, the initial solder joint sample image is sized and grayscale unified, and the defect contour points under the target X-ray image coordinates are converted into solder joint sample image coordinates according to the clipping boundary, scaling ratio and fill offset. The solder joint sample image, solder joint area correction position, solder joint category and converted defect position are written into the solder joint sample record.

5. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 4, characterized in that: Read the solder joint sample image, the solder joint area correction position, the solder joint category and the converted defect position. First, divide the solder grayscale area and the substrate background area in the solder joint sample image, and calculate the corresponding grayscale mean and grayscale standard deviation respectively. Subsequently, candidate defect regions are constructed based on the grayscale and positional relationships of different defects. Among them, voids correspond to continuous pixel regions within the solder grayscale region that deviate from the average grayscale value of the solder; bridging corresponds to continuous pixel regions within the interval between adjacent solder joints that simultaneously contact the grayscale regions of solder on both sides; missing solder corresponds to pixel regions within the designed coverage area of ​​the solder joint that are not covered by the solder grayscale region and whose area exceeds the allowable deviation; and cold solder corresponds to grayscale discontinuous regions where the width between the solder grayscale region and the connection position exceeds the allowable connection gap. Then, the above abnormal grayscale regions are merged or retained according to the shortest pixel distance, and the region boundary, region area, region center position, and the corresponding solder joint sample image of the candidate defect region are recorded. The solder joint defect recognition network takes solder joint sample images as input and uses convolution processing with different receptive ranges to extract local edges, small-sized abnormal grayscale, solder coverage, connectivity of adjacent solder joints, and overall grayscale distribution features. It then generates a defect probability map corresponding to the pixel positions of the solder joint sample image on the feature map after channel merging, as well as solder joint category probabilities for normal solder joints, void defects, bridging defects, missing solder defects, and cold solder joint defects. During recognition, the defect probability map is converted into a defect recognition region according to the defect probability threshold determined by the verification samples. The recognized defect region is then compared with the defect candidate region for overlap judgment, and either merged or retained as the initial defect region. Simultaneously, the initial solder joint category is determined by combining the solder joint category probability and the relationship between the initial defect region and the solder grayscale region, the interval between adjacent solder joints, the designed coverage of the solder joint, and the connection position. This generates an initial solder joint defect recognition result containing the solder joint sample image, the chip number to be inspected, the corrected position of the solder joint region, the initial solder joint category, the initial defect region, the defect probability map, and the solder joint category probability.

6. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 5, characterized in that: Read the initial solder joint defect identification results, including solder joint sample image, solder joint area correction position, initial solder joint category, initial defect area, defect probability map, and solder joint category probability; Subsequently, a boundary search region is set around the outer boundary of the initial defect region, and the intensity of grayscale change is calculated along the outer normal direction of the initial defect region. The position with the largest grayscale change intensity is used as the correction boundary point to form the initial correction contour. For the break position in the initial correction contour, contour points are supplemented according to the shortest pixel path. For the protruding contour segment formed by isolated noise, it is deleted according to the connection length, contour length and minimum effective defect area to obtain the correction defect contour. The defect contours are defined according to the location relationship of the defect categories: void defects are limited to the inside of the solder gray area, bridging defects are limited to the outer boundary of the continuous solder gray structure between adjacent solder joints, missing solder defects are limited to the position not covered by the solder gray area within the design coverage of the solder joint, and cold solder joint defects are limited to the discontinuous area between the solder gray area and the connection position.

7. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 6, characterized in that: For the same weld point sample image, if there are overlapping pixels, or if the shortest pixel distance is less than the boundary breakage distance and the corresponding initial weld point category is the same, the modified defect contours are merged, and the area enclosed by the merged contours is recorded as the effective defect area. Next, based on the minimum effective defect area and the degree of gray-scale anomaly, effective defect areas that do not meet the conditions are screened out. The final solder joint category is determined according to the relationship between the effective defect area and the solder gray-scale area, the interval between adjacent solder joints, the designed coverage area of ​​the solder joint, and the connection position. The defect area, defect center position, defect outer range, and defect area ratio of the effective defect area are calculated. The defect area ratio is selected as the solder gray-scale area, the designed coverage area of ​​the solder joint, the area of ​​the interval between adjacent solder joints, or the designed contact area as the solder joint reference area according to different defect categories.

8. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 7, characterized in that: The sources of gray-scale anomalies in the target X-ray image are classified into three cases: defects in the solder joint itself, projection of internal structure, and overlap of the two. Gray-scale components that appear repeatedly at the same relative position in multiple adjacent solder joint regions are defined as common gray-scale components, and gray-scale components that appear only in the current solder joint region and correspond to the solder gray-scale region, connection position, or adjacent solder joint interval region are defined as individual anomalous components. The system reads information such as the effective defect area, defect probability map, solder joint category probability, solder joint area correction position, solder grayscale area, substrate background area, connection position, and solder joint arrangement. It establishes relative position description data based on the positional offset and circumscribed dimension of the current effective defect area relative to the solder joint area correction center or bridging interval center. Then, it selects reference solder joint areas or reference interval areas along the same solder joint array direction, in the same row or column, with the same design size and within the effective imaging range. The relative position description data is mapped to each reference object to obtain a co-located reference area with the same relative position and size as the current effective defect area. Subsequently, according to the final solder joint category, it extracts the internal solder anomaly area corresponding to voids, the cross-interval connected area corresponding to bridging, the uncovered area within the design coverage range corresponding to missing solder, and the discontinuous connection position area corresponding to cold solder joints from the co-located reference area. It calculates the internal structure projection repetition state value based on the number of co-located grayscale anomaly areas, the average degree of relative position overlap, and the average degree of grayscale similarity, which characterizes the degree to which the current effective defect area recurs along the solder joint array direction.

9. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 8, characterized in that: When a usable co-positional reference area exists, a common gray-level reference map is further established using the median value of the gray-level of pixels at the same relative position in multiple co-positional reference areas. Then, the current effective defect area is compared pixel by pixel with the common gray-level reference map to generate an individual residual map that represents the degree of gray-level deviation of the current area relative to the array re-projection. The residual judgment threshold is jointly determined by the baseline residual of each co-located reference area relative to the common grayscale baseline image, the threshold increment corresponding to the internal structure projection repetition state value, and the threshold reduction corresponding to the category position conformity state value. The category position conformity state value is calculated according to whether the void is located inside the solder and does not connect to the substrate background, whether the bridging crosses the interval between adjacent solder joints and connects the solder on both sides, whether the missing solder is located within the design coverage area and is not covered by the solder, and whether the cold solder joint is located in the discontinuous area between the solder grayscale area and the connection position. Then, the continuous pixel area in the individual residual image that is greater than or equal to the residual judgment threshold is determined as the individual anomaly retention area. If there is no individual anomaly retention area and the internal structure projection repetition state value is higher than the projection repetition judgment threshold, the current effective defect area is marked as the projection influence area. If there is an individual anomaly retention area, the area is used as the confidence evaluation area; otherwise, the current effective defect area is used as the confidence evaluation area.

10. The intelligent identification method for internal solder joint defects of a chip based on X-ray images according to claim 9, characterized in that: Within the confidence evaluation area, the local defect probability state value, category separation state value, category location conformity state value, boundary stability state value, and grayscale individual state value are calculated respectively. The minimum value among these state values ​​is taken as the basic confidence state value. The basic confidence state value is then corrected by combining the internal structure projection repetition state value and the proportion of individual abnormal retention areas. Then, the defect confidence value is determined. For the case where the final solder joint category is a normal solder joint, the maximum value of the defect probability map, the normal solder joint category probability, the boundary stability state of the solder grayscale area, and the individual residual state inside the solder are also read. Continuous abnormal areas in the solder grayscale area that are higher than the residual judgment threshold are set as areas to be reviewed. Finally, the effective defect area is reverse-converted from the solder joint sample image coordinates back to the target X-ray image coordinates. The chip number to be inspected, the target X-ray image name, the corrected position of the solder joint area, the final solder joint category, the effective defect area, the defect area, the defect center position, the defect outer range, the defect area proportion, and the defect confidence value are written into the solder joint defect identification result.

Images