A remanufacturing chip solder ball detection method of multi-modal data analysis

Abstract

This invention discloses a multimodal data analysis method for detecting solder balls in remanufactured chips, belonging to the field of solder ball detection technology. It addresses the issues of reduced utilization and detection reliability in remanufactured chips. The method involves acquiring appearance image data, extracting the solder ball distribution array and its contour morphology, marking the solder ball distribution array and performing a 3D scan to obtain height data and spherical cap point cloud data. The method then constructs the array height coplanarity coefficient and curvature symmetry index, and classifies the geometric mismatch types of the solder ball distribution array accordingly. It accesses a solder ball composition library and obtains target alloying elements, extracts features from the solder ball composition spectral data, and performs comparative analysis to generate compositional comparison features. By verifying the consistency between the compositional comparison features and the geometric mismatch types, it determines whether the geometric anomalies are supported by material composition. When a conflict exists, a re-inspection prompt is generated, improving the reliability and stability of solder ball detection results in remanufactured chips.

Description

Technical Field

[0001] This invention relates to the field of solder ball detection technology, and more specifically, to a method for detecting solder balls in remanufactured chips using multimodal data analysis. Background Technology

[0002] With the continuous shortening of the product upgrade cycle, chips based on ball grid array (BGA) packaging are increasingly widely used in remanufacturing and reuse scenarios. Remanufacturing chips usually requires processes such as disassembly, cleaning, and re-balling. Among these processes, solder balls are key structures for realizing the electrical connection and mechanical support between the chip and the substrate. Their geometry, coplanarity, and material composition directly affect the soldering reliability and service life of the remanufactured chip. Therefore, accurate and reliable detection of the solder ball condition before chip reuse has become an important technical step in the remanufacturing process.

[0003] The existing technology has the following shortcomings: Currently, most existing technologies employ solder ball detection methods based on a single detection dimension, such as two-dimensional appearance image detection or independent three-dimensional height detection. These methods fail to reflect the inherent correlation between solder ball height coplanarity, spherical cap spatial morphology, and material composition. They also lack technical means to jointly analyze and verify the consistency of solder ball geometric anomalies and material composition differences. Consequently, the failure status of solder balls is judged solely based on appearance or geometric parameters, which can easily lead to misjudgments caused by process fluctuations, material aging, or detection noise. This reduces the utilization rate and detection reliability of remanufactured chips. Therefore, a multimodal data analysis-based solder ball detection method for remanufactured chips is proposed.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for detecting solder balls in remanufactured chips using multimodal data analysis. By employing a multimodal collaborative analysis mechanism that combines appearance image data, three-dimensional morphology data, and compositional spectral data, a geometric mismatch determination model and a compositional consistency verification model for the solder ball distribution array are constructed to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting solder balls in remanufactured chips based on multimodal data analysis, comprising the following steps: Step S1: When the chip to be reused is transported to the calibration area, the appearance image data of the chip to be reused is collected, the solder ball distribution array is extracted based on the appearance image data and the contour shape of the solder balls is detected, and the solder ball distribution array is screened and marked based on the contour shape of the solder balls. Step S2: Perform a 3D scan of the marked solder ball distribution array to obtain the height data of the marked solder ball distribution array and calculate the array height coplanarity coefficient. Detect the spherical cap point cloud data of the marked solder ball distribution array, perform curvature fitting processing on the spherical cap point cloud data to generate the curvature symmetry index, and analyze the geometric mismatch type of the marked solder ball distribution array in combination with the height coplanarity coefficient. Step S3: Identify the coplanar failure state based on the geometric mismatch type and determine whether to access the solder ball composition library. Retrieve the judgment result and select to stop detection or enter the solder ball composition library to obtain the target alloy element. Step S4: Detect the compositional spectrum data of the marked tin ball distribution array, identify the target alloying elements based on the compositional spectrum data and analyze the compositional comparison characteristics, and use the compositional comparison characteristics to verify and analyze the geometric mismatch type before determining whether to generate a re-inspection prompt.

[0007] In a preferred embodiment, in step S1, when the chip to be reused is transported to the calibration area, the appearance image data of the chip to be reused is acquired by an industrial camera. The appearance image data refers to a two-dimensional image of the surface of the chip to be reused. After the image data is converted to grayscale, a grayscale image is obtained. Each pixel in the grayscale image is taken as the pixel center in turn. A neighborhood region of the pixel center is constructed with a preset neighborhood radius as the length. The average grayscale value of the pixels in the neighborhood region is calculated to obtain the local background grayscale average value. Compare the gray value at the center of the pixel with the average gray value of its corresponding local background to filter candidate pixels for brightness, and then aggregate and connect the candidate pixels to obtain multiple connected pixel blocks. The size of the enclosing rectangle is calculated based on the coordinates of each connected pixel block. When the aspect ratio of the enclosing rectangle falls within the preset circular ratio range, the connected pixel block is determined as the solder ball region.

[0008] In a preferred embodiment, in step S1, the centroid coordinates of the solder ball region are calculated based on the pixel coordinates in the solder ball region; The horizontal and vertical coordinate values ​​of each centroid coordinate are used as sorting keys to sort the centroid coordinates, dividing the tin ball region into different row sets and column sets, and assigning array index numbers. Based on the adjacency relationship of array index numbers, the solder ball regions are combined into a solder ball distribution array; For the solder ball distribution array, extract its boundary pixels. A boundary pixel is a pixel that belongs to the solder ball distribution array and whose adjacent pixels include at least one that does not belong to the solder ball distribution array. The boundary point sequence is based on the boundary pixels arranged in an adjacent order and used as the outline shape of the solder ball. If a boundary pixel can return to the starting boundary point during traversal, the solder ball distribution array is not marked; otherwise, the solder ball distribution array is marked.

[0009] In a preferred embodiment, in step S2, the solder ball distribution array is scanned by a three-dimensional scanning device to obtain the height value of each solder ball in the solder ball distribution array, and the average value of each height value is used as the height data. The height data of the marked solder ball distribution array are subtracted pairwise to obtain the height difference value. The maximum value of the height difference value is taken as the array height coplanarity coefficient. For the marked solder ball distribution array, extract the cap point cloud data of each solder ball. The cap point cloud data refers to the set of three-dimensional points located in the top region of the solder ball. The top region of the solder ball refers to the three-dimensional space region corresponding to a preset height ratio extending downward from the solder ball height as the reference height.

[0010] In a preferred embodiment, in step S2, based on the coordinates of each three-dimensional point in the spherical cap point cloud data, the maximum and minimum coordinate values ​​of each point on the first coordinate axis in the horizontal direction are statistically analyzed, and the main distribution axis is calculated. The coordinates of each three-dimensional point are divided into the first point cloud sub-region or the second point cloud sub-region using the main distribution axis; The coordinates of the three-dimensional points in each cloud sub-region are sorted from smallest to largest according to their coordinate values ​​on the corresponding coordinate axes of the main distribution axis to obtain the point sequence.

[0011] In a preferred embodiment, in step S2, two adjacent three-dimensional points are selected sequentially in the point sequence, the difference between their height values ​​is calculated, and the difference between their coordinate values ​​on the corresponding coordinate axes of the main distribution axis is calculated. The ratio of the difference in height values ​​to the difference in coordinate values ​​is taken as the height change rate of the adjacent point pair. The curvature symmetry index is calculated based on the height change rate. The array height coplanarity coefficient and curvature symmetry index are compared with the preset height coplanarity threshold and preset curvature symmetry threshold, respectively, and the geometric mismatch types are divided into coplanar mismatch, morphological mismatch and composite mismatch.

[0012] In a preferred embodiment, in step S3, when the geometric mismatch type is identified as coplanar mismatch or composite mismatch, the corresponding solder ball distribution array is determined to be in a coplanar failure state. When the geometric mismatch type is identified as a topographic mismatch, the corresponding solder ball distribution array is determined to be not in a coplanar damaged state. The current geometric anomaly can be judged by the geometric detection results, and the current detection process ends. After the solder ball distribution array is determined to be in a state of coplanar failure, the target alloy element is obtained by retrieving the solder ball composition library; The solder ball composition library is a structured data set consisting of reference solder ball sample data prepared under standard process conditions, including target alloying elements, elemental characteristic spectral lines, and corresponding deviation fluctuation thresholds for different solder systems; The target alloying element refers to the constituent element that participates in solder ball welding and coplanar construction in the solder ball system.

[0013] In a preferred embodiment, in step S4, the compositional spectral data of the marked solder ball distribution array is detected. The compositional spectral data is a spectral intensity vector obtained by sampling within a set wavelength value at a fixed wavelength resolution, including the wavelength value and light intensity value of the solder ball material in the excited state. The target alloy element is retrieved from the tin ball composition library, and the corresponding elemental characteristic spectral lines are called to determine the characteristic wavelength range of the target alloy element. Elemental characteristic spectral lines refer to the set of characteristic spectral lines with definite wavelength positions produced by a target alloying element under controlled energy excitation conditions; The light intensity values ​​falling within the characteristic wavelength range are extracted from the compositional spectral data and accumulated to obtain the characteristic light intensity values ​​of the target alloy element.

[0014] In a preferred embodiment, in step S4, the characteristic light intensity value is compared with the reference light intensity value of the target alloy element under standard process conditions recorded in the solder ball composition library. The reference light intensity value refers to the cumulative light intensity of the target alloying element within its characteristic wavelength range in a reference tin ball sample prepared under standard process conditions. The content deviation value of the target alloying element is calculated based on the difference between the characteristic light intensity value and the reference light intensity value, and the content deviation value is used as the composition comparison characteristic.

[0015] In a preferred embodiment, in step S4, the compositional characteristics of each target alloying element are compared with the corresponding deviation fluctuation thresholds in the tin ball composition library to perform a verification analysis of the geometric mismatch type. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element exceed the corresponding deviation fluctuation threshold, it is determined that there is consistency between the geometric mismatch type and the material composition anomaly. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element do not exceed the corresponding deviation fluctuation threshold, a conflict is determined between the geometric determination result and the compositional determination result. When there is a conflict between the geometric determination result and the composition determination result, a re-inspection prompt is generated. The re-inspection prompt refers to the detection feedback information generated when the verification analysis results between the geometric mismatch type and the composition comparison characteristics show that there is a conflict between the two. Conversely, no retest prompt will be generated, and the current test result will be output as the final test result.

[0016] The technical effects and advantages of this invention are as follows: This invention acquires appearance image data, extracts the solder ball distribution array and its contour morphology, marks the solder ball distribution array and performs 3D scanning to obtain height data and spherical cap point cloud data, constructs the array height coplanarity coefficient and curvature symmetry index respectively, and classifies and analyzes the geometric mismatch type of the solder ball distribution array accordingly. It accesses the solder ball composition library and obtains the target alloy elements, performs feature extraction and comparative analysis on the solder ball composition spectral data, generates compositional comparison features, and determines whether the geometric anomaly has material composition support by verifying the consistency between the compositional comparison features and the geometric mismatch type. When there is a conflict between the two, a re-inspection prompt is generated, which effectively reduces the risk of misjudgment caused by a single detection dimension and improves the reliability and stability of the solder ball detection results of remanufactured chips. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of a multimodal data analysis-based method for detecting solder balls in remanufactured chips according to the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the steps of a remanufactured chip solder ball detection method based on multimodal data analysis according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention acquires appearance image data, extracts the distribution array of solder balls and its contour morphology, marks the solder ball distribution array and performs a 3D scan to obtain height data and spherical cap point cloud data, constructs the array height coplanarity coefficient and curvature symmetry index respectively, and classifies and analyzes the geometric mismatch type of the solder ball distribution array accordingly. It accesses the solder ball composition library and obtains the target alloy elements, performs feature extraction and comparative analysis on the spectral data of the solder ball composition, generates compositional comparison features, and determines whether the geometric anomaly has material composition support by verifying the consistency between the compositional comparison features and the geometric mismatch type. When there is a conflict between the two, a re-inspection prompt is generated, effectively reducing the risk of misjudgment caused by a single detection dimension.

[0021] Example 1, such as Figures 1 to 2As shown, a method for detecting solder balls in remanufactured chips using multimodal data analysis includes the following steps: Step S1: When the chip to be reused is transported to the calibration area, the appearance image data of the chip to be reused is collected, the solder ball distribution array is extracted based on the appearance image data and the contour shape of the solder balls is detected, and the solder ball distribution array is screened and marked based on the contour shape of the solder balls. Step S2: Perform a 3D scan of the marked solder ball distribution array to obtain the height data of the marked solder ball distribution array and calculate the array height coplanarity coefficient. Detect the spherical cap point cloud data of the marked solder ball distribution array, perform curvature fitting processing on the spherical cap point cloud data to generate the curvature symmetry index, and analyze the geometric mismatch type of the marked solder ball distribution array in combination with the height coplanarity coefficient. Step S3: Identify the coplanar failure state based on the geometric mismatch type and determine whether to access the solder ball composition library. Retrieve the judgment result and select to stop detection or enter the solder ball composition library to obtain the target alloy element. Step S4: Detect the compositional spectrum data of the marked tin ball distribution array, identify the target alloying elements based on the compositional spectrum data and analyze the compositional comparison characteristics, and use the compositional comparison characteristics to verify and analyze the geometric mismatch type before determining whether to generate a re-inspection prompt.

[0022] The specific implementation is as follows: In step S1, during the remanufacturing chip solder ball inspection process, the solder balls may undergo asynchronous changes in geometric appearance and material composition. The spatial distribution and overall arrangement of the solder balls are obtained through appearance images. Then, three-dimensional geometric inspection is performed based on the solder ball distribution array to identify the high consistency and morphological deviation of the array layer. When there are abnormalities in the geometric inspection results, component inspection is introduced to verify the geometric judgment results, thereby establishing a correlation judgment mechanism between geometric information and component information, avoiding misjudgments caused by single-dimensional inspection, and providing a basis for whether to trigger re-inspection.

[0023] When the chip to be reused is transported to the calibration area, the appearance image acquisition operation is triggered. The appearance image data of the chip to be reused is acquired by an industrial camera. The appearance image data refers to the two-dimensional image of the surface of the chip to be reused. After grayscale processing of the appearance image data, a grayscale image is obtained. The grayscale image includes the grayscale value corresponding to each pixel. Each pixel in the grayscale image is taken as the pixel center in turn, and a neighborhood region of the pixel center is constructed with a preset neighborhood radius as the length. The average value of the pixel grayscale value in the neighborhood region is calculated to obtain the local background grayscale mean. The gray value at the center of the pixel is compared with the average gray value of the corresponding local background. When the gray value is greater than the average gray value of the local background, the center of the pixel is selected as a candidate pixel for brightness. The candidate pixels for brightness are aggregated and connected to obtain multiple connected pixel blocks; Calculate the size of the bounding rectangle for each connected pixel block. Specifically, obtain the coordinate values ​​of the pixels in the connected pixel block in the horizontal and vertical directions. In the horizontal coordinate, determine the maximum and minimum horizontal coordinate values ​​and calculate the difference between them as the size of the connected pixel block in the horizontal direction. In the vertical coordinate, determine the maximum and minimum vertical coordinate values ​​and calculate the difference between them as the size of the connected pixel block in the vertical direction. Use the dimensions of the connected pixel block in the vertical and horizontal directions as the dimensions of the bounding rectangle of the connected pixel block; When the aspect ratio of the enclosing rectangle falls within the preset circular ratio range, the corresponding connected pixel block is determined as the solder ball region.

[0024] It should be explained that an industrial camera is a high-resolution imaging device used in industrial inspection scenarios, capable of acquiring two-dimensional image data; grayscale processing refers to the process of converting the color pixel information in the appearance image data into single-channel grayscale values; the preset neighborhood radius can be set according to the spatial resolution of the industrial camera and the diameter of the solder ball; the preset circular ratio range can be set according to the projection shape of the solder ball in the two-dimensional image and the proportion distribution of the bounding rectangle of historical normal samples.

[0025] After obtaining the solder ball regions, the centroid coordinates of each solder ball region are calculated. Specifically, the horizontal coordinate values ​​of each pixel are summed and divided by the total number of pixels to obtain the average horizontal coordinate value of the solder ball region; the vertical coordinate values ​​of each pixel are summed and divided by the total number of pixels to obtain the average vertical coordinate value of the solder ball region. The average horizontal coordinates and the average vertical coordinates are combined to form the centroid coordinates of the tin ball region; Using the horizontal coordinate values ​​of each centroid as sorting keys, all centroid coordinates are sorted in ascending order of horizontal coordinate values ​​to obtain a horizontal sorting sequence. In the horizontal sorting sequence, two adjacent centroid coordinates are selected in turn, and the absolute value of the difference between their horizontal coordinate values ​​is calculated to obtain the horizontal adjacent spacing. If the horizontal adjacent spacing is less than the preset adjacent spacing threshold, the corresponding solder ball area is assigned to the same column set. Conversely, the tin ball region is divided into different column sets.

[0026] Similarly, using the vertical coordinate values ​​of each centroid as sorting keys, all centroid coordinates are sorted in ascending order of vertical coordinate values ​​to obtain a vertical sorting sequence. In the vertically sorted sequence, two adjacent centroid coordinates are selected in turn, and the absolute value of the difference between their vertical coordinate values ​​is calculated to obtain the vertical adjacent spacing. If the vertical adjacent spacing is less than the preset adjacent spacing threshold, the corresponding solder ball area will be grouped into the same row set. Conversely, the tin ball region is divided into different row sets; The row and column positions of a solder ball region are determined based on the row set and column set to which it belongs, and array index numbers are assigned to each solder ball region accordingly. Solder ball regions with adjacent array index numbers are combined into a solder ball distribution array, while solder ball regions that do not form a row and column adjacent relationship with other array index numbers are treated as independent solder ball distribution arrays. For the solder ball distribution array, extract its boundary pixels and form the outline shape of the solder ball. The boundary pixel is a pixel that belongs to the solder ball distribution array and whose adjacent pixels have at least one that does not belong to the solder ball distribution array. Select the boundary pixel with the smallest horizontal coordinate and the smallest vertical coordinate under that horizontal coordinate as the starting boundary point. With the starting boundary point as the current point, determine the next boundary pixel among its adjacent pixels, and so on to form a sequence of boundary points arranged in adjacency order. The boundary point sequence serves as the outline shape of the solder balls in the solder ball distribution array. When the boundary pixel points can return to the starting boundary point during the traversal process, the boundary point sequence is considered to be completed. When the boundary pixel cannot return to the starting boundary point during the traversal, it is determined that the overall outline of the solder ball distribution array is incomplete, and the solder ball distribution array is marked.

[0027] It should be noted that the preset adjacent spacing threshold can be set based on the statistical results of the historical spacing between adjacent solder ball areas.

[0028] In step S2, the solder ball distribution array is scanned using a three-dimensional scanning device to obtain the height value of each solder ball vertex in the solder ball distribution array, and each height value is used as height data; Based on the height data of each marked solder ball distribution array, the height data of each marked solder ball distribution array are subtracted pairwise to obtain the height difference value. The maximum value of the height difference value is taken as the array height coplanarity coefficient, which reflects the degree of deviation of the marked solder ball distribution array in the height direction. Furthermore, during the 3D scanning process, for the marked solder ball distribution array, the cap point cloud data of each solder ball is extracted. The cap point cloud data refers to the set of 3D points located in the top region of the solder ball. Among them, the top area of ​​the solder ball refers to the three-dimensional space area corresponding to the preset height ratio extending downward from the reference height, with the solder ball height value as the reference height. Curvature fitting processing is performed on the spherical cap point cloud data. Specifically, for the coordinates of each three-dimensional point in the spherical cap point cloud data, the maximum and minimum coordinate values ​​on the horizontal coordinate axis are counted, and the difference between the two is calculated. The horizontal coordinate axis with the largest difference is determined as the main distribution axis of the spherical cap point cloud in the horizontal direction. Using the coordinate axis corresponding to the main distribution axis as a reference, the coordinate values ​​of each three-dimensional point on the coordinate axis are compared with the midpoint position of the maximum and minimum coordinate values ​​on the coordinate axis. When the coordinate value is greater than the midpoint position, the corresponding three-dimensional point is assigned to the first point cloud sub-region. When the coordinate value is less than the midpoint position, the corresponding three-dimensional point is assigned to the second point cloud sub-region. Thus, the spherical crown point cloud data is divided into point cloud sub-regions located on both sides of the main distribution axis and are symmetrical to each other. After dividing the spherical crown point cloud into the first point cloud sub-region and the second point cloud sub-region, the three-dimensional point coordinates in each point cloud sub-region are sorted in ascending order according to their coordinate values ​​on the corresponding coordinate axis of the main distribution axis to obtain the point sequence. In the point sequence, select the coordinates of two adjacent three-dimensional points in sequence, calculate the difference between their height values, and calculate the difference between their coordinate values ​​on the corresponding coordinate axes of the main distribution axis. The ratio of the difference in height values ​​to the difference in coordinate values ​​is taken as the height change rate of the adjacent point pair. The average height change rate of each height change rate is calculated to obtain the average height change rate of the cloud sub-region at that point. After obtaining the average height change rate of the first and second cloud sub-regions respectively, the difference between the two is calculated and the absolute value is taken to obtain the curvature symmetry index. When the array height coplanarity coefficient is greater than the preset height coplanarity threshold and the curvature symmetry index is less than the preset curvature symmetry threshold, the geometric mismatch type is determined to be coplanar mismatch. When the array height coplanarity coefficient is less than or equal to the preset height coplanarity threshold and the curvature symmetry index is greater than or equal to the preset curvature symmetry threshold, the geometric mismatch type is determined to be a topographic mismatch. When the array height coplanarity coefficient is greater than the preset height coplanarity threshold and the curvature symmetry index is greater than or equal to the preset curvature symmetry threshold, the geometric mismatch type is determined to be a composite mismatch.

[0029] When the array height coplanarity coefficient is less than or equal to the preset height coplanarity threshold, and the curvature symmetry index is less than the preset curvature symmetry threshold, it is determined that the current solder ball distribution array has not triggered the geometric mismatch type classification.

[0030] Among them, when the geometric mismatch type is coplanar mismatch, it indicates that the marked solder ball distribution array has an anomaly in the overall height coplanarity, while the overall shape of the solder ball cap remains consistent; when the geometric mismatch type is composite mismatch, it indicates that the marked solder ball distribution array has anomalies in both the overall height coplanarity and the symmetry of the cap shape; when the geometric mismatch type is morphological mismatch, it indicates that the overall height coplanarity of the marked solder ball distribution array is basically normal, but the shape of the solder ball cap has obvious differences at symmetrical positions.

[0031] It should be noted that the 3D scanning device is a confocal scanning device used to acquire 3D topographic data of an object surface; the preset height ratio can be set according to the nominal height of the solder ball or the scanning resolution; the preset height coplanar threshold can be set according to the height dispersion of a normal solder ball array of the same model; the preset curvature symmetry threshold can be set according to the distribution range of the curvature symmetry index of normal solder balls in historical detection data.

[0032] In step S3, after completing the geometric mismatch type analysis of the marked solder ball distribution array in step S2, the coplanar state of the solder ball distribution array is determined based on the geometric mismatch type.

[0033] When the geometric mismatch type is identified as coplanar mismatch or composite mismatch, it indicates that there is an overall anomaly in the array-level high consistency dimension of the solder ball distribution array. This type of anomaly reflects that the array coplanarity has been disrupted, and the corresponding solder ball distribution array is judged to be in a state of coplanar disruption.

[0034] When the geometric mismatch is identified as a morphological mismatch, the geometric anomaly manifests only as a localized morphological deviation of a single or a small number of solder balls. This anomaly reflects a change in the overall high uniformity of the array and does not have the significance of determining array-level coplanar failure. In this case, the corresponding solder ball distribution array is determined to be not in a coplanar failure state, and the current geometric anomaly is considered to be able to complete the quality assessment through geometric inspection results. Further material-side analysis is not necessary, thus ending the current chip inspection process.

[0035] After the solder ball distribution array is determined to be in a coplanar failure state, characterizing this type of array-level coplanar anomaly may be related to the evolution of alloy composition or changes in material properties of the solder balls during multiple reflows and repairs. Relying solely on geometric information is insufficient to accurately determine the cause of the anomaly. Therefore, a composition detection access command is generated to retrieve the target alloying elements from the solder ball composition library for further analysis and verification of the coplanar failure state.

[0036] The solder ball composition library is a structured dataset consisting of reference solder ball sample data prepared under standard process conditions. It includes target alloying elements, elemental characteristic spectral lines, and corresponding deviation fluctuation thresholds for different solder systems, which are used to characterize the material composition distribution characteristics of solder balls under normal manufacturing conditions.

[0037] The target alloying element refers to the constituent element that participates in solder ball welding and coplanar construction in the solder ball system. The target alloying element is determined according to the type of solder ball and includes elements such as tin, silver, and copper.

[0038] By accessing the solder ball composition library and retrieving the target alloying elements, an objective and comparable benchmark is provided for subsequent quantitative identification and deviation analysis of the actual composition state of the solder balls based on spectral data.

[0039] In step S4, the compositional spectral data of the marked solder ball distribution array is detected. This involves applying a preset energy excitation to the marked solder ball distribution array using a spectrometer and acquiring the compositional spectral data of the solder ball material in the excited state. The compositional spectral data includes wavelength values ​​and light intensity values. The wavelength values ​​characterize the characteristic emission or absorption positions of different alloying elements in the material, and the light intensity values ​​characterize the radiation intensity at the corresponding wavelength. Specifically, the compositional spectral data is represented as a spectral intensity vector obtained by sampling at a fixed wavelength resolution within a set wavelength range, reflecting the energy spectrum distribution characteristics of the solder ball material at the elemental level.

[0040] It should be noted that the spectroscopic detection instrument is an analytical device used to acquire spectral data of the composition of tin ball materials. It achieves quantitative characterization of the target alloying elements of tin balls through controlled energy excitation and high-resolution spectral acquisition.

[0041] For each target alloying element retrieved from the solder ball composition library, the corresponding elemental characteristic spectral information is called to determine the characteristic wavelength range of the target alloying element. Subsequently, the light intensity values ​​falling within the characteristic wavelength range are extracted from the compositional spectral data and accumulated to obtain the characteristic light intensity value of the target alloying element. The characteristic light intensity value is used to characterize the energy spectrum contribution level of the target alloying element in the current solder ball material. Its value reflects the relative content level of the target alloying element. The larger the characteristic light intensity value, the higher the relative content of the target alloying element in the solder ball material.

[0042] After calculating the characteristic light intensity value of the target alloy element, the characteristic light intensity value is compared with the reference light intensity value of the target alloy element under standard process conditions recorded in the solder ball composition library. The content deviation value of the target alloy element is calculated based on the difference between the characteristic light intensity value and the reference light intensity value. The content deviation value is used as the composition comparison feature. The composition comparison feature is used to quantify the degree of deviation of the target alloy element in the current solder ball from the standard process state. The larger the value, the more obvious the deviation between the actual composition state of the target alloy element and the standard composition state.

[0043] It should be noted that the elemental characteristic spectral lines refer to the set of characteristic spectral lines with definite wavelength positions generated by the target alloying element under controlled energy excitation conditions. They are characterized by wavelength values ​​as a parameter and are used to characterize the spectral positions of different target alloying elements in the compositional spectral data. They are pre-stored in the solder ball composition library and are obtained from the statistical results of spectral detection of reference solder ball samples with known compositions under standard process conditions. They are used to determine the characteristic wavelength range corresponding to the target alloying element. The reference light intensity value refers to the cumulative light intensity of the target alloying element in its characteristic wavelength range in the reference solder ball sample prepared under standard process conditions. It is used to characterize the standard energy spectrum contribution level of the target alloying element under normal production conditions. It is recorded in the solder ball composition library and is obtained by performing compositional spectral detection on multiple batches of reference solder ball samples and statistically averaging the light intensity values ​​in the corresponding characteristic wavelength ranges.

[0044] By comparing the compositional characteristics of each target alloying element with the corresponding deviation fluctuation thresholds in the tin ball composition library, the geometric mismatch type is verified and analyzed to determine whether the anomaly underlying the geometric mismatch type can be supported at the material composition level. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element exceed the corresponding deviation fluctuation threshold, it is determined that there is consistency between the geometric mismatch type and the material composition anomaly. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element do not exceed the corresponding deviation fluctuation threshold, a conflict is determined between the geometric determination result and the compositional determination result.

[0045] When there is a conflict between the geometric determination result and the composition determination result, a re-inspection prompt is generated. The re-inspection prompt refers to the detection feedback information generated when the verification analysis results between the geometric mismatch type and the composition comparison characteristics show that there is a conflict between the two. It is used to indicate that the abnormal cause of the current solder ball distribution array cannot be consistently explained by the existing detection results, and guides the subsequent detection process to perform repeated detection on the current chip to avoid misjudgment caused by a single detection dimension. Conversely, no retest prompt will be generated, and the current test result will be output as the final test result.

[0046] It should be noted that the deviation fluctuation threshold refers to the maximum allowable range of fluctuation in the content deviation of the target alloy element relative to the reference light intensity value during actual manufacturing and use. It is used to determine whether the composition state of the solder ball material is still within the stable process range. It is preset by the solder ball composition library. The setting is based on the statistical distribution results of the content deviation obtained after performing composition spectral detection on multiple batches of reference solder ball samples within the standard process window. It is used to determine the composition comparison characteristics.

[0047] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0048] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0050] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0051] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting solder balls in remanufactured chips using multimodal data analysis, characterized in that: Includes the following steps: Step S1: When the chip to be reused is transported to the calibration area, the appearance image data of the chip to be reused is collected, the solder ball distribution array is extracted based on the appearance image data and the contour shape of the solder balls is detected, and the solder ball distribution array is screened and marked based on the contour shape of the solder balls. Step S2: Perform a 3D scan of the marked solder ball distribution array to obtain the height data of the marked solder ball distribution array and calculate the array height coplanarity coefficient. Detect the spherical cap point cloud data of the marked solder ball distribution array, perform curvature fitting processing on the spherical cap point cloud data to generate the curvature symmetry index, and analyze the geometric mismatch type of the marked solder ball distribution array in combination with the height coplanarity coefficient. Step S3: Identify the coplanar failure state based on the geometric mismatch type and determine whether to access the solder ball composition library. Retrieve the judgment result and select to stop detection or enter the solder ball composition library to obtain the target alloy element. Step S4: Detect the compositional spectrum data of the marked tin ball distribution array, identify the target alloying elements based on the compositional spectrum data and analyze the compositional comparison characteristics, and use the compositional comparison characteristics to verify and analyze the geometric mismatch type before determining whether to generate a re-inspection prompt.

2. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 1, characterized in that: In step S1, when the chip to be reused is transported to the calibration area, the appearance image data of the chip to be reused is acquired by an industrial camera. The appearance image data refers to a two-dimensional image of the surface of the chip to be reused. After the image data is converted to grayscale, a grayscale image is obtained. Each pixel in the grayscale image is taken as the pixel center in turn. A neighborhood region of the pixel center is constructed with a preset neighborhood radius as the length. The average grayscale value of the pixels in the neighborhood region is calculated to obtain the local background grayscale average value. Compare the gray value at the center of the pixel with the average gray value of its corresponding local background to filter candidate pixels for brightness, and then aggregate and connect the candidate pixels to obtain multiple connected pixel blocks. The size of the enclosing rectangle is calculated based on the coordinates of each connected pixel block. When the aspect ratio of the enclosing rectangle falls within the preset circular ratio range, the connected pixel block is determined as the solder ball region.

3. The method for detecting solder balls in remanufactured chips based on multimodal data analysis according to claim 2, characterized in that: In step S1, the centroid coordinates of the solder ball region are calculated based on the pixel coordinates in the solder ball region; The horizontal and vertical coordinate values ​​of each centroid coordinate are used as sorting keys to sort the centroid coordinates, dividing the tin ball region into different row sets and column sets, and assigning array index numbers. Based on the adjacency relationship of array index numbers, the solder ball regions are combined into a solder ball distribution array; For the solder ball distribution array, extract its boundary pixels. A boundary pixel is a pixel that belongs to the solder ball distribution array and whose adjacent pixels include at least one that does not belong to the solder ball distribution array. The boundary point sequence is based on the boundary pixels arranged in an adjacent order and used as the outline shape of the solder ball. If a boundary pixel can return to the starting boundary point during traversal, the solder ball distribution array is not marked; otherwise, the solder ball distribution array is marked.

4. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 1, characterized in that: In step S2, the solder ball distribution array is scanned using a three-dimensional scanning device to obtain the height value of each solder ball in the solder ball distribution array, and the average value of each height value is used as the height data. The height data of the marked solder ball distribution array are subtracted pairwise to obtain the height difference value. The maximum value of the height difference value is taken as the array height coplanarity coefficient. For the marked solder ball distribution array, extract the cap point cloud data of each solder ball. The cap point cloud data refers to the set of three-dimensional points located in the top region of the solder ball. The top region of the solder ball refers to the three-dimensional space region corresponding to a preset height ratio extending downward from the solder ball height as the reference height.

5. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 4, characterized in that: In step S2, based on the coordinates of each three-dimensional point in the spherical cap point cloud data, the maximum and minimum coordinate values ​​of each point on the first coordinate axis in the horizontal direction are counted, and the main distribution axis is calculated. The coordinates of each three-dimensional point are divided into the first point cloud sub-region or the second point cloud sub-region using the main distribution axis; The coordinates of the three-dimensional points in each cloud sub-region are sorted from smallest to largest according to their coordinate values ​​on the corresponding coordinate axes of the main distribution axis to obtain the point sequence.

6. The method for detecting solder balls in remanufactured chips based on multimodal data analysis according to claim 5, characterized in that: In step S2, in the point sequence, two adjacent three-dimensional points are selected in sequence, the difference between their height values ​​is calculated, and the difference between their coordinate values ​​on the corresponding coordinate axes of the main distribution axis is calculated. The ratio of the difference in height values ​​to the difference in coordinate values ​​is taken as the height change rate of the adjacent point pair. The curvature symmetry index is calculated based on the height change rate. The array height coplanarity coefficient and curvature symmetry index are compared with the preset height coplanarity threshold and preset curvature symmetry threshold, respectively, and the geometric mismatch types are divided into coplanar mismatch, morphological mismatch and composite mismatch.

7. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 6, characterized in that: In step S3, when the geometric mismatch type is identified as coplanar mismatch or composite mismatch, the corresponding solder ball distribution array is determined to be in a coplanar failure state. When the geometric mismatch type is identified as a topographic mismatch, the corresponding solder ball distribution array is determined to be not in a coplanar damaged state. The current geometric anomaly can be judged by the geometric detection results, and the current detection process ends. After the solder ball distribution array is determined to be in a state of coplanar failure, the target alloy element is obtained by retrieving the solder ball composition library; The solder ball composition library is a structured data set consisting of reference solder ball sample data prepared under standard process conditions, including target alloying elements, elemental characteristic spectral lines, and corresponding deviation fluctuation thresholds for different solder systems; The target alloying element refers to the constituent element that participates in solder ball welding and coplanar construction in the solder ball system.

8. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 7, characterized in that: In step S4, the compositional spectral data of the marked solder ball distribution array is detected. The compositional spectral data is sampled at a fixed wavelength resolution within a set wavelength range to obtain a spectral intensity vector, which includes the wavelength value and light intensity value of the solder ball material in the excited state. The target alloy element is retrieved from the tin ball composition library, and the corresponding elemental characteristic spectral lines are called to determine the characteristic wavelength range of the target alloy element. Elemental characteristic spectral lines refer to the set of characteristic spectral lines with definite wavelength positions produced by a target alloying element under controlled energy excitation conditions; The light intensity values ​​falling within the characteristic wavelength range are extracted from the compositional spectral data and accumulated to obtain the characteristic light intensity values ​​of the target alloy element.

9. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 8, characterized in that: In step S4, the characteristic light intensity value is compared with the reference light intensity value of the target alloy element under standard process conditions recorded in the solder ball composition library; The reference light intensity value refers to the value of the target alloying element in a reference solder ball sample prepared under standard process conditions. The cumulative light intensity within its characteristic wavelength range; The content deviation value of the target alloying element is calculated based on the difference between the characteristic light intensity value and the reference light intensity value, and the content deviation value is used as the composition comparison characteristic.

10. The method for detecting solder balls in remanufactured chips using multimodal data analysis according to claim 9, characterized in that: In step S4, the compositional characteristics of each target alloying element are compared with the corresponding deviation fluctuation thresholds in the tin ball composition library to verify and analyze the geometric mismatch type. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element exceed the corresponding deviation fluctuation threshold, it is determined that there is consistency between the geometric mismatch type and the material composition anomaly. When the geometric mismatch type is coplanar mismatch or complex mismatch, and the compositional comparison characteristics of the target alloying element do not exceed the corresponding deviation fluctuation threshold, a conflict is determined between the geometric determination result and the compositional determination result. When there is a conflict between the geometric determination result and the composition determination result, a re-inspection prompt is generated. The re-inspection prompt refers to the detection feedback information generated when the verification analysis results between the geometric mismatch type and the composition comparison characteristics show that there is a conflict between the two. Conversely, no retest prompt will be generated, and the current test result will be output as the final test result.

Images