A background schlieren-based electronic package warpage detection method and system

Abstract

The present application belongs to the technical field of electronic packaging warping detection, and discloses an electronic packaging warping detection method and system based on background speckle, which comprises the following steps: (1) projecting a phase shift stripe and a gray code image to a sample to be measured under thermal air flow disturbance, and synchronously shooting an image of the sample to be measured and a background speckle image as a measurement image; (2) based on the background speckle image under no thermal air flow disturbance and the background speckle image in the measurement image, a thermal air flow displacement field is calculated by using a cross-correlation algorithm, a displacement field in a predetermined direction is obtained by superimposing two-dimensional projection vectors based on the thermal air flow displacement field, and then the image of the sample to be measured in the measurement image is pixel by pixel remapped to obtain a corrected image, and then the warping deformation information of the sample to be measured is obtained. The present application effectively compensates the influence caused by thermal refraction, so that the measurement result is more accurate and reliable, and provides a favorable guarantee for high-precision detection of electronic packaging structures.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of electronic package warpage detection, and more particularly relates to an electronic package warpage detection method and system based on background speckle. BACKGROUND

[0002] Packaging technology is widely used in the integrated circuit industry, but the temperature change in the electronic packaging process will cause the warpage deformation of the packaging structure, leading to reliability problems such as chip cracking and solder fatigue failure, so it is extremely critical to detect the warpage deformation of the packaging structure.

[0003] Traditional contact detection methods, such as micro three-coordinate measuring machines, cannot meet the requirements of high efficiency, precision and non-destructive testing in the microelectronic manufacturing industry. Among many non-contact measurement methods, such as confocal microscopes, white light interferometers, digital image correlation and digital fringe projection, digital fringe projection technology is widely used in high-precision detection of packaging structure three-dimensional topography due to its advantages of full-field measurement, fast speed and simple structure.

[0004] However, in the thermal reliability measurement of the packaging structure, the heat source will change the temperature distribution of the air in the measurement environment, and then affect the density and refractive index of the air. When the light passes through these areas with temperature gradient, the light will be bent due to the different refractive indices of the air layers with different temperatures, resulting in distortion of the collected images and reducing the measurement accuracy and reliability of the digital fringe projection technology in complex temperature environment, which is difficult to meet the high-precision detection requirements of the packaging structure. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides an electronic package warpage detection method and system based on background speckle, which aims to solve the problem of low warpage detection precision of electronic package in complex temperature environment.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, an electronic package warpage detection method based on background speckle is provided, which comprises the following steps:

[0007] (1) Projecting phase shift fringes and Gray code images on the to-be-measured sample under the disturbance of hot air flow, and synchronously shooting the images of the to-be-measured sample and the background speckle images as measurement images;

[0008] (2) Based on the background speckle images under no hot air flow disturbance and the background speckle images in the measurement images, the hot air flow displacement field is calculated by using the cross-correlation algorithm, the displacement field in the predetermined direction is obtained by two-dimensional projection vector superposition based on the hot air flow displacement field, and then the image of the to-be-measured sample in the measurement image is re-mapped pixel by pixel to obtain the corrected image, and then the warpage deformation information of the to-be-measured sample is obtained.

[0009] Further, the three-dimensional reconstruction is performed on the test sample based on the corrected image to obtain the warping deformation information of the test sample.

[0010] Further, the background speckle image, as an information carrier of the flow field change, is composed of white-black circular speckles with random distribution and a diameter of 5 pixels, and the speckle duty cycle is 50%.

[0011] Further, the calculation formula of the offset amount Δx of the hot gas flow displacement field in the x direction of the camera imaging plane is as follows:

[0012]

[0013] The calculation formula of the offset amount Δy of the hot gas flow displacement field in the y direction of the camera imaging plane is as follows:

[0014]

[0015] wherein, Z D is the distance between the background plane and the hot gas flow, Z A is the distance between the camera and the hot gas flow, W is the effective width of the hot gas flow field, n is the refractive index of the hot gas flow, n0 is the refractive index of air, and f is the focal length of the camera lens.

[0016] Further, the multiple network iteration algorithm based on query window deformation is used to calculate the hot gas flow displacement field.

[0017] Further, the mutual correlation algorithm is used to perform coarse calculation of the displacement field according to the 1 / 4 principle, and Δs is obtained. Then, the query window is deformed using the coarse calculation result Δs of the displacement field, and the mutual correlation calculation is performed again to obtain the displacement field increment Δs'. Through multiple iteration calculations and continuous reduction of the query window size and sampling interval, the displacement field is updated as Δs = Δs + Δs'.

[0018] Further, the initial query area size of the multiple network iteration algorithm based on query window deformation is 128 pixels, and the final size is reduced to 32 pixels, and the overlap rate is 50%.

[0019] Further, the bilinear interpolation algorithm is used to calculate the pixel value, and the gray value of the remapped pixel point is determined by linear interpolation according to the gray values of the surrounding four pixel points.

[0020] Further, the three-dimensional reconstruction of the test sample is performed by using the complementary Gray code auxiliary phase unwrapping, and the corresponding formula is as follows:

[0021]

[0022] wherein, Φ(x, y) is a continuous phase field modulated by the object height, and φ(x, y) is a wrapped phase.

[0023] The application also provides a background schlieren-based electronic package warping detection system for implementing the above-mentioned background schlieren-based electronic package warping detection method, which comprises a plurality of cameras, a projector, a heating table, a background plate and a computing unit, the cameras are used to shoot the background speckle image without thermal air flow disturbance as a reference image and shoot the image of the sample to be measured and the background speckle image under thermal air flow disturbance as a measurement image, and transmit the shot images to the computing unit; the background plate is used to provide a background for shooting the background speckle image; the heating table is used to heat the air between the camera and the sample to be measured to cause thermal air flow disturbance; the projector is used to project phase shift stripes and Gray code images to the sample to be measured under thermal air flow disturbance; and the computing unit is used to calculate the thermal air flow displacement field, perform pixel-by-pixel remapping on the image of the sample to be measured in the measurement image and perform three-dimensional reconstruction on the sample to be measured to obtain the warping deformation information of the sample to be measured according to the received information.

[0024] Overall, compared with the prior art, the background schlieren-based electronic package warping detection method and system provided by the application mainly have the following beneficial effects:

[0025] 1. The application precisely corrects the image distortion caused by thermal air flow through the background schlieren technology. Under the interference of thermal air flow, the digital stripe projection system will produce measurement errors, and the background schlieren technology effectively compensates for the influence of thermal refraction through accurate calculation of light deflection, so that the measurement result is more accurate and reliable, providing a favorable guarantee for high-precision detection of electronic package structures.

[0026] 2. The application is suitable for complex structure measurement. For a package structure with complex microstructure and high-density reflection, the background schlieren technology is used for correction, and the reconstructed topography is complete and the edge profile is clear.

[0027] 3. The detection system of the application can not only accurately measure the three-dimensional topography of the package structure, but also has low hardware cost and simple operation, can provide accurate data support for reflow soldering process, solder joint reliability evaluation, etc., helps to improve the process control capability of electronic package manufacturing, promotes the development of electronic package detection towards real-time and automation, and has high practical value. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flowchart of a background schlieren-based electronic package warping detection method provided by the application;

[0029] Figure 2 is a schematic diagram of a background schlieren-based electronic package warping detection system provided by the application;

[0030] Figure 3 is a schematic diagram of the offset of the thermal air flow displacement field on the camera imaging plane constructed according to the preferred embodiment of the present application;

[0031] Figure 4 is a schematic diagram of the multiple network iteration algorithm based on query window deformation constructed according to the preferred embodiment of the present application, wherein (a) corresponds to the multiple network iteration algorithm based on query window deformation, and (b) corresponds to the query window deformation iteration operation;

[0032] Figure 5 is a schematic diagram of the complementary Gray code assisted phase unwrapping constructed according to the preferred embodiment of the present application;

[0033] Figure 6 is a schematic diagram of the random speckle on the background plane constructed according to the preferred embodiment of the present application, wherein the diameter of the black circular speckle is 5 pixels;

[0034] Figure 7 is a three-dimensional topography measurement result of a digital fringe projection system on a sample to be measured under thermal air flow disturbance constructed according to the preferred embodiment of the present application, wherein (a) is the measurement result before correction using the background schlieren technique, and (b) is the measurement result after correction using the background schlieren technique.

[0035] In all the drawings, the same reference signs are used to represent the same elements or structures, wherein: 1-optical experiment platform, 2-sample to be measured, 3A-first CCD camera, 3B-second CCD camera, 3C-third CCD camera, 4-DLP projector, 5A-first background plate, 5B-second background plate, 6-heating table, 7-computing unit. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0037] Please refer to Figure 1 , Figure 2 and Figure 3 , the present application provides a background schlieren based electronic packaging warpage detection method, which mainly comprises the following steps:

[0038] Step one, take the background speckle image without thermal air flow disturbance as the reference image; then, project the phase shift fringe and Gray code image on the sample to be measured under thermal air flow disturbance, and synchronously take the image of the sample to be measured and the background speckle image as the measurement image.

[0039] In one embodiment, the camera is used to take the background speckle image when there is no thermal airflow disturbance to obtain the reference image in the x-z and y-z planes; the heating table is set to heat the sample to be measured, and after the temperature is stable, the projector continuously projects the phase shift fringe and the Gray code image onto the sample to be measured, and synchronously triggers three cameras to collect the sample image containing thermal airflow disturbance on the heating table and the background speckle image, and obtains the measurement image in the x-y, x-z and y-z planes.

[0040] The background speckle image, as an information carrier of the flow field change, is composed of white background and black circular speckles with a diameter of 5 pixels randomly distributed, and the speckle duty cycle is close to 50%. A black square area with a size of 20*20 mm is set in the background speckle image, which is used for system calibration before displacement field calculation.

[0041] The image filtering technology is used to reduce the system error caused by the high frequency component of the speckle image, and the random error caused by the speckle image noise is reduced by increasing the gray level gradient square sum. The contrast-limited adaptive histogram equalization method is used to enhance the local contrast of the random speckle image, and the contrast threshold is used to suppress the image noise amplification, so as to improve the data quality of the displacement field.

[0042] In step two, the cross-correlation algorithm is used to calculate the thermal airflow displacement field based on the reference image and the background speckle image in the measurement image, and the displacement field in the predetermined direction is obtained by two-dimensional projection vector superposition based on the thermal airflow displacement field, and then the image of the sample to be measured in the measurement image is re-mapped pixel by pixel to obtain the corrected image.

[0043] In one embodiment, the cross-correlation algorithm is used to calculate the thermal airflow displacement field in the x-z and y-z planes according to the reference image and the measurement image collected in the x-z and y-z planes; the thermal airflow displacement field satisfies the small disturbance assumption, and the displacement field in the x-y plane is calculated by two-dimensional projection vector superposition based on the displacement field in the x-z and y-z planes, and then the sample image containing thermal airflow disturbance on the heating table collected by the camera is re-mapped pixel by pixel.

[0044] The offset Δx of the thermal airflow displacement field in the x direction of the camera imaging plane is calculated according to the following relationship:

[0045]

[0046] The offset Δy of the thermal airflow displacement field in the y direction of the camera imaging plane is calculated according to the following relationship:

[0047]

[0048] where Z D is the distance between the background plane and the hot gas flow, Z A is the distance between the camera and the hot gas flow, W is the effective width of the hot gas flow field, n is the refractive index of the hot gas flow, n0 is the refractive index of air, and f is the focal length of the camera lens.

[0049] The calculation of the displacement field uses a multiple network iteration algorithm based on query window deformation. The algorithm first uses a cross-correlation algorithm based on the 1 / 4 principle to perform a rough calculation of the displacement field to obtain Δs, then performs a deformation operation on the query window using the rough calculation result Δs of the displacement field, and performs a cross-correlation calculation again to obtain a displacement field increment Δs'. Through multiple iteration calculations and continuous reduction of the query window size and sampling interval, the displacement field is updated as Δs = Δs + Δs'.

[0050] The initial query area size of the multiple network iteration algorithm based on query window deformation is 128 pixels, and the final size is reduced to 32 pixels. The overlap rate is 50%, and the peak detection uses a three-point Gaussian method to achieve sub-pixel accuracy.

[0051] When performing pixel-by-pixel remapping on the image of the sample to be measured containing hot gas flow disturbance on the heating table collected by the camera, a bilinear interpolation algorithm is used to calculate the pixel value. According to the gray values of the surrounding four pixel points, the gray value of the remapped pixel point is determined through linear interpolation, ensuring the continuity and accuracy of the remapped image.

[0052] Step three, based on the corrected image, the three-dimensional reconstruction of the sample to be measured is performed to obtain the warping deformation information of the sample to be measured.

[0053] In one embodiment, the three-dimensional reconstruction of the sample to be measured is performed according to the obtained corrected image and the detection system calibration information, so as to obtain the warping deformation information of the sample to be measured.

[0054] The three-dimensional reconstruction of the sample to be measured is performed using complementary Golay code auxiliary phase unwrapping, and the corresponding formula is:

[0055]

[0056] where Φ(x, y) is a continuous phase field modulated by the height of the object, and φ(x, y) is a wrapped phase.

[0057] The application further provides a background speckle-based electronic package warping detection system for implementing the background speckle-based electronic package warping detection method as described above, which comprises a plurality of cameras, a projector, a heating table, a background plate and a computing unit. The cameras are used to take a background speckle image without thermal air flow disturbance as a reference image and take an image of a sample to be measured and a background speckle image under thermal air flow disturbance as measurement images, and transmit the taken images to the computing unit. The background plate is used to provide a background for taking the background speckle image. The heating table is used to heat the air between the camera and the sample to be measured to cause thermal air flow disturbance. The projector is used to project phase shift fringes and Gray code images to the sample to be measured under thermal air flow disturbance. The computing unit is used to calculate a thermal air flow displacement field, perform pixel-by-pixel remapping on the image of the sample to be measured in the measurement images and perform three-dimensional reconstruction on the sample to be measured to obtain warping deformation information of the sample to be measured according to the received information.

[0058] The application will be further described in detail below with specific examples.

[0059] Please refer to Figure 1 A background speckle-based electronic package warping detection method comprises the following steps:

[0060] S1, a sample to be measured is prepared and placed on a heating table. The positions of the camera and the projector are adjusted and fixed, the camera and the projector are jointly calibrated, the geometric and optical parameters of the camera and the projector in the detection system are determined, and the mapping relationship between the phase and the three-dimensional space coordinates is established.

[0061] S2, a background speckle image is taken by the camera in an environment without heating the sample to be measured, i.e. without thermal air flow disturbance, as a reference image for subsequent displacement field calculation.

[0062] S3, the sample to be measured is uniformly heated by using a suitable heating temperature to generate thermal air flow around the sample to be measured. After the heating temperature is stable, the projector projects phase shift fringes and Gray code images to the surface of the sample to be measured, and synchronously triggers three cameras to collect images containing sample information and thermal air flow influence, as well as a background speckle image, to ensure the synchronicity and integrity of image collection, so as to obtain comprehensive measurement data.

[0063] S4, the speckle images collected before and after heating are processed by using a cross-correlation algorithm to calculate displacement field information caused by thermal air flow. The collected sample images are pixel-by-pixel remapped according to the calculated displacement field to correct image distortion caused by thermal air flow.

[0064] S5, combining the calibration information, using a three-dimensional reconstruction algorithm on the corrected image to perform three-dimensional reconstruction on the sample to be measured; obtaining the warping deformation information of the sample by analyzing the reconstructed three-dimensional model, thereby realizing accurate measurement and evaluation of the sample under the disturbance of the hot air flow, and providing accurate data support for quality control and reliability analysis in the field of electronic packaging.

[0065] Further preferably, in step S1, the camera and the projector are both built on an optical experiment platform to reduce the interference of external vibration.

[0066] Further preferably, in step S1, the calibration board adopts a 9x7 distributed black bottom white circle calibration board.

[0067] Further preferably, in step S2, the speckle image is composed of white bottom black circular speckles randomly distributed with a diameter of 5 pixels, and the speckle duty cycle is close to 50%, as shown in Figure 6

[0068] Further preferably, in step S2, a black square area with a size of 20x20mm in the background speckle image, as shown in Figure 6

[0069] Further preferably, in step S3, the three cameras are synchronously triggered by the rising edge of the projector to ensure that the exposure time of the projector and the camera is equal.

[0070] Further preferably, in step S3, the system error caused by the high frequency component of the speckle image is reduced by image filtering technology, and the random error caused by speckle image noise is reduced by increasing the gray level gradient square sum of the sub-area.

[0071] Further preferably, in step S3, the local contrast of the random speckle image is enhanced by using the contrast-limited adaptive histogram equalization method, and the image noise amplification is suppressed by using the contrast threshold, so as to improve the data quality of the displacement field.

[0072] Further preferably, in step S4, a multiple network iteration algorithm based on query window deformation is used, as shown in Figure 4 The hot air flow displacement field is calculated by multiple iteration calculation and continuous reduction of the query window size and sampling interval. The offset of the hot air flow displacement field in the camera imaging plane is shown in Figure 3 The calculation formula of the offset Δx of the hot air flow displacement field in the x direction of the camera imaging plane is:

[0073]

[0074] The calculation formula of the offset Δy of the hot air flow displacement field in the y direction of the camera imaging plane is:​​

[0075]

[0076] where Z D represents the distance between the background plane and the hot air flow, Z A represents the distance between the camera and the hot air flow, W represents the effective width of the hot air flow field, n represents the refractive index of the hot air flow, n0 represents the refractive index of air, and f represents the focal length of the camera lens.

[0077] Further preferably, in step S4, the initial query region size is 128 pixels, the final size is reduced to 32 pixels, the overlap rate is 50%, and the peak detection uses a three-point Gaussian method to achieve sub-pixel accuracy.

[0078] Further preferably, in step S5, a three-dimensional reconstruction is performed using complementary Gray code assisted phase unwrapping, as shown in Figure 5 where M = log2T, and the first M = log2T Gray codes are used to obtain the phase order k1 according to the following relationship:

[0079]

[0080] k1(x, y) = LUT[V1(x, y)]

[0081] All M + 1 = 1 + log2T Gray codes are used to obtain the phase order k2 according to the following relationship:

[0082]

[0083] Thus, the truncated phase is unwrapped according to the following relationship:

[0084]

[0085] where Φ(x, y) is a continuous phase field modulated by the height of the object, φ(x, y) is a wrapped phase, LUT() is used to look up the known relationship between the decimal phase values V1, V2 and the phase orders k1, k2, INT() represents the forward integer operation, T is the period of the phase shift fringe, and GC i represents the i-th Gray code.

[0086] The structure of the detection system suitable for the above detection method is set as follows:

[0087] (1) System layout

[0088] As Figure 2As shown, the first CCD camera 3A is installed on one side of the gantry structure built by aluminum profiles, with the lens direction perpendicular to the optical platform, ensuring that the camera field of view can completely cover the heating stage 6 and the area of the sample 2 to be measured, avoiding shadows and occlusions. The DLP projector 4 is installed on the other side of the gantry structure, at an angle to the optical platform, projecting downward, ensuring that the phase shift fringe and the Gray code image projected by the projector can uniformly cover the heating stage 6 and the area of the sample 2 to be measured. The second CCD camera 3B and the third CCD camera 3C are fixed on the optical platform through an optical connecting rod, ensuring that the two cameras are on the same horizontal line, with the lens optical axes of the two cameras perpendicular to each other and perpendicular to the first background plate 5A and the second background plate 5B, respectively, and the distances from the lens of the two cameras to the background plates along the optical axis are equal, and the fields of view of the two cameras cover the plane of the heating stage 6 and the area above it. The first background plate 5A and the second background plate 5B are fixed on the bracket built by aluminum profiles, and the background plates are perpendicular to the optical platform.

[0089] (2) Focal length of camera and projector

[0090] The distance between the first CCD camera 3A and the DLP projector 4 and the sample 2 to be measured is determined, and the focal length of the first CCD camera 3A and the DLP projector 4 is adjusted to ensure clear imaging and projection pattern. The same operation is performed on the second CCD camera 3B and the third CCD camera 3C to ensure clear background speckle imaging.

[0091] (3) Calibration of camera and projector

[0092] The system calibration before three-dimensional reconstruction is realized by a calibration board. The calibration board is placed in the field of view of the first CCD camera 3A, the DLP projector 4 projects the phase shift fringe and the Gray code image on the calibration board, and the camera is triggered synchronously to capture the phase shift fringe and the Gray code image on the calibration board. By changing the pose of the calibration board and detecting the center of the calibration board, the correspondence between the calibration board coordinate system and the camera and projector pixel coordinates is established, and the system parameters and external parameters are calculated using the optical model and the camera-projection geometry relationship to realize the calibration of the three-dimensional measurement system. The calibration of the second CCD camera 3B and the third CCD camera 3C is realized by the corresponding relationship between the physical size and the pixel size of the 20x20mm black square region in the background speckle image.

[0093] (4) Setting of phase shift fringe and Gray code image

[0094] The number of steps and the period of the phase shift fringe are set using the computing unit 7 for phase measurement. Then, a complementary Gray code image matching the phase shift fringe period is set for eliminating decoding errors, and the phase shift fringe and the Gray code image are burned into the GUI of the DLP projector 4, and the camera is triggered synchronously by the DLP projector 4 to capture the image.

[0095] (5) Setting of background speckle

[0096] The parameters of the random degree, diameter size and duty cycle of the background speckle distribution are set by using the computing unit 7.

[0097] In one specific embodiment, the sample 2 to be measured is a BGA package chip, the radius of the tin ball is 0.3 mm, and the center distance between adjacent tin balls in the horizontal and vertical directions is 1 mm. The topography of the BGA package chip is measured to verify the measurement accuracy and robustness of the detection system under the disturbance of the hot air flow.

[0098] The three-dimensional topography detection system of the present example is shown in FIG. 1, which comprises an optical experiment platform 1, a sample 2 to be measured, a first CCD camera 3A, a second CCD camera 3B, a third CCD camera 3C, a DLP projector 4, a first background plate 5A, a second background plate 5B, a heating table 6 and a computing unit 7. Figure 2

[0099] The sample 2 to be measured is placed on the heating table 6; the first CCD camera 3A and the DLP projector 4 are installed on a gantry structure built of aluminum profiles; the second CCD camera 3B and the third CCD camera 3C are fixed on the optical platform through a connecting rod; the first background plate 5A and the second background plate 5B are fixed on a bracket built of aluminum profiles; the computing unit 7 is connected with the first CCD camera 3A, the second CCD camera 3B, the third CCD camera 3C and the DLP projector 4, for generating phase shift stripes, Gray code images and speckle patterns, and controlling the synchronization triggering of the cameras and the projector; the computing unit 7 is used for calculating the hot air flow displacement field, correcting the distorted images and three-dimensionally reconstructing the sample to be measured.

[0100] The specific implementation steps of the electronic package warping detection method combined with the background schlieren technology of the present application are as follows:

[0101] Step one, layout installation and calibration of the detection system.

[0102] Step 1.1: Layout installation of the system

[0103] The aluminum profile gantry structure is built on the optical experiment platform 1, the first CCD camera 3A and the DLP projector 4 are installed on the gantry structure, the second CCD camera 3B and the third CCD camera 3C are fixed on the optical platform through a connecting rod, and the first background plate 5A and the second background plate 5B are installed on the aluminum profile bracket. The focal lengths of the first CCD camera 3A, the second CCD camera 3B, the third CCD camera 3C and the DLP projector 4 are adjusted for the sample 2 to be measured and the background speckle, so that clear images are presented in the computing unit 7.

[0104] Step 1.2: Calibration of the cameras and the projector

[0105] ​The position of the calibration board is reasonably selected so that the field of view of the first CCD camera 3A and the projection pattern of the DLP projector 4 can both completely cover the calibration board. The pose of the calibration board is changed, and images are captured by the first CCD camera 3A and stored by the computing unit 7. The calibration of the three-dimensional measurement system is realized by detecting the center of the calibration board.

[0106] The specific implementation method of step two is as follows:

[0107] Step 2.1: Acquisition of reference images

[0108] In an environment without heating the sample 2, i.e. without thermal air flow disturbance, the second CCD camera 3B and the third CCD camera 3C are used to capture speckle images on the first background board 5A and the second background board 5B respectively, as reference images for subsequent thermal air flow displacement field calculation.

[0109] Step 2.2: Acquisition of measurement images

[0110] The sample 2 is uniformly heated by the heating table 6, so that a thermal air flow is generated around the sample 2. During the heating process, the DLP projector 4 projects phase shift stripes and Gray code images onto the surface of the sample 2, and the first CCD camera 3A, the second CCD camera 3B and the third CCD camera 3C are triggered synchronously to capture images of the sample 2 containing thermal air flow disturbance and background speckle images, as measurement images for subsequent three-dimensional reconstruction and thermal air flow displacement field calculation.

[0111] The specific implementation method of step three is as follows:

[0112] Step 3.1: Improve the quality of speckle images

[0113] Image filtering, improvement of sub-region gray level gradient square sum, and contrast-limited adaptive histogram equalization are used to improve the quality of the background speckle images captured by the second CCD camera 3B and the third CCD camera 3C before and after the heating table 6 is turned on, so as to improve the data quality of the displacement field.

[0114] Step 3.2: Calculate the thermal air flow displacement field

[0115] For the background speckle images captured by the second CCD camera 3B and the third CCD camera 3C before and after the heating table 6 is turned on, a multiple network iteration algorithm based on query window deformation is used to calculate the thermal air flow displacement field by multiple iterations and continuously reducing the query window size and sampling interval, as shown in Figure 4

[0116] The specific implementation method of step four is as follows:

[0117] Step 4.1: Correct the distorted images

[0118] ​According to the calculated displacement field, the image of the sample 2 containing the thermal airflow disturbance collected by the first CCD camera 3A is remapped pixel by pixel using a bilinear interpolation algorithm to correct the image distortion caused by the thermal airflow.

[0119] Step 4.2: three-dimensional reconstruction

[0120] Combined with the calibration information, a three-dimensional reconstruction algorithm is used to correct the three-dimensional topography of the sample 2 before and after correction as shown in FIG. 6, wherein (a) is the three-dimensional topography of the sample 2 before correction, and a large number of artifacts exist on the surface; (b) is the three-dimensional topography of the sample 2 after correction, and the surface artifacts are significantly reduced, the tin ball topography is complete, and the edge contour is clear. Figure 7

[0121] Those skilled in the art will readily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for detecting warpage in electronic packaging based on background schlieren, characterized in that, The method includes the following steps: (1) Project phase-shifted fringes and Gray code images onto the sample under thermal gas disturbance, and simultaneously capture images of the sample and background speckle images as measurement images; (2) Based on the background speckle image without thermal airflow disturbance and the background speckle image in the measurement image, the thermal airflow displacement field is calculated by cross-correlation algorithm. Based on the thermal airflow displacement field, the displacement field in the predetermined direction is obtained by superimposing the two-dimensional projection vector. Then, the image of the sample to be tested in the measurement image is remapped pixel by pixel to obtain the corrected image, and then the warping deformation information of the sample to be tested is obtained. The algorithm employs a multi-network iterative algorithm based on query window deformation to calculate the thermal gas displacement field. First, a coarse calculation of the displacement field is performed using a cross-correlation algorithm based on the 1 / 4 principle, yielding... Then, the coarse calculation results of the displacement field are used. The query window is deformed, and the cross-correlation calculation is performed again to obtain the displacement field increment. The displacement field is updated through multiple iterative calculations and by continuously reducing the query window size and sampling interval. .

2. The electronic packaging warpage detection method based on background schlieren as described in claim 1, characterized in that: Three-dimensional reconstruction of the sample under test is performed based on the corrected image to obtain information on the warping deformation of the sample under test.

3. The electronic packaging warpage detection method based on background schlieren as described in claim 1, characterized in that: The background speckle image, serving as a carrier of information about flow field changes, consists of randomly distributed black circular speckles on a white background, each with a diameter of 5 pixels, and a speckle duty cycle of 50%.

4. The electronic packaging warpage detection method based on background schlieren as described in claim 1, characterized in that: Thermal gas displacement field on the camera imaging plane Offset in direction The calculation formula is: Thermal gas displacement field on the camera imaging plane Offset in direction The calculation formula is: in, It is the distance between the background plane and the hot airflow. It is the distance between the camera and the hot airflow. It is the effective width of the hot airflow field. It is the refractive index of hot airflow. It is the refractive index of air. It is the focal length of the camera lens.

5. The electronic packaging warpage detection method based on background schlieren as described in claim 1, characterized in that: The multi-network iterative algorithm based on query window deformation initially sets the query region size to 128 pixels, eventually reducing it to 32 pixels with an overlap rate of 50%.

6. The method for detecting electronic package warpage based on background schlieren as described in any one of claims 1-5, characterized in that: The pixel value is calculated using a bilinear interpolation algorithm. Based on the gray values ​​of the four surrounding pixels, the gray value of the remapped pixel is determined by linear interpolation.

7. The method for detecting electronic package warpage based on background schlieren as described in any one of claims 1-5, characterized in that: Complementary Gray code-assisted phase expansion is used to reconstruct the three-dimensional sample under test. The corresponding formula is: in, It is a continuous phase field modulated by the height of the object. It is a wrap-around phase.

8. A background schlieren-based electronic package warpage detection system for implementing the background schlieren-based electronic package warpage detection method according to any one of claims 1-7, characterized in that: The detection system includes multiple cameras, a projector, a heating stage, a background plate, and a computing unit. The cameras capture background speckle images without thermal airflow disturbance as reference images and images of the sample under test and the background speckle images under thermal airflow disturbance as measurement images, transmitting the captured images to the computing unit. The background plate provides a background for capturing the background speckle images. The heating stage heats the air between the camera and the sample under test to create thermal airflow disturbance. The projector projects phase-shifted fringes and Gray code images onto the sample under thermal airflow disturbance. The computing unit calculates the thermal airflow displacement field based on the received information, performs pixel-by-pixel remapping of the sample image in the measurement images, and performs three-dimensional reconstruction of the sample to obtain its warping deformation information.

Images