Method for optical visual inspection of surface defects of diodes

Abstract

The application discloses a kind of diode surface defect optical visual inspection methods, it is related to diode optical detection technical field, including the following steps: acquisition obtains preheating after bright field image sequence and preheating after dark field image sequence;Based on preheating after bright field image sequence and preheating after dark field image sequence, morphology complementary matching is carried out, and the bubble candidate sequence of each frame image is obtained;Based on bubble candidate sequence;The present application is based on the moving curve type of real bubble, temperature gradient attenuation curve and viscosity decay function are combined to carry out physical correction, accurately predict the spatial position of bubble in subsequent high-temperature welding procedure;By cross-frame association tracking and moving curve fitting to bubble candidate sequence, the instantaneous velocity and motion direction of each bubble are accurately extracted. The migration number probability density distribution curve is generated using kernel density estimation, and whether the bubble satisfies the physical law of thermal migration is adaptively judged based on the main peak interval, so as to accurately distinguish real bubbles from false defects.

Description

Technical Field

[0001] This invention relates to the field of diode optical inspection technology, and in particular to an optical visual inspection method for diode surface defects. Background Technology

[0002] In diode packaging, infrared preheating is often used to eliminate pin stress. For transparent or semi-transparent plastic packages, to achieve efficient optical detection of internal bubble defects and ensure product reliability, a detection technology based on a multimodal joint sparse coding model is commonly used. The core principle of this technology is as follows: when constructing a multimodal image analysis model for diode surface defects, to simplify the calculation process and control the processing time per frame to match the production line's requirements for detection speed, it is usually assumed that the same bubble defect maintains a fixed spatial position in multiple images under different lighting parameter combinations. That is, by acquiring image sequences under various lighting angles, the gray values ​​at the same pixel coordinate position are used to construct an observation vector. Based on this fixed position assumption, joint sparse coding decomposition is performed to separate the residual component representing the defect and the primitive component representing the normal texture, thereby completing the detection and localization of the bubble defect. However, in actual production scenarios, during the preheating process, diodes are subjected to temperature gradients, causing thermally induced migration of air bubbles inside the encapsulation. Due to the inconsistency in the thermal expansion coefficients of the encapsulation material and the gas inside the bubbles, the bubbles move slowly along the temperature gradient, with migration distances ranging from zero to hundreds of micrometers. Furthermore, differences in preheating time due to factors such as conveyor belt speed fluctuations and uneven aging of heating lamps among different diodes result in a random distribution of bubble migration distances within each batch. The aforementioned technology uses a fixed-position assumption for uniform processing, failing to dynamically adjust the analysis model based on the actual bubble displacement. When bubble migration occurs, the same bubble is projected onto different pixel coordinates in images with different lighting parameters, causing the observation vector of the same physical point to alias the bubble signal with the background texture signal. For migrating bubbles, the fixed-position assumption struggles to provide a correct signal decomposition basis, making it impossible to accurately separate defect residual components, resulting in numerous false positives (misclassifying normal texture as defects) or false negatives (misclassifying real bubbles as texture). This issue directly and severely impacts the accuracy and long-term reliability of diode product quality inspection: In production line inspection scenarios, false positives lead to the incorrect rejection of qualified products, increasing manual re-inspection costs and reducing production line throughput; false negatives allow diodes with bubble defects to flow into downstream packaging or end-use applications. During subsequent high-temperature soldering, undetected bubbles may expand due to heat, potentially causing the molded enclosure to burst and leading to device failure. In applications with extremely high reliability requirements, such as automotive-grade diodes or power modules, these defects can directly cause malfunctions in the vehicle's electronic systems or short-circuit burnouts, not only reducing the manufacturer's yield rate and market reputation but also threatening the safety of end-use products, increasing recall risks, and failing to meet the semiconductor packaging industry's core requirements for defect detection accuracy and product reliability. Summary of the Invention

[0003] The purpose of this invention is to solve the problems of false positives and false negatives caused by the fixed position assumption of existing detection methods, which cannot adapt to the thermal migration of bubbles and cannot predict subsequent welding risks or guide process adjustments. Therefore, an optical visual detection method for diode surface defects is proposed.

[0004] To achieve the above objectives, the present invention employs the following technology: an optical visual inspection method for diode surface defects, comprising the following steps: The preheated bright-field image sequence and the preheated dark-field image sequence were acquired. Based on the preheated bright field image sequence and the preheated dark field image sequence, morphological complementary matching is performed to obtain the bubble candidate sequence for each frame of the image. Based on the bubble candidate sequence, cross-frame correlation tracking is performed on the candidate regions belonging to the same physical bubble in the bubble candidate sequence to obtain the spatial position temporal sequence of each bubble, and the movement curve of each bubble is fitted. Based on the movement curve of each bubble, the instantaneous velocity and direction of motion of each bubble are calculated; an instantaneous velocity vector is formed based on the instantaneous velocity and direction of motion of each bubble; the temperature gradient direction of the preheating process is obtained, the three-dimensional temperature gradient direction is projected along the imaging optical axis onto the diode principal plane and mapped to the pixel coordinate system through the camera intrinsic parameters to obtain a two-dimensional temperature gradient direction vector, and the instantaneous velocity vector is projected onto the two-dimensional temperature gradient direction vector to obtain the projected velocity component; The equivalent viscosity coefficient and preheating time of the molding compound material are obtained, and the theoretical characteristic velocity is determined based on the average temperature gradient value, preheating time, bubble thermal expansion characteristic constant and equivalent viscosity coefficient. The dimensionless migration number of each candidate bubble is obtained by dividing the projected velocity component by the theoretical characteristic velocity. Based on the dimensionless migration number of all candidate bubbles in the current batch, kernel density estimation is performed to generate a migration number probability density distribution curve and determine the main peak interval. If the dimensionless migration number of a candidate bubble falls into the main peak interval, the candidate bubble is determined to satisfy the physical law of thermally induced migration and is regarded as a real bubble. If it does not fall into the main peak interval, it is determined to be a false defect. Based on the actual bubble movement curve, the spatial position in subsequent processes is predicted, and bubble migration trajectory prediction data is obtained. Based on bubble migration trajectory prediction data, a risk warning for high-temperature welding process is output. Based on the instantaneous velocity and direction of motion of each real bubble, the optical visual inspection results of diode surface defects are obtained.

[0005] Furthermore, the method for obtaining the three-dimensional temperature gradient direction of the preheating process includes: A miniature thermocouple array is arranged along the axial and radial directions of the diode to synchronously collect temperature data from each temperature measuring point during the stable operation phase of the preheating process. The collected spatial temperature field data is interpolated and fitted to construct a temperature distribution map of the area where the encapsulated body is located in the preheating temperature zone, and the direction of the three-dimensional temperature gradient is determined according to the normal direction of the isothermal surface in the temperature distribution map.

[0006] Furthermore, the methods for calculating the theoretical characteristic velocity include: The equivalent viscosity coefficient of the batch of molding compound material under the temperature conditions of the preheating process is obtained. The preheating time is determined according to the conveyor belt speed and the effective length of the heating zone in the preheating process. The average temperature gradient value inside the molding compound is determined by thermal simulation or temperature measurement calibration. The product of the average temperature gradient, preheating time, and bubble thermal expansion constant is used as the numerator, and the equivalent viscosity coefficient is used as the denominator. The resulting quotient is used as the theoretical characteristic velocity.

[0007] Furthermore, the method for obtaining a bubble candidate sequence for each frame of the image by performing morphological complementary matching based on the preheated bright-field image sequence and the preheated dark-field image sequence includes: Based on the bright field image sequence after preheating, each frame of the bright field image sequence is extracted to obtain multiple candidate dark spot regions. The spatial location, dark spot area, dark spot circularity, and dark spot grayscale contrast of each candidate dark spot region are recorded. Based on the dark field image sequence after preheating, the corresponding frame images in the dark field image sequence after preheating are extracted to obtain multiple candidate bright spot regions. The spatial location, bright spot area, bright spot circularity and bright spot grayscale contrast of each candidate bright spot region are recorded. Based on multiple candidate dark spot regions and multiple candidate bright spot regions, the candidate dark spot regions and candidate bright spot regions in the same frame are paired according to their spatial location to obtain paired dark spot regions and bright spot regions. Based on the paired dark spot regions and bright spot regions, the spatial distance, area ratio, roundness difference and gray-level contrast product between each pair of paired dark spot regions and bright spot regions are calculated. Determine whether the product of spatial distance, area ratio, roundness difference, and grayscale contrast satisfies the predefined complementary constraint conditions; If the spatial distance, area ratio, roundness difference, and gray-level contrast product all satisfy the predefined complementary constraint conditions, then the spatial position corresponding to the paired dark spot region and the bright spot region is determined as the bubble candidate region of the frame image. Based on the bubble candidate regions of each frame of the image, the bubble candidate regions of each frame of the image are summarized to obtain the bubble candidate sequence.

[0008] Furthermore, complementary constraints include: Spatial distance constraint: The spatial distance between the dark spot region and the bright spot region is less than the maximum allowable value of the optical misalignment calibration residual between the bright field imaging system and the dark field imaging system. The area ratio constraint ensures that the ratio of the dark spot area to the bright spot area is within the range determined by the theoretical ratio of the absorption cross-section of the bubble in bright field to the scattering cross-section in dark field. The roundness difference constraint is that the absolute value of the difference between the roundness of dark spots and the roundness of bright spots is less than the preset shape consistency tolerance based on the consistency of bubble shape in both bright and dark fields. The gray-scale contrast product constraint requires that the product of the gray-scale contrast of the dark spot and the gray-scale contrast of the bright spot be greater than the minimum detection threshold determined based on the complementary enhancement principle of signal-to-noise ratio in bright and dark field imaging.

[0009] Furthermore, methods for fitting the movement curve of each bubble based on the bubble candidate sequence include: Based on the bubble candidate sequence, cross-frame correlation tracking is performed on the candidate regions belonging to the same physical bubble in the bubble candidate sequence to obtain the spatial location time sequence of each bubble; Based on the spatial location time sequence of each bubble, least squares fitting is performed using a linear model and a parabolic model respectively to obtain linear fitting curves and parabolic fitting curves. Calculate the first residual sum of squares between the linear fitting curve and the spatial location time series, and the second residual sum of squares between the quadratic parabolic fitting curve and the spatial location time series, and compare the magnitudes of the first residual sum of squares and the second residual sum of squares. If the sum of squares of the first residual is less than the sum of squares of the second residual, then the linear fitting curve is selected as the moving curve of the bubble. If the second residual sum of squares is less than the first residual sum of squares, then the quadratic parabola fitted curve is selected as the moving curve of the bubble.

[0010] Furthermore, methods for calculating the instantaneous velocity and direction of motion of each bubble based on its movement curve include: Based on the movement curve of each bubble, determine the type of movement curve of each bubble. If the movement curve is a linear fitting curve, take the slope of the movement curve as the constant instantaneous velocity of the bubble throughout its entire journey, and extract the spatial direction angle of the movement curve as the instantaneous direction of the bubble throughout its entire journey. If the moving curve is a quadratic parabola fitting curve, then take the first derivative of the moving curve to obtain an expression for the linear change of instantaneous velocity with time. Based on this expression, the value of the expression at each acquisition time is taken as the instantaneous velocity of the corresponding frame. Based on this expression, the instantaneous direction is calculated frame by frame according to the tangent direction of the curve at each acquisition time.

[0011] Furthermore, methods for predicting the spatial position of bubbles in subsequent processes based on the actual bubble movement curves, and obtaining bubble migration trajectory prediction data, include: Based on the movement curve of the real bubble, the type of the real bubble movement curve is obtained. If the movement curve is a linear fitting curve, the slope of the movement curve is used as the constant migration speed. Based on the constant migration speed, the spatial position of the bubble in the subsequent process is calculated using a linear extrapolation formula. If the moving curve is a quadratic parabola fitting curve, then the first derivative of the moving curve is taken to obtain the instantaneous velocity expression; based on the instantaneous velocity expression, the instantaneous velocity expression is integrated to obtain a quadratic function of displacement changing with time; based on the quadratic function of displacement changing with time, the spatial position of the bubble in the subsequent process is calculated by extrapolation of this quadratic function. The temperature gradient decay curve and viscosity decay function at the end of the preheating process are obtained. Based on the temperature gradient decay curve, viscosity decay function and the spatial position of bubbles in subsequent processes, the velocity parameters in the linear extrapolation formula or quadratic function are corrected to obtain bubble migration trajectory prediction data.

[0012] Furthermore, methods for generating risk warnings for high-temperature welding processes based on bubble migration trajectory prediction data include: Obtain a distribution map of the welding area in the high-temperature welding process, and divide the welding area into three risk level areas: core welding area, transition welding area, and edge welding area; Based on the bubble migration trajectory prediction data, the predicted spatial position of each real bubble at the start of the high-temperature welding process is extracted, and the minimum distance between the predicted spatial position and the core welding area is calculated. Based on the instantaneous velocity, bubble area, and minimum distance of each real bubble, the risk coefficient of each bubble is calculated according to a preset weighted formula; Based on the risk coefficient of each bubble, the risk coefficients of all real bubbles are summed to obtain a comprehensive risk index. The comprehensive risk index is then compared with preset low-risk, medium-risk, and high-risk thresholds. When the comprehensive risk index is less than the low risk threshold, a normal pass warning signal is output; when the comprehensive risk index is between the low risk threshold and the medium risk threshold, a maintenance reminder warning signal is output; when the comprehensive risk index is between the medium risk threshold and the high risk threshold, a shutdown and re-inspection warning signal is output; when the comprehensive risk index is greater than the high risk threshold, an emergency shutdown warning signal is output.

[0013] Furthermore, based on the instantaneous velocity and direction of motion of each real bubble, a method for evaluating the consistency of preheating process parameters and obtaining optical visual inspection results of diode surface defects includes: Based on the instantaneous velocity of each real bubble, the instantaneous velocities of all real bubbles are obtained, forming a velocity sample set; based on the motion direction of each real bubble, the motion directions of all real bubbles are obtained, forming a direction sample set. Based on the velocity sample set, the velocity mean and velocity standard deviation of the velocity sample set are calculated; based on the direction sample set, the direction mean and direction standard deviation of the direction sample set are calculated. Based on the average speed, the average speed is compared with the preset theoretical characteristic speed to calculate the speed deviation rate; based on the average direction, the average direction is compared with the temperature gradient direction of the preheating process to calculate the direction deviation angle. When the speed deviation rate is less than the preset first deviation threshold and the directional deviation angle is less than the preset second deviation threshold, the preheating process parameters are deemed to be consistent and qualified. When the speed deviation rate is greater than or equal to the first deviation threshold or the directional deviation angle is greater than or equal to the second deviation threshold, it is determined that there is an abnormality in the preheating process parameters, and corresponding process adjustment suggestions are output according to the magnitude of the speed deviation rate and the directional deviation angle.

[0014] In summary, due to the adoption of the above-mentioned optical visual inspection method for diode surface defects, the beneficial effects of this invention are: This invention avoids the cross-temporal registration problem caused by changes in bubble morphology before and after preheating by acquiring only bright-field and dark-field image sequences after preheating, eliminating the need for a reference image before preheating. This significantly reduces production line modification costs and hardware deployment difficulties. Simultaneously, by utilizing the optically opposite properties of bubbles appearing as dark spots in bright-field and bright spots in dark-field, dual verification is performed using complementary constraints composed of spatial distance, area ratio, circularity difference, and gray-level contrast product. This effectively suppresses false positives caused by background texture and single-modal noise, solving the problem of numerous false defects caused by misaligned bubble positions in existing sparse coding models. This invention accurately extracts the instantaneous velocity and direction of motion of each bubble by performing cross-frame correlation tracking and motion curve fitting on a candidate bubble sequence, adaptively selecting either a linear or parabolic fitting curve. Furthermore, the instantaneous velocity vector is projected onto the temperature gradient direction and dimensionlessly converted with theoretical characteristic velocities. A migration number probability density distribution curve is generated using kernel density estimation. Based on the main peak interval, it adaptively determines whether the bubble satisfies the physical laws of thermally induced migration, thus accurately distinguishing between real bubbles and false defects. This method requires no preset velocity or direction thresholds, is entirely data-driven, and overcomes the shortcomings of existing fixed threshold methods that cannot adapt to batch-to-batch process fluctuations, significantly improving the accuracy of distinguishing between real and false defects. This invention, based on the actual bubble movement curve type, combines temperature gradient decay curves and viscosity decay functions for physical correction to accurately predict the spatial position of bubbles in subsequent high-temperature welding processes. It divides the welding area into three risk-level zones: a core welding zone, a transition welding zone, and an edge welding zone. The risk coefficient of each bubble is calculated using its area, instantaneous velocity, and minimum distance from the core welding zone. These factors are then summed to obtain a comprehensive risk index, which is then output as a four-level early warning signal. This represents a significant leap from defect detection to welding reliability risk warning, solving the problem of existing technologies that only output a binary result of pass / fail and cannot assess the specific impact of bubbles on subsequent processes. This invention achieves a quantitative assessment of the uniformity of the temperature gradient and the stability of the conveyor belt in the preheating process by statistically analyzing the instantaneous velocity and direction of motion of all real bubbles, calculating the velocity deviation rate and direction deviation angle, and comparing them with theoretical characteristic velocities and temperature gradient directions. When process parameters are abnormal, the statistical evaluation model outputs specific process adjustment suggestions such as adjusting the heating lamp power, correcting the conveyor belt speed, or changing the encapsulation material. This upgrades the detection system from passive quality inspection to an active process diagnostic tool, solving the inherent deficiency of existing detection methods that cannot guide process optimization. Attached Figure Description

[0015] Figure 1 A flowchart of the present invention is shown; Figure 2 A flowchart illustrating the generation of the migration number probability density distribution curve according to the present invention is shown; Figure 3 The flowchart of the present invention for determining whether a bubble satisfies the physical laws of thermally induced migration is shown. Detailed Implementation

[0016] The optical visual inspection method for diode surface defects according to the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] To more clearly and intuitively demonstrate the practical application effects and advantages of the optical visual inspection method for diode surface defects of the present invention, and to verify its feasibility and effectiveness, the present invention will be further described below with reference to embodiments. Through specific scenario simulations and data calculations, the method is explained in detail how it plays a role in the actual detection of diode surface defects, helping readers to better understand the technical details and practical value of the invention. The present invention will be further described below with reference to embodiments; The Marangoni migration, a type of thermally induced bubble migration in existing technologies, is based on the classical physical model that assumes bubbles are in a continuous liquid medium and that a surface tension gradient caused by a temperature gradient drives the bubbles to move towards higher temperature regions. However, in the preheating process of diode encapsulation, the encapsulation material, such as epoxy resin, gradually transforms from a highly elastic state to a glassy state or has already completed the cross-linking and curing reaction during the preheating stage. In this state, the encapsulation matrix exhibits solid-state mechanical characteristics, and the bubbles are not in a freely flowing liquid mother liquor, but are confined within the closed cavities of the solid or viscoelastic matrix. Within the specific temperature range and time window of the preheating process of this invention, the bubble motion behavior, while not entirely equivalent to the classic liquid Marangoni migration, still follows the thermodynamically driven directional migration law. Specifically, the driving mechanism is as follows: the non-uniform temperature field inside the molding compound material during preheating leads to an asymmetric distribution of thermal expansion stress and local creep behavior. The gas in the closed cavity expands due to heat, generating internal pressure. When the internal pressure reaches a critical equilibrium with the local yield strength of the solid matrix, micro- and nano-scale creep slip or stress-induced diffusion will occur at the interface between the bubble wall and the molding compound. Since the direction of the temperature gradient determines the dominant direction of material softening and thermal expansion, the equivalent centroid of the bubble will undergo directional, low-speed, and continuous macroscopic displacement along the temperature gradient direction. Based on the aforementioned mechanistic insights, this invention unifies and abstracts the quasi-static thermodynamic migration process in this solid matrix into a description of the physical laws of thermally induced migration. Although the microscopic mechanism differs from liquid Marangoni convection, in terms of macroscopic kinematics, both exhibit monotonous, directional migration of bubbles along the temperature gradient direction. Under preset preheating process parameters, their migration velocity has a stable mapping relationship with the temperature gradient and the material's viscoelastic parameters. This invention utilizes this macroscopic kinematic characteristic—the statistical regularity of instantaneous velocity and direction of motion—as a criterion for distinguishing between real bubbles and random pseudo-defects, such as dust and scratches. Therefore, the viscosity coefficient involved in this invention should be broadly understood as the equivalent viscosity coefficient characterizing the resistance of the encapsulated matrix to bubble migration under preheating conditions. This parameter can be obtained in advance through process parameter inversion calibration of the preheating process or dynamic thermomechanical analysis (DMA) of the material. Example

[0018] See Figure 1 - Figure 3 An optical visual inspection method for diode surface defects includes the following steps: After the diode completes its preheating process, a series of multimodal images under bright-field illumination are continuously acquired, recording the evolution of the dark spot morphology of each bubble under reflected light over time, resulting in a bright-field image sequence after preheating. Similarly, after the diode completes its preheating process, a series of multimodal images under dark-field illumination are continuously acquired, recording the evolution of the bright spot morphology of each bubble under scattered light over time, resulting in a dark-field image sequence after preheating. By acquiring bright-field and dark-field image sequences only after preheating, a reference image before preheating is not required, avoiding the challenge of image registration before and after preheating. Furthermore, the optical response characteristics of the opposite dark and bright spots of bubbles in bright and dark fields are utilized. This provides a physical basis for subsequent verification and testing, and solves the problems of high production line modification costs and large registration errors caused by existing technologies that rely on fixed position assumptions and preheating baseline acquisition. Compared with existing technologies that require image acquisition before and after preheating and complex rigid body registration or sparse coding alignment, this method only requires one acquisition after preheating, which avoids the risk of cross-temporal registration failure caused by changes in bubble morphology before and after preheating from the source. Moreover, it does not require adding a preheating inspection station, which significantly reduces hardware costs and production line modification difficulty. This is something that existing methods based on fixed position assumptions or cross-temporal registration cannot achieve.

[0019] Based on the preheated bright field image sequence and the preheated dark field image sequence, morphological complementary matching is performed to obtain the bubble candidate sequence for each frame of the image. It should be noted that the method for obtaining the bubble candidate sequence for each frame of the image by performing morphological complementary matching based on the preheated bright-field image sequence and the preheated dark-field image sequence includes: Based on the bright field image sequence after preheating, each frame of the bright field image sequence is extracted to obtain multiple candidate dark spot regions. The spatial location, dark spot area, dark spot circularity, and dark spot grayscale contrast of each candidate dark spot region are recorded. Based on the dark field image sequence after preheating, the corresponding frame images in the dark field image sequence after preheating are extracted to obtain multiple candidate bright spot regions. The spatial location, bright spot area, bright spot circularity and bright spot grayscale contrast of each candidate bright spot region are recorded. Based on multiple candidate dark spot regions and multiple candidate bright spot regions, the candidate dark spot regions and candidate bright spot regions in the same frame are paired according to their spatial location to obtain paired dark spot regions and bright spot regions. Based on the paired dark spot regions and bright spot regions, the spatial distance, area ratio, roundness difference and gray-level contrast product between each pair of paired dark spot regions and bright spot regions are calculated. Determine whether the spatial distance, area ratio, circularity difference, and grayscale contrast product satisfy the complementary constraint conditions predefined based on the optical opposite characteristics of bright and dark fields; If the spatial distance, area ratio, roundness difference, and gray-level contrast product all satisfy the complementary constraint conditions predefined based on the optical opposite characteristics of bright and dark fields, then the spatial positions corresponding to the paired dark spot region and the bright spot region are determined as the bubble candidate regions of the frame image. Based on the candidate bubble regions of each frame, the candidate bubble regions of each frame are summarized to obtain a candidate bubble sequence. By using nearest neighbor pairing and complementary constraints of the bright and dark field images within the same frame, where the complementary constraints are spatial distance constraints, area ratio constraints, circularity difference constraints, and gray-level contrast product constraints, the opposite optical properties of bubbles appearing as dark spots in bright fields and bright spots in dark fields are used for dual verification. This enables the high-confidence identification of candidate bubble regions in a single frame image, effectively suppressing false positives caused by background texture and single-modal noise, and solving the problem of a large number of false defects caused by the disordered bubble position in existing sparse coding models. Compared with existing technologies that rely on only a single modal image for defect detection, this method uses the physical complementary properties of bright and dark fields to achieve self-reference verification. Even after the bubble migrates, its bright and dark field morphology still maintains an opposite correspondence, and it can be accurately identified without relying on the assumption of fixed position. This is an accuracy that traditional single-modal detection methods cannot achieve.

[0020] Specifically, complementary constraints include: Spatial distance constraint: The spatial distance between the dark spot region and the bright spot region is less than the maximum allowable value of the optical misalignment calibration residual between the bright field imaging system and the dark field imaging system. The area ratio constraint is that the ratio of the dark spot area to the bright spot area is within the range of the area ratio determined by the theoretical ratio of the absorption cross section of the bubble in the bright field to the scattering cross section in the dark field. The upper and lower limits of this area ratio range adaptively shrink as the bubble size increases. The roundness difference constraint is that the absolute value of the difference between the roundness of dark spots and the roundness of bright spots is less than the preset shape consistency tolerance based on the consistency of bubble shape in both bright and dark fields. The product constraint of gray-scale contrast means that the product of the gray-scale contrast of dark spots and the gray-scale contrast of bright spots is greater than the minimum detection threshold determined based on the complementary enhancement principle of signal-to-noise ratio in bright and dark field imaging. This minimum detection threshold is adaptively increased as the background noise level increases. False alarms caused by optical misalignment between bright and dark fields are eliminated by spatial distance constraints; the area ratio adaptively shrinks with bubble size to meet the detection needs of bubbles of different sizes; non-bubble morphology interference is eliminated by roundness difference constraints; and noise resistance is enhanced by the gray-scale contrast product adaptively increasing with noise level constraints. These dynamic constraints avoid the shortcomings of traditional fixed thresholds, which are greatly affected by batch variations and environmental fluctuations, thus improving detection accuracy and solving the problem that threshold parameters in existing technologies are difficult to adapt to dynamic changes in the production line. Compared with existing technologies that use static parameters such as fixed area thresholds and fixed gray-scale thresholds, the constraints of this method can be automatically adjusted according to the actual bubble size and background noise level, maintaining stable detection performance under different batches and different lighting conditions, which is something that static threshold methods cannot achieve.

[0021] Specifically, the method for determining the area ratio range is shown in the following example: An optical simulation model of a bubble is constructed. The bubble diameter, the refractive index of the encapsulation, and the illumination wavelength are used as input parameters to calculate the absorption cross section of the bubble under bright field illumination and the scattering cross section of the bubble under dark field illumination. Dividing the absorption cross-section value by the scattering cross-section value yields the baseline value of the theoretical area ratio. Then, using Monte Carlo simulation, the variation range of the area ratio under different bubble sizes, different batches of encapsulating materials, and different illumination angle fluctuations is simulated. The 95% confidence interval in the simulation results is taken as the upper and lower limits of the dynamic range. Finally, in actual testing, based on the statistical distribution of bubble size of the current batch of diodes, the upper and lower limits of the corresponding area ratio range are selected from the pre-generated lookup table to achieve adaptive adjustment of the area ratio range according to the bubble size; Larger bubbles tend to have a more stable ratio of bright-field absorption cross-section to dark-field scattering cross-section, requiring a narrowing of the area ratio range to eliminate spurious defects. Conversely, smaller bubbles exhibit greater ratio fluctuations, necessitating a wider range to preserve the true size of the bubble. Input data includes the bubble diameter range, material refractive index, and wavelength from the simulation. Output data comprises the upper and lower limits of the area ratio range for different size grades. This configuration ensures that the area ratio constraint in the complementary constraint conditions has a physical basis and adaptability, unlike the arbitrarily set fixed thresholds in existing technologies.

[0022] Based on the candidate bubble sequence, the movement curve of each bubble is obtained by fitting. It should be noted that the methods for fitting the movement curve of each bubble based on the bubble candidate sequence include: Based on the bubble candidate sequence, cross-frame correlation tracking is performed on the candidate regions belonging to the same physical bubble in the bubble candidate sequence. Kalman filtering is used to predict the expected spatial position of the bubble in the next frame image. The expected spatial position is then matched with the actual detected candidate region in the next frame image using Hungarian optimal matching to determine the spatial position of the same bubble in each frame image, thus obtaining the spatial position time sequence of each bubble. Based on the spatial location time sequence of each bubble, least squares fitting is performed using a linear model and a parabolic model respectively to obtain linear fitting curves and parabolic fitting curves. Calculate the first residual sum of squares between the linear fitting curve and the spatial location time series, and the second residual sum of squares between the quadratic parabolic fitting curve and the spatial location time series, and compare the magnitudes of the first residual sum of squares and the second residual sum of squares. If the sum of squares of the first residual is less than the sum of squares of the second residual, then the linear fitting curve is selected as the movement curve of the bubble, which corresponds to the physical model of the bubble undergoing uniform thermal migration in the preheating process. If the second residual sum of squares is less than the first residual sum of squares, a quadratic parabolic fitting curve is selected as the bubble's movement curve. This curve corresponds to the physical model of the bubble undergoing uniformly accelerated thermal migration during the preheating process. Through cross-frame tracking combining Kalman filtering and Hungarian matching, stable correlation is achieved in multi-bubble scenarios, avoiding tracking loss due to bubble migration trajectory intersections or brief occlusions. By comparing the residuals of the linear fitting curve and the quadratic parabolic fitting curve, a uniform or uniformly accelerated motion model is adaptively selected, overcoming the shortcomings of traditional methods where a single motion assumption cannot fit actual non-uniform migration, improving the fitting accuracy of the movement curve, and solving the problem that existing technologies cannot track the dynamic displacement trajectory of bubbles. Compared with traditional detection methods that assume uniform bubble motion or ignore acceleration, this method can automatically identify the acceleration state of bubble migration and select a matching physical model, achieving accurate fitting of non-uniform motion in thermal migration.

[0023] Specifically, the state vector of the Kalman filter is defined as the spatial position coordinates of each bubble on the two-dimensional image plane and its velocity component, with a total of four dimensions; the observation vector of the Kalman filter is the center coordinates of the candidate bubble region obtained by morphological complementary matching in the current frame, with a total of two dimensions; the process noise covariance matrix and the observation noise covariance matrix of the Kalman filter are pre-calibrated according to the statistical characteristics of bubble position jitter in the actual image sequence, that is, a sequence of images containing uniformly moving bubbles is acquired, the true trajectory is manually marked, and the variance of the position prediction error and the observation error is calculated as the diagonal element of the noise covariance matrix; the prediction step of the Kalman filter outputs the expected position and prediction covariance of each bubble in the next frame; the update step of the Kalman filter uses the actual observation position of the current frame to correct the state estimate.

[0024] The Hungarian matching algorithm takes as input the set of all predicted bubble positions from the Kalman filter and the set of all actually detected candidate region positions in the current frame, constructing a cost matrix. The cost of the cost matrix is ​​defined as the Euclidean distance between the predicted and candidate positions. The Hungarian algorithm finds the matching scheme that minimizes the total cost, treating successfully matched candidate regions as continuations of the same physical bubble, and initializing unmatched candidate regions as new bubbles. Tracking terminates after several consecutive frames of unmatched bubbles. The Kalman filter provides smooth predictions, reducing jumps caused by noise; the Hungarian matching solves the identity association problem in multi-bubble scenarios. The combination of the two achieves stable and real-time bubble trajectory tracking. The input data is the set of bubble candidate region positions for each frame; the output data is the temporal sequence of the spatial positions of each bubble. This setup avoids tracking interruptions caused by temporary occlusion or brief disappearance of bubbles, which is impossible with existing single-frame detection methods.

[0025] Based on the movement curve of each bubble, the instantaneous velocity and direction of motion of each bubble are calculated. It should be noted that the methods for calculating the instantaneous velocity and direction of motion of each bubble based on its movement curve include: Based on the movement curve of each bubble, determine the type of movement curve of each bubble. If the movement curve is a linear fitting curve, take the slope of the movement curve as the constant instantaneous velocity of the bubble throughout its entire journey, and extract the spatial direction angle of the movement curve as the instantaneous direction of the bubble throughout its entire journey. If the movement curve is a quadratic parabolic fitting curve, the first derivative of the movement curve is calculated to obtain an expression for the linear change of instantaneous velocity with time. Based on this expression, the value of the expression at each acquisition moment is taken as the instantaneous velocity of the corresponding frame. Based on this expression, the instantaneous direction is calculated frame by frame according to the tangent direction of the curve at each acquisition moment. For a first-order linear fitting curve, which is uniform velocity, the constant instantaneous velocity and motion direction are directly extracted. For a quadratic parabolic fitting curve, which is uniformly accelerated, the expression for the change of instantaneous velocity with time and the motion direction frame by frame are obtained by differentiation. This achieves accurate extraction of motion parameters under different motion modes, providing reliable velocity and direction data for subsequent discrimination based on physical laws, and solving the problem that traditional methods cannot obtain the dynamic motion parameters of bubbles.

[0026] Based on the instantaneous velocity and direction of motion of each bubble, determine whether each bubble satisfies the physical laws of thermally induced migration; If a bubble satisfies the physical laws of thermal migration, it is determined to be a real bubble; if a bubble does not satisfy the physical laws of thermal migration, it is determined to be a false defect. It should be noted that the methods for determining whether each bubble satisfies the physical laws of thermally induced migration based on the instantaneous velocity and direction of motion of each bubble include: Based on the instantaneous velocity and direction of motion of each bubble, the instantaneous velocity vector of each bubble is calculated; Based on the instantaneous velocity vector of each bubble, the instantaneous velocity vector of each bubble is projected onto the temperature gradient direction of the preheating process to obtain the projected velocity component of each bubble. Obtain the viscosity coefficient and preheating time of the molding compound material. Based on the viscosity coefficient and preheating time of the molding compound material, calculate the theoretical characteristic velocity. Based on the theoretical characteristic velocity and the projected velocity component of each bubble, divide the projected velocity component of all bubbles by the theoretical characteristic velocity to obtain the dimensionless migration number of each bubble. Based on the dimensionless transference number of all bubbles, the kernel density of the dimensionless transference number of all bubbles is estimated to generate the transference number probability density distribution curve. Based on the probability density distribution curve of the migration number, the main peak interval is determined by finding the local maximum point of the probability density distribution curve; Based on the main peak interval and the dimensionless migration number, determine whether the dimensionless migration number falls within the main peak interval; If the dimensionless migration number falls within the main peak interval, the bubble is determined to satisfy the physical law of thermally induced migration; if the dimensionless migration number falls outside the main peak interval, the bubble is determined to not satisfy the physical law of thermally induced migration. By dividing the bubble projection velocity component by the theoretical characteristic velocity to obtain the dimensionless migration number, the influence of process parameter differences between different batches on the absolute velocity is eliminated. The migration number probability density distribution curve is adaptively generated through kernel density estimation, and the statistical main peak interval is used to automatically distinguish between real bubbles that conform to the physical laws of thermally induced migration and pseudo-defects that do not conform to the laws. Pseudo-defects, such as static interferences such as surface dust and scratches, avoid the subjectivity of traditional methods that rely on manually setting fixed velocity and direction thresholds. This solves the problem of misjudgment caused by fixed thresholds being difficult to adapt to process fluctuations between batches in existing technologies. Compared with existing technologies that rely on empirical thresholds or simple statistical discrimination, this method does not require any preset velocity or direction thresholds. The discrimination boundary is determined adaptively by data, and the discrimination accuracy can be maintained even in different batches and under different process conditions.

[0027] It should be further explained that when projecting the instantaneous velocity vector of the bubble onto the temperature gradient direction, the dimensional transformation relationship between the actual velocity in three-dimensional physical space and the pixel displacement acquired from the two-dimensional image plane is involved. This invention, specifically addressing the unique characteristics of diode packaging inspection scenarios, employs the following techniques to handle the conversion from three-dimensional velocity to two-dimensional projection and the problem of missing depth information: Firstly, in the hardware architecture design of the detection system, the optical axes of both the bright-field imaging system and the dark-field imaging system are perpendicular to the main plane of the diode encapsulation, that is, perpendicular to the plane where the diode pins are arranged. During the preheating process, the diode is placed in a fixed position on the conveyor belt. The temperature gradient direction inside the encapsulation is pre-calibrated, and its main gradient component lies within a two-dimensional cross-section parallel to the diode's main plane. In the depth direction, because the encapsulation thickness is much smaller than the planar dimension, and the heating boundary conditions on the upper and lower surfaces are symmetrical, the temperature gradient value in this direction is extremely small and its contribution to the macroscopic migration trajectory of the bubble is negligible. Secondly, during preheating, the bubble expands due to heat and moves along the main direction of the temperature gradient. Its migration trajectory appears as a directional curve along the principal plane of the diode in three-dimensional space. Since the imaging optical axis is perpendicular to this principal plane, the perspective scaling effect produced when the tiny displacement of the bubble in the depth direction is projected onto the two-dimensional image is extremely weak. The system uses a telecentric lens or a long focal length objective to further suppress perspective errors, ensuring that the change in the bubble's position in the two-dimensional image has a fixed linear proportional relationship with its actual displacement on the physical plane. This proportional relationship is pre-determined using a spatial calibration plate and is pre-stored as a scale factor in the detection algorithm. Finally, for calculating the projection of the bubble velocity vector onto the temperature gradient direction, the system does not directly recover the bubble's three-dimensional velocity vector. Instead, it pre-calibrates the temperature gradient direction as a direction vector within the two-dimensional image plane. This direction vector is obtained by projecting the three-dimensional temperature gradient direction along the imaging optical axis onto the diode's principal plane, and then mapping it to the pixel coordinate system via camera intrinsic parameters. Therefore, when calculating the projected velocity component, it is only necessary to perform a vector dot product operation between the instantaneous bubble velocity vector tracked in the two-dimensional image and this pre-stored two-dimensional direction vector to obtain the projected velocity component. This method avoids the ill-posed problem of depth information recovery at the physical level, significantly reducing algorithm complexity while ensuring computational accuracy, and meeting the stringent real-time requirements of online production line inspection.

[0028] Specifically, the method for obtaining the temperature gradient direction in the preheating process is as follows: Miniature thermocouple arrays are arranged along the axial and radial directions of the diode within the diode molding compound or on the tooling fixture; during the stable operation phase of the preheating process, temperature data from each temperature measuring point are collected synchronously. Interpolation fitting is performed on the collected spatial temperature field data to construct a temperature distribution map of the region where the encapsulator is located within the preheating temperature zone. The normal direction of the isothermal surface in this temperature distribution map is calculated. Since the placement posture of the diode encapsulator in the preheating station is fixed, the two-dimensional projection direction of this normal direction in the image acquisition coordinate system is the temperature gradient direction of the preheating process referred to in this method. The vector of the temperature gradient direction is pre-stored in the memory of the detection system as a system configuration parameter. During the actual detection process, for each bubble in the image frame, the instantaneous velocity vector is projected onto the pre-stored direction vector to obtain the projected velocity component of the bubble along the temperature gradient direction.

[0029] Specifically, the method for calculating the theoretical characteristic velocity is as follows: First, the equivalent viscosity coefficient of the batch of molding compound material under typical temperature conditions in the preheating process is determined in advance by using the technical data sheet of the molding compound material or by dynamic thermomechanical analysis. This coefficient reflects the degree of macroscopic hindrance of the molding compound matrix to the migration of air bubbles. Secondly, the set value of the conveyor belt speed and the effective length of the heating zone in the preheating process are obtained, and then the actual heating duration of the batch of diodes in the preheating zone is calculated, i.e., the preheating time. Subsequently, based on the power distribution of the infrared heating lamp tubes and the thermal conductivity of the encapsulation during the preheating process, the average temperature gradient value along the length of the diode inside the encapsulation was determined through finite element thermal simulation or actual measurement and calibration of the embedded thermocouple.

[0030] Finally, the equivalent viscosity coefficient, preheating time, average temperature gradient, and the bubble thermal expansion constant determined by the gas law are comprehensively calculated. The calculation rule is as follows: the product of the average temperature gradient, preheating time, and thermal expansion constant is used as the numerator, and the equivalent viscosity coefficient is used as the denominator. The resulting quotient is the theoretical characteristic velocity. This theoretical characteristic velocity characterizes the baseline migration rate that an ideal bubble should possess under the current preheating process settings when driven by thermodynamics.

[0031] Specifically, the kernel function used for kernel density estimation is a Gaussian kernel, and the selection of its bandwidth parameter follows these steps: The first step is to collect sample diodes containing a large number of real bubbles and determine the dimensionless transport number distribution range of the real bubbles through manual calibration or high-precision X-ray verification. The second step involves using a 10x cross-validation method to randomly divide the sample set into ten subsets, using nine subsets for training and one subset for validation in a loop, and calculating the log-likelihood loss under different bandwidth values. The third step is to select the bandwidth value that minimizes the average log-likelihood loss as the final bandwidth parameter. After training, the dimensionless transference number of all candidate bubbles in the current batch of diodes is input into the model to generate probability density distribution curves. The migration numbers of real bubbles, influenced by the same temperature gradient and material properties, should cluster near the main peak; the migration numbers of spurious defects are randomly distributed, forming discrete points far from the main peak. By finding the local maxima of the probability density distribution curve, the main peak interval can be automatically identified without manually setting a velocity threshold. The input data is a dimensionless migration number array of all candidate bubbles in the current batch; the output data is the migration number probability density distribution curve and its main peak interval. This setting makes the discrimination boundary completely data-driven, adaptable to process fluctuations in different batches, and solves the problem that fixed thresholds in existing technologies cannot adapt to dynamic changes in the production line.

[0032] Based on the actual bubble movement curve, the spatial position in subsequent processes is predicted, and bubble migration trajectory prediction data is obtained. It should be noted that methods for predicting the spatial position of bubbles in subsequent processes based on the actual bubble movement curve, and obtaining bubble migration trajectory prediction data, include: Based on the movement curve of the real bubble, the type of the real bubble movement curve is obtained. If the movement curve is a linear fitting curve, the slope of the movement curve is used as the constant migration speed. Based on the constant migration speed, the spatial position of the bubble in the subsequent process is calculated using a linear extrapolation formula. If the moving curve is a quadratic parabola fitting curve, then the first derivative of the moving curve is taken to obtain the instantaneous velocity expression; based on the instantaneous velocity expression, the instantaneous velocity expression is integrated to obtain a quadratic function of displacement changing with time; based on the quadratic function of displacement changing with time, the spatial position of the bubble in the subsequent process is calculated by extrapolation of this quadratic function. The temperature gradient decay curve and viscosity decay function at the end of the preheating process are obtained. Based on the temperature gradient decay curve, viscosity decay function and the spatial position of bubbles in subsequent processes, the velocity parameters in the linear extrapolation formula or quadratic function are corrected to obtain bubble migration trajectory prediction data. Based on linear and parabolic fitting curves, linear extrapolation or quadratic functions are used respectively, and physical corrections are made by combining the temperature gradient decay and viscosity change after the preheating process. This makes the predicted bubble migration trajectory closer to the actual physical process, solving the problem that existing technologies cannot predict the spatial position of bubbles in subsequent welding processes, and providing a data foundation for welding risk early warning. Compared with existing technologies that only output a binary result of whether the current defect exists, this method realizes the quantitative prediction of the spatial position of bubbles in subsequent high-temperature welding processes. The physical correction model considers the influence of temperature gradient decay and viscosity change on migration speed, and the prediction accuracy is significantly higher than that of simple mathematical extrapolation.

[0033] Specifically, the temperature gradient decay curve at the end of the preheating process refers to the curve showing the decay relationship between the maximum temperature gradient value and time as the temperature field inside the encapsulation gradually homogenizes after the diode leaves the heating zone and enters the natural cooling or air cooling section. The method for obtaining this temperature gradient decay curve is as follows: During the debugging phase of the testing system, a matching diode of the same model was used, with a miniature thermocouple sensor pre-embedded inside its plastic casing. Simulate the actual production line cycle, so that the diode under test goes through a complete preheating process and enters the subsequent transfer station; use a data acquisition card to continuously record the temperature change process of each temperature measurement point from leaving the preheating temperature zone to entering the welding process. The data on the temperature change process is processed to extract the temperature gradient amplitude at fixed time intervals. A scatter plot is then created with time on the horizontal axis and temperature gradient amplitude on the vertical axis, and a smooth fit is applied. The resulting fitted line segment is the temperature gradient decay curve. This curve is pre-set as an empirical model curve in the detection algorithm to correct for the decay trend of bubble migration speed after preheating.

[0034] Specifically, the viscosity decay function describes the physical process during the cooling phase after preheating, where the equivalent viscosity coefficient of the encapsulant matrix gradually increases due to the decrease in temperature, leading to a gradual increase in resistance to bubble migration. This viscosity decay function is obtained from material property parameter tables provided by the encapsulant material supplier or dynamic thermomechanical analysis test reports issued by a third-party materials laboratory. These test reports typically provide curves showing the changes in storage modulus and loss modulus of the encapsulant material with temperature in the cured state. Based on the time-temperature equivalence principle and empirical rules of the Arrhenius type, the detection system converts the modulus change data in the test report into a mapping table of the equivalent viscosity coefficient versus temperature. This mapping table is the discretized expression of the viscosity decay function. In actual prediction calculations, the system obtains the corresponding equivalent viscosity coefficient by looking up this mapping table based on the estimated encapsulant temperature value at the current moment, and then dynamically corrects the bubble migration speed. If measured data is lacking, empirical parameters of thermomechanical properties commonly used in similar epoxy encapsulants can be used as default values.

[0035] Based on bubble migration trajectory prediction data, a risk warning for high-temperature welding processes is output. It should be noted that the methods for outputting risk warnings for high-temperature welding processes based on bubble migration trajectory prediction data include: Obtain a distribution map of the welding area in the high-temperature welding process, and divide the welding area into three risk level areas: core welding area, transition welding area, and edge welding area; Based on the bubble migration trajectory prediction data, the predicted spatial position of each real bubble at the start of the high-temperature welding process is extracted, and the minimum distance between the predicted spatial position and the core welding area is calculated. Based on the instantaneous velocity, bubble area, and minimum distance of each real bubble, the risk coefficient of each bubble is calculated according to a preset weighted formula; Based on the risk coefficient of each bubble, the risk coefficients of all real bubbles are summed to obtain a comprehensive risk index. The comprehensive risk index is then compared with preset low-risk, medium-risk, and high-risk thresholds. When the comprehensive risk index is less than the low risk threshold, a normal pass warning signal is output; when the comprehensive risk index is between the low risk threshold and the medium risk threshold, a maintenance prompt warning signal is output; when the comprehensive risk index is between the medium risk threshold and the high risk threshold, a shutdown re-inspection warning signal is output; when the comprehensive risk index is greater than the high risk threshold, an emergency shutdown warning signal is output. By classifying the welding area and using risk coefficients for bubble location, instantaneous velocity, bubble area, and minimum distance, the predicted bubble migration trajectory data is transformed into a quantified comprehensive risk index, which outputs four levels of early warning signals. This achieves a functional leap from defect detection to welding reliability risk early warning, solving the problem that existing technologies cannot assess the specific impact of bubbles on subsequent welding processes and cannot guide production lines to take graded response measures. Compared to existing technologies that only output a simple pass or fail judgment, this method can quantitatively assess the threat level of each bubble to welding reliability and give graded early warning instructions based on the comprehensive risk index, namely, normal pass, maintenance reminder, shutdown for re-inspection, and emergency shutdown, providing production line operators with clear and actionable guidelines.

[0036] Specifically, the weighted formula for the risk coefficient adopts a three-tiered structure: The first layer is the bubble's inherent properties layer, including bubble area and instantaneous velocity, reflecting the bubble's expansion potential and migration activity; the second layer is the spatial threat layer, including the minimum distance between the predicted bubble location and the core welding area, with smaller distances indicating greater threats; the third layer is the process correction layer, including the temperature gradient uniformity index of the current preheating process. The sub-risk value for each layer is calculated using the corresponding membership function: bubble area is mapped using an S-shaped membership function, with larger bubbles having higher risk weights; instantaneous velocity is mapped using a Gaussian membership function, where excessively slow or fast velocities increase risk; and minimum distance is mapped using an inverse proportional membership function, where closer distances result in higher risk. The risk coefficient is obtained by multiplying each layer's sub-risk value by a preset weighting coefficient and then summing the results.

[0037] The weighting coefficients are determined using the analytic hierarchy process (AHP): First, multiple semiconductor packaging process experts conduct pairwise comparisons of four factors—bubble area, movement speed, spatial distance, and temperature uniformity—to construct a judgment matrix. Then, the eigenvector corresponding to the largest eigenvalue of the matrix is ​​calculated, and the weights of each factor are obtained after normalization. The core welding area is the region with the highest stress concentration during high-temperature welding, and the risk of bubbles approaching this area should be significantly amplified; large-area bubbles are more prone to expansion and bursting during welding, thus receiving a higher weight. Specific numerical values ​​for the weighting coefficients are as follows: bubble area weight 0.35, instantaneous speed weight 0.25, spatial distance weight 0.3, temperature uniformity weight 0.1, and the sum of all weights is 1. The weighted formula for the risk coefficient takes the area, instantaneous velocity, and minimum distance of each bubble as input data, and outputs the risk coefficient as output data. This setup gives risk quantification physical meaning and empirical support, distinguishing it from simple counting or fixed threshold determination in existing technologies.

[0038] Based on the instantaneous velocity and direction of motion of each real bubble, the consistency of the preheating process parameters is evaluated, and the optical visual inspection results of diode surface defects are obtained. It should be noted that the methods for evaluating the consistency of preheating process parameters and obtaining optical visual inspection results of diode surface defects based on the instantaneous velocity and direction of motion of each real bubble include: Based on the instantaneous velocity of each real bubble, the instantaneous velocities of all real bubbles are obtained, forming a velocity sample set; based on the motion direction of each real bubble, the motion directions of all real bubbles are obtained, forming a direction sample set. Based on the velocity sample set, the velocity mean and velocity standard deviation of the velocity sample set are calculated; based on the direction sample set, the direction mean and direction standard deviation of the direction sample set are calculated. Based on the average speed, the average speed is compared with the preset theoretical characteristic speed to calculate the speed deviation rate; based on the average direction, the average direction is compared with the temperature gradient direction of the preheating process to calculate the direction deviation angle. When the speed deviation rate is less than the preset first deviation threshold and the directional deviation angle is less than the preset second deviation threshold, the preheating process parameters are deemed to be consistent and qualified. When the speed deviation rate is greater than or equal to the first deviation threshold or the direction deviation angle is greater than or equal to the second deviation threshold, it is determined that there is an abnormality in the preheating process parameters. Based on the magnitude of the speed deviation rate and the direction deviation angle, the statistical evaluation model outputs corresponding process adjustment suggestions, including adjusting the heating lamp power, correcting the conveyor belt speed, or replacing the encapsulation material. By statistically analyzing the instantaneous velocity and direction of motion of all real bubbles, calculating the velocity deviation rate and direction deviation angle, and comparing them with the theoretical characteristic velocity and temperature gradient direction, a quantitative assessment of the uniformity of the temperature gradient and the stability of the conveyor belt in the preheating process is achieved. When process parameters are abnormal, specific process adjustment suggestions are output, such as adjusting the power of the heating lamps, correcting the conveyor belt speed, or changing the encapsulation material. This solves the shortcomings of existing detection methods that only output binary results of pass or fail and cannot guide process optimization, upgrading the detection system to a process diagnostic tool. Compared with existing technologies that only indicate whether the product is qualified, this method can accurately locate the specific causes of process abnormalities, namely uneven temperature gradient, conveyor belt speed fluctuations, and abnormal material viscosity, and provide targeted adjustment suggestions, upgrading the detection system from passive quality inspection to proactive process optimization.

[0039] Specifically, the statistical evaluation model adopts a rule mapping structure based on a decision tree. The input layer consists of three statistical quantities: speed deviation rate, direction deviation angle, and speed standard deviation. The output layer provides the process consistency judgment result and specific process adjustment suggestions. The rule mapping library of the statistical evaluation model is constructed through the following steps: The first step is to collect historical production data, including speed deviation rate, direction deviation angle, speed standard deviation, and the corresponding actual causes of process anomalies, such as heating lamp aging, conveyor belt motor failure, and material batch changes. The second step is to discretize the data, dividing the continuous variable into several intervals; The third step is to train the data using the C4.5 decision tree algorithm, and generate a decision tree using the information gain ratio as the splitting criterion. The fourth step is to extract IF-THEN rules from the decision tree to form a rule mapping library; After training, the speed deviation rate, direction deviation angle, and speed standard deviation of the current batch are input into the model. The decision tree makes judgments layer by layer based on the feature values, eventually reaching a leaf node and outputting the corresponding process adjustment suggestions. Examples of rule mapping libraries include: If the speed deviation rate is greater than zero and the directional deviation angle is less than five degrees, the temperature gradient is considered too large, and it is recommended to reduce the heating lamp power. If the speed deviation rate is less than zero and the directional deviation angle is less than five degrees, the temperature gradient is considered too small, and it is recommended to increase the heating lamp power. If the absolute value of the speed deviation rate is normal and the directional deviation angle is greater than ten degrees, the temperature gradient direction is considered to be off, and it is recommended to check the heating lamp arrangement. If the speed standard deviation is greater than one time the theoretical characteristic speed, the conveyor belt speed is considered to be fluctuating, and it is recommended to repair the motor or encoder. If the speed deviation rate and directional deviation angle are normal, but the bubble area is generally smaller than the normal range, the viscosity of the encapsulation material is considered to be too high, and it is recommended to change the material batch. This allows the detection system to not only inform the process of abnormalities, but also locate the root cause of the abnormalities and provide actionable correction instructions.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the optical visual inspection method for diode surface defects and the inventive concept, should be covered within the scope of protection of the present invention.

Claims

1. An optical visual inspection method for diode surface defects, characterized in that, Includes the following steps: The preheated bright-field image sequence and the preheated dark-field image sequence were acquired. Based on the preheated bright field image sequence and the preheated dark field image sequence, morphological complementary matching is performed to obtain the bubble candidate sequence for each frame of the image. Based on the bubble candidate sequence, cross-frame correlation tracking is performed on the candidate regions belonging to the same physical bubble in the bubble candidate sequence to obtain the spatial position temporal sequence of each bubble, and the movement curve of each bubble is fitted. Based on the movement curve of each bubble, the instantaneous velocity and direction of motion of each bubble are calculated; an instantaneous velocity vector is formed based on the instantaneous velocity and direction of motion of each bubble; the temperature gradient direction of the preheating process is obtained, the three-dimensional temperature gradient direction is projected along the imaging optical axis onto the diode principal plane and mapped to the pixel coordinate system through the camera intrinsic parameters to obtain the two-dimensional temperature gradient direction vector, and the instantaneous velocity vector is projected onto the two-dimensional temperature gradient direction vector to obtain the projected velocity component; The equivalent viscosity coefficient and preheating time of the molding compound material are obtained, and the theoretical characteristic velocity is determined based on the average temperature gradient value, preheating time, bubble thermal expansion characteristic constant and equivalent viscosity coefficient. The dimensionless migration number of each candidate bubble is obtained by dividing the projected velocity component by the theoretical characteristic velocity. Based on the dimensionless migration number of all candidate bubbles in the current batch, kernel density estimation is performed to generate a migration number probability density distribution curve and determine the main peak interval; if the dimensionless migration number of a candidate bubble falls into the main peak interval, the candidate bubble is determined to satisfy the physical law of thermally induced migration and is regarded as a real bubble. If it does not fall within the main peak range, it is determined to be a false defect; Based on the actual bubble movement curve, the spatial position in subsequent processes is predicted, and bubble migration trajectory prediction data is obtained. Based on bubble migration trajectory prediction data, a risk warning for high-temperature welding process is output. Based on the instantaneous velocity and direction of motion of each real bubble, the optical visual inspection results of diode surface defects are obtained.

2. The optical visual inspection method for diode surface defects according to claim 1, characterized in that, Methods for obtaining the three-dimensional temperature gradient direction in the preheating process include: A miniature thermocouple array is arranged along the axial and radial directions of the diode to synchronously collect temperature data from each temperature measuring point during the stable operation phase of the preheating process. The collected spatial temperature field data is interpolated and fitted to construct a temperature distribution map of the area where the encapsulated body is located in the preheating temperature zone, and the direction of the three-dimensional temperature gradient is determined according to the normal direction of the isothermal surface in the temperature distribution map.

3. The optical visual inspection method for diode surface defects according to claim 2, characterized in that, Methods for calculating theoretical characteristic velocities include: The equivalent viscosity coefficient of the batch of molding compound material under the temperature conditions of the preheating process is obtained. The preheating time is determined according to the conveyor belt speed and the effective length of the heating zone in the preheating process. The average temperature gradient value inside the molding compound is determined by thermal simulation or temperature measurement calibration. The product of the average temperature gradient, preheating time, and the constant of the bubble's thermal expansion characteristic is used as the numerator, and the equivalent viscosity coefficient is used as the denominator. The resulting quotient is used as the theoretical characteristic velocity.

4. The optical visual inspection method for diode surface defects according to claim 1, characterized in that, The method for obtaining a bubble candidate sequence for each frame of image by performing morphological complementary matching based on the preheated bright-field image sequence and the preheated dark-field image sequence includes: Based on the bright field image sequence after preheating, each frame of the bright field image sequence is extracted to obtain multiple candidate dark spot regions. The spatial location, dark spot area, dark spot circularity, and dark spot grayscale contrast of each candidate dark spot region are recorded. Based on the dark field image sequence after preheating, the corresponding frame images in the dark field image sequence after preheating are extracted to obtain multiple candidate bright spot regions. The spatial location, bright spot area, bright spot circularity and bright spot grayscale contrast of each candidate bright spot region are recorded. Based on multiple candidate dark spot regions and multiple candidate bright spot regions, the candidate dark spot regions and candidate bright spot regions in the same frame are paired according to their spatial location to obtain paired dark spot regions and bright spot regions. Based on the paired dark spot regions and bright spot regions, the spatial distance, area ratio, roundness difference and gray-level contrast product between each pair of paired dark spot regions and bright spot regions are calculated. Determine whether the product of spatial distance, area ratio, roundness difference, and grayscale contrast satisfies the predefined complementary constraint conditions; If the spatial distance, area ratio, roundness difference, and gray-level contrast product all satisfy the predefined complementary constraint conditions, then the spatial position corresponding to the paired dark spot region and the bright spot region is determined as the bubble candidate region of the frame image. Based on the bubble candidate regions of each frame of the image, the bubble candidate regions of each frame of the image are summarized to obtain the bubble candidate sequence.

5. The optical visual inspection method for diode surface defects according to claim 4, characterized in that, Complementary constraints include: Spatial distance constraint: The spatial distance between the dark spot region and the bright spot region is less than the maximum allowable value of the optical misalignment calibration residual between the bright field imaging system and the dark field imaging system. The area ratio constraint ensures that the ratio of the dark spot area to the bright spot area is within the range determined by the theoretical ratio of the absorption cross-section of the bubble in bright field to the scattering cross-section in dark field. The roundness difference constraint is that the absolute value of the difference between the roundness of dark spots and the roundness of bright spots is less than the preset shape consistency tolerance based on the consistency of bubble shape in both bright and dark fields. The gray-scale contrast product constraint requires that the product of the gray-scale contrast of the dark spot and the gray-scale contrast of the bright spot be greater than the minimum detection threshold determined based on the complementary enhancement principle of signal-to-noise ratio in bright and dark field imaging.

6. The optical visual inspection method for diode surface defects according to claim 1, characterized in that, Methods for fitting the movement curve of each bubble based on the bubble candidate sequence include: Based on the bubble candidate sequence, cross-frame correlation tracking is performed on the candidate regions belonging to the same physical bubble in the bubble candidate sequence to obtain the spatial location time sequence of each bubble; Based on the spatial location time sequence of each bubble, least squares fitting is performed using a linear model and a parabolic model respectively to obtain linear fitting curves and parabolic fitting curves. Calculate the first residual sum of squares between the linear fitting curve and the spatial location time series, and the second residual sum of squares between the quadratic parabolic fitting curve and the spatial location time series, and compare the magnitudes of the first residual sum of squares and the second residual sum of squares. If the sum of squares of the first residual is less than the sum of squares of the second residual, then the linear fitting curve is selected as the moving curve of the bubble. If the second residual sum of squares is less than the first residual sum of squares, then the quadratic parabola fitted curve is selected as the moving curve of the bubble.

7. The optical visual inspection method for diode surface defects according to claim 6, characterized in that, Methods for calculating the instantaneous velocity and direction of motion of each bubble based on its movement curve include: Based on the movement curve of each bubble, determine the type of movement curve of each bubble. If the movement curve is a linear fitting curve, take the slope of the movement curve as the constant instantaneous velocity of the bubble throughout the entire process, and extract the spatial direction angle of the movement curve as the instantaneous direction of the bubble throughout the entire process. If the moving curve is a quadratic parabola fitting curve, then take the first derivative of the moving curve to obtain an expression for the linear change of instantaneous velocity with time. Based on this expression, the value of the expression at each acquisition time is taken as the instantaneous velocity of the corresponding frame. Based on this expression, the instantaneous direction is calculated frame by frame according to the tangent direction of the curve at each acquisition time.

8. The optical visual inspection method for diode surface defects according to claim 7, characterized in that, Methods for predicting bubble migration trajectory prediction data based on the actual bubble movement curve to predict its spatial position in subsequent processes include: Based on the movement curve of the real bubble, the type of the real bubble movement curve is obtained. If the movement curve is a linear fitting curve, the slope of the movement curve is used as the constant migration speed. Based on the constant migration speed, the spatial position of the bubble in the subsequent process is calculated using a linear extrapolation formula. If the moving curve is a quadratic parabola fitting curve, then the first derivative of the moving curve is taken to obtain the instantaneous velocity expression; based on the instantaneous velocity expression, the instantaneous velocity expression is integrated to obtain a quadratic function of displacement changing with time; based on the quadratic function of displacement changing with time, the spatial position of the bubble in the subsequent process is calculated by extrapolation of this quadratic function. The temperature gradient decay curve and viscosity decay function at the end of the preheating process are obtained. Based on the temperature gradient decay curve, viscosity decay function and the spatial position of bubbles in subsequent processes, the velocity parameters in the linear extrapolation formula or quadratic function are corrected to obtain bubble migration trajectory prediction data.

9. The optical visual inspection method for diode surface defects according to claim 1, characterized in that, Methods for generating risk warnings for high-temperature welding processes based on bubble migration trajectory prediction data include: Obtain a distribution map of the welding area in the high-temperature welding process, and divide the welding area into three risk level areas: core welding area, transition welding area, and edge welding area; Based on the bubble migration trajectory prediction data, the predicted spatial position of each real bubble at the start of the high-temperature welding process is extracted, and the minimum distance between the predicted spatial position and the core welding area is calculated. Based on the instantaneous velocity, bubble area, and minimum distance of each real bubble, the risk coefficient of each bubble is calculated according to a preset weighted formula; Based on the risk coefficient of each bubble, the risk coefficients of all real bubbles are summed to obtain a comprehensive risk index. The comprehensive risk index is then compared with preset low-risk, medium-risk, and high-risk thresholds. When the comprehensive risk index is less than the low risk threshold, a normal pass warning signal is output; when the comprehensive risk index is between the low risk threshold and the medium risk threshold, a maintenance reminder warning signal is output; when the comprehensive risk index is between the medium risk threshold and the high risk threshold, a shutdown and re-inspection warning signal is output; when the comprehensive risk index is greater than the high risk threshold, an emergency shutdown warning signal is output.

10. The optical visual inspection method for diode surface defects according to claim 1, characterized in that, The methods for evaluating the consistency of preheating process parameters and obtaining optical visual inspection results of diode surface defects based on the instantaneous velocity and direction of motion of each real bubble include: Based on the instantaneous velocity of each real bubble, the instantaneous velocities of all real bubbles are obtained, forming a velocity sample set; based on the motion direction of each real bubble, the motion directions of all real bubbles are obtained, forming a direction sample set. Based on the velocity sample set, the velocity mean and velocity standard deviation of the velocity sample set are calculated; based on the direction sample set, the direction mean and direction standard deviation of the direction sample set are calculated. Based on the average speed, the average speed is compared with the preset theoretical characteristic speed to calculate the speed deviation rate; based on the average direction, the average direction is compared with the temperature gradient direction of the preheating process to calculate the direction deviation angle. When the speed deviation rate is less than the preset first deviation threshold and the directional deviation angle is less than the preset second deviation threshold, the preheating process parameters are deemed to be consistent and qualified. When the speed deviation rate is greater than or equal to the first deviation threshold or the directional deviation angle is greater than or equal to the second deviation threshold, it is determined that there is an abnormality in the preheating process parameters, and corresponding process adjustment suggestions are output according to the magnitude of the speed deviation rate and the directional deviation angle.

Images