Method for on-line detection of surface defects of semiconductor device housings

Abstract

The application discloses a kind of surface defects of semiconductor equipment shell online detection method, belong to semiconductor equipment manufacturing detection technical field, solve the technical problem that existing detection method is difficult to detect multiple tiny defects simultaneously on metal high light surface and obtain defect three-dimensional geometric parameters.The gist of the method includes: when the shell moving on conveying line enters detection station, simultaneously trigger three kinds of light source modules of bright field, dark field and structured light, respectively collect reflection image, scattering image and phase modulation image;Three-dimensional topography data is demodulated from phase modulation image to generate height map;The color channel of reflection image is registered with height image pixel level, and multi-channel fusion feature map is constructed;The feature map is input into multi-branch convolutional neural network, and the output defect category and geometric parameter.The application effectively solves the problem of phase information loss caused by metal high light through multi-exposure fusion or multi-modal interpolation completion strategy, ensures the integrity of three-dimensional topography reconstruction.

Description

Technical Field

[0001] This invention relates to the field of semiconductor equipment manufacturing inspection technology. More specifically, this invention relates to an online method for detecting surface defects in the casing of semiconductor equipment. Background Technology

[0002] Semiconductor device casings are typically made of metals such as stainless steel or aluminum alloys, which have high reflectivity. In optical inspection, specular reflection from metal surfaces can easily cause localized overexposure or shadowing in camera images, making it difficult to clearly capture minute defects. Existing inspection methods mostly use a single bright-field or dark-field illumination method. However, for various defect types such as scratches, dents, particles, and stains, a single illumination method cannot provide sufficient contrast for all types of defects, often requiring multiple switching of illumination conditions, leading to a complex inspection process and reduced efficiency.

[0003] Two-dimensional image detection can only provide planar location and appearance information of defects, but cannot obtain depth or height data. For defects such as pits and scratches, their severity depends mainly on depth rather than planar dimensions, making it difficult to quantitatively classify defects based solely on two-dimensional images. To obtain three-dimensional information, some methods introduce structured light measurement. However, when applying structured light projection to highly reflective metallic surfaces, local specular reflection can easily lead to oversaturation of the fringe image, causing permanent loss of phase information and resulting in holes or errors in the 3D reconstruction. Existing methods for addressing saturation (such as multi-exposure fusion or adaptive projection intensity modulation) either increase detection time or raise system complexity and calibration difficulty.

[0004] Furthermore, the surface textures of the outer shell (such as grinding marks and milling marks) may appear as linear grayscale features similar to scratches in the image, which can easily lead to false detections. Traditional image filtering or morphological processing may blur the true defects while removing textures, while deep learning-based methods face problems such as insufficient model generalization ability and the need for a large amount of labeled data.

[0005] The aforementioned difficulties make it challenging for existing technologies to simultaneously achieve high-precision identification of multiple minute defects and acquisition of three-dimensional geometric parameters within a single online inspection process. Summary of the Invention

[0006] One object of the present invention is to solve at least the above-mentioned problems and to provide at least the advantages that will be described later.

[0007] Another objective of this invention is to provide an online detection method for surface defects in semiconductor device housings, which solves the technical problem that existing detection methods are unable to simultaneously detect multiple minute defects on highly reflective metal surfaces and obtain the three-dimensional geometric parameters of the defects.

[0008] To achieve these objectives and other advantages according to the present invention, an online method for detecting surface defects in a semiconductor device housing is provided, comprising the following steps: S1. Synchronous acquisition of multimodal optical signals: When the semiconductor equipment housing moving on the conveyor line enters the inspection station, the first light source module, the second light source module, and the third light source module are simultaneously triggered; wherein, the first light source module illuminates the housing surface in a coaxial bright field illumination mode and acquires a first reflection image; the second light source module illuminates the housing surface in a ring dark field illumination mode and acquires a second scattering image; the third light source module projects structured light stripes onto the housing surface and acquires a phase modulation image; S2. Three-dimensional topography reconstruction: Demodulate the three-dimensional topography data of the shell surface from the phase modulation image and generate a height map; S3. Five-channel feature fusion: The first reflection image is a color image. Its three color channels, R, G, and B, are registered pixel-level with the height map and the second scattering image to construct a five-channel fusion feature map. The five channels are, in order, the R channel, the G channel, the B channel, the height map channel, and the grayscale channel of the second scattering image. S4. Defect Detection: The five-channel fused feature map is input into a multi-branch convolutional neural network; the multi-branch convolutional neural network includes at least a first branch and a second branch, the first branch is used to extract the morphological features of the defect, and the second branch is used to extract the texture features of the defect; the outputs of the first branch and the second branch are fused, and then the defect category and the geometric parameters of the defect are output. S5. Grading determination: The semiconductor device casing is graded according to the geometric parameters of the defect and the location information of the defect.

[0009] Preferably, in step S4, the first branch uses deformable convolution, the second branch uses a non-local attention module, and the multi-branch convolutional neural network further includes a third branch, which uses a coordinate attention mechanism to extract the correlation features between defects and the processing texture direction.

[0010] Preferably, the defect categories output in step S4 include scratches, dents, particles, and stains; the defect geometric parameters output in step S4 include the maximum depth and maximum length of the defect.

[0011] Preferably, the grading determination in step S5 specifically involves: If the defect category is particles, then mark them as repairable or scrapped based on the particle diameter and whether they are located in a preset critical area; If the defect category is stain, it will be marked as repairable or scrapped based on the stain area and whether it is located in a preset critical area; If the defect category is scratch or dent, it is classified according to its maximum depth, maximum length, and whether it is located in a preset critical area.

[0012] Preferably, the preset key area includes the sealing surface and the electrode interface area; wherein, the threshold for graded determination is set as follows: When the defect category is scratch or dent, if the maximum depth is ≤5μm, the maximum length is ≤1mm and it is not located in any critical area, it is marked as repair-free; if the maximum depth is ≤20μm and the maximum length is ≤5mm and it is not located in any critical area, it is marked as repairable; if the maximum depth is >20μm or the maximum length is >5mm or it is located in any critical area, it is marked as scrap. When the defect category is particles, if the particle diameter is ≤30μm and it is not located on the sealing surface or electrode interface area, it is marked as repairable; if the particle diameter is >30μm or it is located in any critical area, it is marked as scrap. When the defect category is stain, if the stain area is ≤0.5mm 2 If the stain is not located on the sealing surface or electrode interface area, it is marked as repairable; if the stain area is >0.5mm... 2 If it is located in any critical area, it is marked as scrap.

[0013] Preferably, a multi-angle dark field analysis step is included after step S1 and before step S2, which includes: Step S1a, Acquisition of partitioned dark field images: Set the second light source module as an independently controllable multi-partition ring dark field light source, and divide it into N independently controllable partitions, N≥4; light up each partition in sequence, and acquire the corresponding dark field image through the second camera when each partition is lit, and acquire a total of N partitioned dark field images; Step S1b, Anisotropy Coefficient Calculation: For each pixel position (x,y), based on its gray value sequence I1(x,y), I2(x,y), …, I in the N partitioned dark field images... n Given (x,y), calculate the anisotropy coefficient A(x,y) of this pixel: A(x,y) = (max(I i ) - min(I i )) / (max(I i ) + min(I i ) + ε); Where ε is a preset small positive number to avoid division by zero, max(I i ) and min(I i These represent the maximum and minimum grayscale values ​​of the pixel under different zone lighting conditions; Step S1c, Texture-Defect Differentiation: The anisotropy coefficient A(x,y) is compared with a preset threshold A. th Comparison: If A(x,y) ≥ A th If A(x,y) is a directional texture, then the pixel is determined to belong to the directional texture. th If so, the pixel is determined to be a genuine defect; Step S1d, False Defect Filtering: A texture mask is generated based on the determination result of step S1c. When constructing the five-channel fusion feature map in step S3, the feature values ​​corresponding to the processed texture pixels are set to invalid according to the texture mask, so that the multi-branch convolutional neural network in step S4 only detects and classifies the real defect areas.

[0014] Preferably, it also includes a hierarchical feedback-driven continuous learning optimization step, which is performed after step S5: Step S6a, Confidence-Aware Pseudo-Label Generation: For products marked as repair-free in step S5, extract their five-channel fusion feature map, and construct a normal region mask based on the defect detection results output in step S4; use the feature regions covered by the normal region mask as positive sample pseudo-labels, use the regions marked as scrap as hard negative sample pseudo-labels, and calculate the confidence weight w=exp(−D) for the regions marked as repairable based on their defect geometric parameters. max / δ), where δ is a preset scale parameter, used as a pseudo-label for soft negative samples; Step S6b, Multi-level Optimization Pool Management: Construct a three-level optimization pool, including a short-term sliding window pool, a medium-term prototype pool, and a long-term cache pool; add the positive sample pseudo-labels, hard negative sample pseudo-labels, and soft negative sample pseudo-labels generated in step S6a to the three-level optimization pool; adopt a class-incremental learning strategy, and when the number of samples of the newly added defect category exceeds a preset threshold, automatically create a new category prototype and assign a new category ID; Step S6c, Incremental Update of Model Parameters: An incremental update of the model parameters is triggered every preset time interval or when the number of samples accumulated in the short-term sliding window pool described in step S6b reaches a preset batch size; an elastic weight-reinforced regularized loss function is used. ; Where L is the total loss, L t For the detection loss in the current batch, F i For parameter θ i The diagonal elements of the Fisher information matrix, θ i,o These are the parameters of the old model, and λ is the tradeoff coefficient. Step S6d, Adaptive adjustment of grading threshold: Based on the detection results of the validation batch data by the model updated in step S6c, the threshold parameters for grading determination are adjusted using the Bayesian optimization method. ​Step S6e, Closed-loop optimization of light source parameters: The white light intensity of the first light source module, the ultraviolet light intensity of the second light source module, and the near-infrared structured light intensity of the third light source module in step S1 are used as adjustable parameters. The confidence level of defect detection output in step S4 is used as the optimization target. Bayesian optimization or reinforcement learning methods are used to coordinately adjust the output intensity of the three light source modules.

[0015] The present invention has at least the following beneficial effects: First, by simultaneously triggering coaxial bright-field illumination, annular dark-field illumination, and structured light fringe projection, reflected images, scattered images, and phase-modulated images are acquired synchronously at the same inspection station. This eliminates the need for multiple switching of illumination conditions or time-division scanning, shortening the inspection cycle for a single shell and improving online inspection efficiency. Furthermore, addressing the saturation problem of structured light in highly reflective metal regions, this invention integrates a multi-exposure adaptive acquisition or neighborhood height interpolation mechanism without adding extra inspection cycles.

[0016] Second, three-dimensional topography data is demodulated from the phase-modulated image to generate a height map. Then, the color channels of the reflection image are pixel-level registered with the height map to construct a five-channel fusion feature map. This allows the detection network to simultaneously obtain surface color, texture, and depth information, enabling joint identification of various defects such as scratches, dents, particles, and stains. It also outputs the maximum depth and length of the defects, providing a quantitative basis for subsequent grading. To address the potential loss of phase information due to high metallic reflectivity, this invention further employs a multi-exposure fusion or neighborhood height interpolation completion strategy to ensure the integrity of the three-dimensional topography, thereby obtaining reliable depth information even on highly reflective surfaces.

[0017] Third, the morphological feature branch and texture feature branch in the multi-branch convolutional neural network are used to extract features of linear, point-like, and regionally distributed defects, respectively, improving the detection accuracy for different types of defects. Based on this, a coordinate attention mechanism in the third branch is introduced, which helps to distinguish the directional correlation between defects and processed textures, reducing the false detection rate.

[0018] Fourth, a multi-angle dark field analysis step is added after step S1. By sequentially lighting up the ring dark field light source of multiple partitions, the degree of change of the gray value of each pixel with the illumination direction, i.e., the anisotropy coefficient, can be calculated. This can effectively distinguish between directional processed textures and isotropic real defects, and the texture pixels are shielded from the five-channel fused feature map, further reducing misjudgments caused by texture interference.

[0019] Fifth, a hierarchical feedback-driven continuous learning optimization step is added. Pseudo-labels are automatically generated using the hierarchical results such as repairable, scrap-proof, etc. generated during the inspection process. Samples are managed through a three-level optimization pool, model parameters are updated by elastic weights, hierarchical thresholds are adjusted by Bayesian optimization, and light source parameters are optimized in a closed loop. This enables the inspection system to adapt to production line drift, product iteration, and the emergence of new defects, and maintain stable inspection performance in long-term operation, reducing the need for manual intervention and downtime retraining.

[0020] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to embodiments, so that those skilled in the art can implement it based on the description.

[0022] This invention discloses an online detection method for surface defects in semiconductor device housings, comprising the following steps: S1. Synchronous acquisition of multimodal optical signals: When the semiconductor equipment housing moving on the conveyor line enters the inspection station, the first light source module, the second light source module, and the third light source module are simultaneously triggered; wherein, the first light source module illuminates the housing surface in a coaxial bright field illumination mode and acquires a first reflection image; the second light source module illuminates the housing surface in a ring dark field illumination mode and acquires a second scattering image; the third light source module projects structured light stripes onto the housing surface and acquires a phase modulation image; S2. Three-dimensional topography reconstruction: Demodulate the three-dimensional topography data of the shell surface from the phase modulation image and generate a height map; S3. Five-channel feature fusion: The first reflection image is a color image. Its three color channels, R, G, and B, are registered pixel-level with the height map and the second scattering image to construct a five-channel fusion feature map. The five channels are, in order, the R channel, the G channel, the B channel, the height map channel, and the grayscale channel of the second scattering image. S4. Defect Detection: The five-channel fused feature map is input into a multi-branch convolutional neural network; the multi-branch convolutional neural network includes at least a first branch and a second branch, the first branch is used to extract the morphological features of the defect, and the second branch is used to extract the texture features of the defect; the outputs of the first branch and the second branch are fused, and then the defect category and the geometric parameters of the defect are output. S5. Grading determination: Based on the geometric parameters of the defect and the location information of the defect, the semiconductor device casing is graded in terms of quality; In step S4, the first branch uses deformable convolution, the second branch uses a non-local attention module; the multi-branch convolutional neural network also includes a third branch, which uses a coordinate attention mechanism to extract the correlation features between defects and processing texture directions; The defect categories output in step S4 include scratches, pits, particles, and stains; the defect geometric parameters output in step S4 include the maximum depth and maximum length of the defect. The grading determination in step S5 is as follows: if the defect type is particles, they are marked as repairable or scrapped based on the particle diameter and whether they are located in a preset critical area; if the defect type is stains, they are marked as repairable or scrapped based on the stain area and whether they are located in a preset critical area; the preset critical area includes the sealing surface and the electrode interface area; if the defect type is scratches or pits, they are graded based on their maximum depth, maximum length and whether they are located in a preset critical area.

[0023] The preset key areas include the sealing surface and the electrode interface area; the threshold for graded determination is set as follows: When the defect category is scratch or dent, if the maximum depth is ≤5μm, the maximum length is ≤1mm and it is not located in any critical area, it is marked as repair-free; if the maximum depth is ≤20μm and the maximum length is ≤5mm and it is not located in any critical area, it is marked as repairable; if the maximum depth is >20μm or the maximum length is >5mm or it is located in any critical area, it is marked as scrap. When the defect category is particles, if the particle diameter is ≤30μm and it is not located on the sealing surface or electrode interface area, it is marked as repairable; if the particle diameter is >30μm or it is located in any critical area, it is marked as scrap. When the defect category is stain, if the stain area is ≤0.5mm 2 If the stain is not located on the sealing surface or electrode interface area, it is marked as repairable; if the stain area is >0.5mm... 2 If it is located in any critical area, it is marked as scrap.

[0024] In the above technical solution, an encoder or photoelectric sensor is installed on the conveyor line. When the semiconductor equipment housing enters the inspection station, a synchronization signal is triggered. The encoder can be installed on the drive roller of the conveyor line, and the photoelectric sensor can be installed at the entrance of the inspection station. When the housing blocks the light path, a trigger pulse is generated. The housing moves at a preset speed, which can be set from 50mm / s to 200mm / s according to the production line cycle. The first light source module uses a white LED coaxial light source with an illumination angle of 0° to 15°. The coaxial light source is arranged in front of the camera lens through a beam splitter, so that the light shines perpendicularly onto the housing surface, and the first reflected image is a color image. The second light source module uses an ultraviolet LED ring dark field light source with an illumination angle of 45° to 75°. The ring light source is installed around the camera lens, and the ultraviolet wavelength can be selected as 365nm or 395nm. The second camera collects a grayscale second scattering image. The third light source module uses a digital projector to project near-infrared sinusoidal grating stripes onto the housing surface. The near-infrared wavelength can be selected as 850nm or 940nm, and the third camera collects a phase-modulated image. All three cameras are industrial-grade area-array CMOS cameras, each equipped with a bandpass filter corresponding to its wavelength: the camera for white light coaxial light source can be fitted with a 400nm to 700nm broadband filter, the camera for ultraviolet light source can be fitted with a filter with a center wavelength of 365nm and a bandwidth of 20nm, and the camera for near-infrared structured light can be fitted with a filter with a center wavelength of 850nm and a bandwidth of 30nm. The three cameras are synchronized via a hardware trigger line, acquiring images simultaneously.

[0025] When demodulating the 3D topography from the phase-modulated image, Fourier transform profilometry is used. The specific steps are as follows: Perform a 2D Fourier transform on the stripe image; extract the fundamental frequency component in the frequency domain using a bandpass filter (such as a Hanning window). The center frequency of the bandpass filter corresponds to the fundamental frequency, and the bandwidth is set to 0.5-1.0 times the fundamental frequency to filter out zero-frequency and high-frequency noise. Then, perform an inverse Fourier transform to obtain the wrapping phase. The wrapping phase is then used to obtain a continuous phase through a branching method or a phase expansion algorithm based on the quality map. The phase-height mapping calibration uses the classic multi-frequency heterodyne method or polynomial fitting method, establishing the mapping relationship between phase difference and height by measuring standard blocks with known heights. Combined with pre-calibrated camera and projector geometric parameters, the relative height of each point on the shell surface is calculated to generate a single-channel height map with a height resolution at the micrometer level. The camera and projector calibration can use the Zhang Zhengyou calibration method: capture checkerboard images in multiple poses, establish the homography matrix between the camera pixel coordinates and the projector projection coordinates, and then obtain the system geometric parameters.

[0026] To address the issue of phase information loss due to localized saturation on highly reflective metallic surfaces, this invention further employs the following strategy: First, saturated pixel detection is performed on the acquired phase modulation image (pixels with a grayscale value greater than 250 are marked as saturated). For isolated or small saturated regions (less than 20 consecutive saturated pixels), the missing height data is estimated using radial basis function interpolation or bicubic interpolation, utilizing the height values ​​of unsaturated pixels within a 3×3 window surrounding the same pixel location. For large saturated regions (more than 20 consecutive saturated pixels), a multi-level exposure mode of the third light source module is automatically triggered: three stripe images are continuously acquired at 50%, 100%, and 150% of the standard exposure time, respectively. After independently demodulating each image, the unsaturated phase value with the highest modulation degree is selected by pixel and fused to generate the final height map. The total time for the above interpolation and multi-exposure processing does not exceed 100ms, and is completed within 100ms without affecting the online detection cycle.

[0027] The first reflected image is a color image. Its R, G, and B color channels are registered pixel-level with the height map and the second scattering image. The registration is based on a checkerboard calibration board that predetermines the homography matrix between cameras. The channels are spatially aligned through image remapping to construct a five-channel fused feature map, where R, G, and B are color channels, H is the height map channel, and D is the grayscale channel of the second scattering image.

[0028] The five-channel fused feature map is input into a pre-trained multi-branch convolutional neural network. The network design is as follows: three branches share a backbone feature extraction network, which can use the first four convolutional blocks of ResNet-18, with the output feature map size being 1 / 16 of the input image. The first branch connects three deformable convolutional layers after the backbone feature map, each with a kernel size of 3×3 and 64 output channels, used to extract the morphological features of scratches and dents. The second branch connects two non-local attention modules, with 64 output channels, used to extract the texture features of particles and stains. The third branch connects two coordinate attention layers, with 64 output channels, used to extract the correlation features between defects and the processing texture direction. The outputs of the three branches are concatenated along the channel dimension to obtain a 192-channel feature map, which is then passed through two fully connected layers (the first layer outputs 128 dimensions, and the second layer outputs 6 dimensions) to output the defect category and geometric parameters, respectively. The defect categories include scratches, dents, particles, and stains, and a softmax output probability distribution is used; the geometric parameters include the maximum depth and maximum length of the defect, and a linear regression output continuous value is used. The network is trained using a pre-labeled shell defect dataset, which contains over 1000 shell samples. Quality inspectors use image annotation software to delineate the defects pixel by pixel, and a high-precision profilometer measures the maximum depth and length of the defects as ground truth. The loss function consists of a classification cross-entropy loss and a regression mean squared error loss, which are weighted and summed.

[0029] The location coordinates of the defect are used to determine whether it is located in a pre-defined critical area, which includes the sealing surface and electrode interface area. The specific rules for the graded determination are as follows: If the defect category is particles, then the defect is judged based on the particle diameter: if the diameter is ≤30μm and not located in the sealing surface or electrode interface area, it is marked as repairable; if the diameter is >30μm or located in any critical area, it is marked as scrap. If the defect category is stain, then the determination is based on the stain area: area ≤ 0.5mm 2 Those not located on the sealing surface or electrode interface area are marked as repairable; area > 0.5 mm². 2 If it is located in any critical area, mark it as scrap; If the defect category is scratch or dent, its maximum depth and maximum length are compared with preset thresholds. When the maximum depth is ≤5μm, the maximum length is ≤1mm and it is not located in any critical area, it is marked as repair-free; when 5μm < maximum depth ≤20μm, the maximum length is ≤5mm and it is not located in any critical area, it is marked as repairable; if the maximum depth is >20μm or the maximum length is >5mm or it is located in any critical area, it is marked as scrap.

[0030] The entire testing process is controlled by an industrial computer or embedded processor, and the test results are output to the production line management system via Ethernet for subsequent automatic sorting or manual re-inspection.

[0031] The beneficial effects of this technical solution include the following aspects. First, by simultaneously triggering coaxial bright-field illumination, annular dark-field illumination, and structured light stripe projection, color reflection images, dark-field scattering images, and phase-modulated images are acquired synchronously at the same inspection station. This avoids the cycle time extension caused by multiple illumination switching or time-division scanning in traditional methods. Surface color, texture, scattering characteristics, and 3D height information can be obtained in a single inspection, improving the efficiency of online inspection. Second, the RGB color channels and height map of the reflection image and the dark-field scattering image are pixel-level registered and a five-channel fusion feature map is constructed. This enables the subsequent deep learning network to simultaneously use color to distinguish stains, use depth to quantify pits and scratches, and use dark-field scattering to enhance the contrast of fine scratches. This achieves joint detection of four types of defects: scratches, pits, particles, and stains, and can output the maximum depth and maximum length of defects, solving the problem that two-dimensional images cannot quantitatively assess the severity of defects. Third, in the multi-branch convolutional neural network, the deformable convolutional branch can adaptively extract the irregular morphological features of scratches and pits, the non-local attention branch can capture the global texture distribution of particles and stains, and the coordinate attention branch uses the directionality of the processing texture to suppress false defects. The synergistic effect of the three branches significantly reduces the false detection rate of processing textures. Fourth, the hierarchical judgment rule distinguishes between particles / stains and scratches / pits, and uses critical areas (sealing surface, electrode interface) as a veto condition, avoiding the incorrect release of small particles in non-critical areas, while ensuring that any deep defects located on the sealing surface are scrapped, meeting the stringent industrial requirements for cleanliness and airtightness of semiconductor equipment housings. Fifth, the entire detection process does not rely on manual feature engineering, can adapt to the processing differences of different batches of housings, and all thresholds (5μm, 20μm, 1mm, 5mm) are set based on common standards in the semiconductor industry, making it practically operable. Through the above methods, this solution achieves high-precision identification of various minute defects and acquisition of three-dimensional geometric parameters in a single online inspection process. It can be deployed on semiconductor equipment housing production lines to replace manual visual inspection or multi-station combined inspection.

[0032] In another technical solution, after acquiring the conventional full-ring dark field image (i.e., the second scattering image) in step S1 and before step S2, a multi-angle dark field analysis step is also included, which includes: Step S1a, Partitioned Dark Field Image Acquisition: After completing the conventional full-ring dark field image acquisition in Step S1, the second light source module is set as an independently controllable multi-partitioned ring dark field light source, divided into N independently controllable partitions, N≥4; the conveyor line speed is reduced to below 20mm / s or briefly stopped, and then each partition is lit sequentially. While each partition is lit, the corresponding partitioned dark field image is acquired through the second camera, for a total of N partitioned dark field images. The conventional full-ring dark field image is used to construct the D channel in the subsequent five-channel fusion feature map, while the partitioned dark field images are only used to calculate the anisotropy coefficients to generate the texture mask and do not participate in the five-channel fusion feature map. Step S1b, Anisotropy Coefficient Calculation: For each pixel position (x,y), based on its gray value sequence I1(x,y), I2(x,y), …, I in the N partitioned dark field images... n Given (x,y), calculate the anisotropy coefficient A(x,y) of this pixel: A(x,y) = (max(I i ) - min(I i )) / (max(I i ) + min(I i ) + ε); Where ε is a preset small positive number to avoid division by zero, max(I i ) and min(I i These represent the maximum and minimum grayscale values ​​of the pixel under different zone lighting conditions; Step S1c, Texture-Defect Differentiation: The anisotropy coefficient A(x,y) is compared with a preset threshold A. th Comparison: If A(x,y) ≥ A th If A(x,y) is a directional texture, then the pixel is determined to belong to the directional texture. th If so, the pixel is determined to be a genuine defect; Step S1d, False Defect Filtering: Based on the determination result of step S1c, a texture mask is generated. When constructing the five-channel fusion feature map in step S3 (the D channel in this feature map comes from the conventional full-ring dark field image acquired in step S1), the feature values ​​corresponding to the processed texture pixels are set to invalid according to the texture mask, so that the multi-branch convolutional neural network in step S4 only detects and classifies the real defect areas.

[0033] ​In the above technical solution, a multi-angle dark field analysis step is added after step S1 and before step S2. In step S1, three-modal synchronous acquisition has been completed, where all partitions of the second light source module are simultaneously illuminated to obtain a full-ring dark field image. This image is used for the D channel in the subsequent five-channel fusion feature map. Based on this, the second light source module uses an independently controllable multi-partition ring dark field light source. This ring light source can be divided into 4, 6, or 8 independently controllable partitions, each partition consisting of several ultraviolet LEDs with wavelengths of 365nm or 395nm. The ring light source is installed around the camera lens, with each partition evenly distributed along the circumference. During partition dark field image acquisition, each partition is illuminated sequentially, with only one partition illuminated at a time, while the other partitions remain off. When each partition is illuminated, the corresponding dark field image is acquired through the second camera, resulting in a total of N partition dark field images, where N can be 4, 6, or 8. During the acquisition process, the conveyor line can be briefly stopped or kept moving at a low speed (e.g., the speed is reduced to below 20mm / s). The exposure time for each image can be set from 5ms to 20ms to ensure image clarity and avoid motion blur.

[0034] For each pixel location, an anisotropy coefficient is calculated based on its grayscale value sequence in the N partitioned dark-field images. The specific formula is: A(x,y) = (max(I i ) - min(I i )) / (max(I i ) + min(I i ) + ε), where I i Let max(I) be the grayscale value of the pixel in the i-th dark field image. i ) and min(I i The values ​​are the maximum and minimum grayscale values ​​of the pixel under different lighting zones, respectively. ε is a small positive number, which can be 0.001 or 0.0001, to avoid a zero denominator. The anisotropy coefficient ranges from 0 to 1. Preset threshold A. th It can be set to a value between 0.3 and 0.5, for example, 0.4. Compare the calculated A(x,y) with A_th: if A(x,y) ≥ A... th If A(x,y) is a directional machining texture (such as grinding or milling texture), then the pixel is determined to belong to a directional machining texture. th If the pixel is found to be a genuine defect (scratches, dents, grains, or stains), then it is determined that the pixel belongs to a real defect.

[0035] ​Based on the above determination results, a texture mask is generated. This mask is a binary image, where the position corresponding to the texture pixel is marked as 1, and the real defect pixel is marked as 0. In the subsequent step S3, when constructing the five-channel fusion feature map, the feature values ​​corresponding to the processed texture pixels are set to invalid based on this texture mask. Specifically, the mask is multiplied pixel by pixel with the five-channel feature map, so that all feature values ​​of the texture region become 0. In this way, the multi-branch convolutional neural network in step S4 only detects and classifies the real defect region, and the processed texture is excluded from the detection range. The entire multi-angle dark field analysis process is controlled by an industrial control computer, and the lighting order and duration of each partition can be pre-programmed.

[0036] The beneficial effects of this technical solution include the following: First, by sequentially illuminating multiple zones of the annular dark-field light source and acquiring dark-field images under different illumination directions, the anisotropic characteristics of scattered light can be used to quantitatively distinguish between directional processed textures and real defects of isotropic or weakly anisotropic nature. Second, the calculation of the anisotropy coefficient A(x,y) involves only simple range and summation operations, with low computational load, and can be completed in real time during online detection without the need for additional deep learning models or complex image processing. Third, the generated texture mask directly masks texture pixels during the feature fusion stage, avoiding misjudgment of texture regions by subsequent neural networks and reducing the false detection rate, especially suitable for metal shell surfaces with obvious directional processed textures. Fourth, this solution only requires modifying the annular dark-field light source to be independently controlled by each zone, with minimal hardware changes, low cost, and full compatibility with the existing three-modal synchronous acquisition system.

[0037] In another technical solution, a hierarchical feedback-driven continuous learning optimization step is also included, which is performed after step S5: Step S6a, Confidence-Aware Pseudo-Label Generation: For products marked as repair-free in step S5, extract their five-channel fusion feature map, and construct a normal region mask based on the defect detection results output in step S4; use the feature regions covered by the normal region mask as positive sample pseudo-labels, use the regions marked as scrap as hard negative sample pseudo-labels, and calculate the confidence weight w=exp(−D) for the regions marked as repairable based on their defect geometric parameters. max / δ), where δ is a preset scale parameter, used as a pseudo-label for soft negative samples; Step S6b, Multi-level Optimization Pool Management: Construct a three-level optimization pool, including a short-term sliding window pool, a medium-term prototype pool, and a long-term cache pool; add the positive sample pseudo-labels, hard negative sample pseudo-labels, and soft negative sample pseudo-labels generated in step S6a to the three-level optimization pool; adopt a class-incremental learning strategy, and when the number of samples of the newly added defect category exceeds a preset threshold, automatically create a new category prototype and assign a new category ID; Step S6c, Incremental Update of Model Parameters: An incremental update of the model parameters is triggered every preset time interval or when the number of samples accumulated in the short-term sliding window pool described in step S6b reaches a preset batch size; an elastic weight-reinforced regularized loss function is used. ; Where L is the total loss, L t For the detection loss in the current batch, F i For parameter θ i The diagonal elements of the Fisher information matrix, θ i,o The parameters are those of the old model, and λ is the tradeoff coefficient; the diagonal elements of the Fisher information matrix are F i The specific calculation method is as follows: After the initial model training is completed, the basic Fisher information is calculated using the entire initial training dataset: , of which F i,b The diagonal elements of the basic Fisher information matrix are calculated based on the initial training dataset, where N is the total number of initial training samples. During each incremental update, the online Fisher information is calculated using the current batch of samples within the short-term sliding window pool from step S6b. , of which F i,o Let M be the diagonal elements of the online Fisher information matrix calculated based on the current batch of samples, where M is the batch size. The final F... i Take the maximum of the two, i.e., F i =max(F i,b ,F i,o This process balances the retention of old knowledge with the adaptation to new data. The calculation is performed automatically before each incremental update, and the computational cost is negligible. All parameters in the above formulas are dimensionless.

[0038] Step S6d, Adaptive Adjustment of Grading Thresholds: Based on the detection results of the validation batch data using the model updated in step S6c, the grading threshold parameters are adjusted using a Bayesian optimization method. The search space for the grading threshold parameters is set as follows: depth threshold between 1 μm and 50 μm, length threshold between 0.1 mm and 10 mm, particle diameter threshold between 10 μm and 100 μm, and stain area threshold between 0.1 mm and 100 μm. 2 Up to 2mm 2 between.

[0039] The optimization objective is to maximize the defect recall rate under a preset false positive rate constraint (e.g., false positive rate ≤ 0.5%). Bayesian optimization uses a Gaussian process as the surrogate model, the expectation improvement (EI) function, and 30 iterations. In each iteration, validation batch data is used to evaluate the detection performance under the current threshold combination. The optimized thresholds are automatically updated and used for subsequent product classification.

[0040] Step S6e, Closed-Loop Optimization of Light Source Parameters: The white light intensity of the first light source module, the ultraviolet intensity of the second light source module, and the near-infrared structured light intensity of the third light source module in step S1 are used as adjustable parameters, with the confidence level of defect detection output in step S4 as the optimization target. The confidence level is defined as the maximum softmax probability value output by the classification branch of the multi-branch convolutional neural network, ranging from [0,1]. The closer the value is to 1, the more reliable the detection result. Bayesian optimization or reinforcement learning methods are used to collaboratively adjust the output intensity of the three light source modules to maximize the confidence level.

[0041] In the above technical solution, after the hierarchical judgment is completed in step S5, a hierarchical feedback-driven continuous learning optimization step is executed. For products marked as repair-free in step S5, their five-channel fusion feature maps are extracted, and a normal region mask is constructed based on the defect detection results output in step S4. The method for constructing the normal region mask is as follows: all pixel regions in the detection results that are not judged as defects are taken as candidates, and then regions within a preset width (e.g., 5 to 10 pixels) from the boundary of each detection box are removed to eliminate potential annotation noise. The remaining connected regions are the normal region mask. The feature regions covered by the normal region mask are used as positive sample pseudo-labels, that is, the feature vectors of these regions and their corresponding category labels are "background" or "no defect". Regions marked as scrap are used as hard negative sample pseudo-labels, and their labels are the corresponding defect categories (e.g., scratches, dents, etc.). The confidence weight of regions marked as repairable is calculated according to their defect geometric parameters. The weight formula is w=exp(−D max / δ), where D max The maximum depth of the defect is represented by δ, which is a scale parameter that can be set to 10 μm or 15 μm. This weight value ranges from 0 to 1, with smaller defects receiving higher weights, and is used as a soft negative sample pseudo-label. Positive samples, hard negative samples, and soft negative sample pseudo-labels are all stored in the optimization pool for later use.

[0042] A three-tiered optimization pool is constructed. The short-term sliding window pool stores the feature-pseudo-label pairs of the most recent M products, where M can be set to 500 or 1000. When the window is full, older samples are discarded according to a first-in, first-out (FIFO) principle. The medium-term prototype pool stores historical distributions using the category mean as the prototype. Specifically, for each defect category (including scratches, dents, particles, stains, and background), the mean of its feature vector is calculated as the prototype, which is updated periodically (e.g., every 200 samples). The long-term cache pool stores representative samples that trigger model version updates. The selection strategy for representative samples can employ a core set selection algorithm (e.g., Herding algorithm) or the k-nearest neighbor method: calculating the distance between each sample and its nearest neighbor, selecting samples with a distance greater than a preset threshold, or selecting samples that cover the feature space boundary. A class-incremental learning strategy is adopted. When the number of samples in a new defect category exceeds a preset threshold (e.g., 30), a new category prototype is automatically created and a new category ID is assigned, while simultaneously updating the classifier's output dimension. New defect categories can be triggered through manual verification or automatically discovered based on clustering algorithms.

[0043] The incremental update of model parameters is triggered at preset time intervals (e.g., every 30 minutes) or when the number of samples accumulated in the short sliding window pool reaches a preset batch size (e.g., 32 or 64). Elastic weights are used to consolidate the regularized loss function. ; Among them, L t F represents the detection loss for the current batch (including classification cross-entropy and regression mean squared error). i For parameter θ i The diagonal elements of the Fisher information matrix, θ i,o Here are the parameters for the old model, and λ is the tradeoff coefficient, which can be set to a value between 0.1 and 1. The Fisher information matrix can be calculated and saved using the entire training dataset after the initial training of the old model; it can also be approximated using samples from the optimization pool during subsequent incremental updates. During incremental updates, several rounds of fine-tuning are performed using samples from the optimization pool, and the learning rate can be set to 1 / 10 of the initial training learning rate.

[0044] When adaptively adjusting the grading threshold, the grading threshold parameters (5μm, 20μm, 1mm, 5mm) are adjusted using a Bayesian optimization method based on the detection results of the validation batch data using the updated model. The validation batch data can be randomly drawn from a long-term cache pool or use historical product data that has been manually verified. The objective function of the Bayesian optimization is to maximize the defect recall rate under a preset false positive rate constraint (e.g., a false positive rate not exceeding 0.5%), and the number of optimization iterations can be set to 20 to 50.

[0045] In the closed-loop optimization of light source parameters, the white light intensity of the first light source module, the ultraviolet intensity of the second light source module, and the near-infrared structured light intensity of the third light source module in step S1 are used as adjustable parameters. The average confidence level of all defect detection results output in step S4 is used as the optimization target. Bayesian optimization or reinforcement learning methods are used to collaboratively adjust the output intensity of the three light source modules. Specifically, the adjustment range of the light source intensity can be defined as 50% to 100% of their respective rated values, and the optimization target is to maximize the average confidence level. Light source parameter optimization is triggered after each certain batch (e.g., 1000 shells). The optimized parameters are written to the light source driver through a digital-to-analog conversion interface or serial communication and applied to subsequent batches. The entire continuous learning and optimization process is executed by an offline server or industrial control computer background and does not affect the main process of online detection.

[0046] The beneficial effects of this technical solution include the following: First, it automatically generates positive sample pseudo-labels using the normal areas of repair-free products and negative sample pseudo-labels using scrapped and repairable areas, continuously obtaining training data without manual annotation and significantly reducing data annotation costs. Second, the three-level optimization pool addresses short-term distribution drift, long-term knowledge retention, and new category discovery, respectively. The incremental learning strategy enables the system to adapt to new defects that may arise during product iteration, avoiding catastrophic forgetting. Third, the elastic weight consolidation regularization loss function protects important parameters from the old task when updating model parameters, maintaining the detection capability of the original defect categories while introducing new samples. Fourth, Bayesian optimization automatically adjusts the grading threshold, ensuring a high recall rate under false alarm rate constraints and adapting to dynamic changes in production line quality standards. Fifth, closed-loop optimization of light source parameters enables the system to automatically adjust the lighting intensity based on detection confidence feedback, compensating for the effects of light source aging and environmental changes, and extending the stable operation cycle of the equipment. Through the above mechanisms, the detection system can continuously evolve after deployment, reducing the need for manual intervention and downtime retraining, and improving long-term operational stability and adaptability.

[0047] Example 1 A batch of semiconductor equipment has an aluminum alloy casing with a directional textured surface after grinding, and its dimensions are 400mm × 300mm. The conveyor speed is set to 100mm / s. When the casing enters the inspection station, the photoelectric sensor triggers a synchronization signal. The first light source module uses a white LED coaxial light source with an illumination angle of 10° to capture color reflection images. The second light source module uses an ultraviolet LED ring dark field light source (wavelength 365nm) with an illumination angle of 60°. The ring light source is installed around the camera lens to capture grayscale scattering images. The third light source module uses a digital projector to project near-infrared sinusoidal grating stripes (wavelength 850nm) onto the casing surface to capture phase-modulated images. All three cameras are 2-megapixel industrial CMOS cameras, each equipped with a bandpass filter corresponding to its wavelength, and are synchronously acquired via a hardware trigger line.

[0048] A two-dimensional Fourier transform is performed on the phase-modulated image. The fundamental frequency component is extracted in the frequency domain using a Hanning window, and the inverse transform yields the wrapped phase. After expansion using a branching method and incorporating calibration parameters, the height is calculated to generate a single-channel height map. The R, G, and B channels of the color reflectance image are then pixel-level registered with the height map and the dark-field scattering image. After remapping based on the homography matrix determined by the checkerboard calibration board, the registration error between each channel image is minimized to no more than one pixel, thus constructing a five-channel fused feature map.

[0049] The five-channel fused feature map is input into a multi-branch convolutional neural network. The network backbone consists of the first four convolutional blocks of a ResNet-18. The first branch connects to three deformable convolutional layers (3×3 kernels, 64 output channels), the second branch connects to two non-local attention layers (64 output channels), and the third branch connects to two coordinate attention layers (64 output channels). The three outputs are concatenated and then passed through two fully connected layers to obtain the defect category and geometric parameters. This multi-branch convolutional neural network outputs the defect category and geometric parameters, and for each detected defect, it outputs a confidence score, i.e., the maximum probability value output by the softmax branch of the classification branch, to characterize the reliability of the detection result. The network output shows: a scratch in the upper right corner of the shell surface, with a maximum depth of 8μm and a maximum length of 2.5mm; a pit in the lower left corner, with a maximum depth of 12μm and a maximum length of 1.8mm; and a particle in the center, with a diameter of 15μm.

[0050] Based on the coordinates of the defect locations, neither the scratches nor the dents are located on the sealing surface or in the electrode interface area. For the scratches, 8μm is between 5μm and 20μm, and 2.5mm is less than 5mm, therefore it is marked as repairable. For the dents, 12μm is between 5μm and 20μm, and 1.8mm is less than 5mm, also marked as repairable. The final inspection report indicates that the casing has three repairable defects, and polishing or cleaning is recommended. The entire inspection process took approximately 1.2 seconds.

[0051] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. An online detection method for surface defects in semiconductor device housings, characterized in that, Includes the following steps: S1. Synchronous acquisition of multimodal optical signals: When the semiconductor equipment housing moving on the conveyor line enters the inspection station, the first light source module, the second light source module, and the third light source module are simultaneously triggered; wherein, the first light source module illuminates the housing surface in a coaxial bright field illumination mode and acquires a first reflection image; the second light source module illuminates the housing surface in a ring dark field illumination mode and acquires a second scattering image; the third light source module projects structured light stripes onto the housing surface and acquires a phase modulation image; S2. Three-dimensional topography reconstruction: Demodulate the three-dimensional topography data of the shell surface from the phase modulation image and generate a height map; S3. Five-channel feature fusion: The first reflection image is a color image. Its three color channels, R, G, and B, are registered pixel-level with the height map and the second scattering image to construct a five-channel fusion feature map. The five channels are, in order, the R channel, the G channel, the B channel, the height map channel, and the grayscale channel of the second scattering image. S4. Defect Detection: The five-channel fused feature map is input into a multi-branch convolutional neural network; the multi-branch convolutional neural network includes at least a first branch and a second branch, the first branch is used to extract the morphological features of the defect, and the second branch is used to extract the texture features of the defect; the outputs of the first branch and the second branch are fused, and then the defect category and the geometric parameters of the defect are output. S5. Grading determination: The semiconductor device casing is graded according to the geometric parameters of the defect and the location information of the defect.

2. The online detection method for surface defects in a semiconductor device casing as described in claim 1, characterized in that, In step S4, the first branch uses deformable convolution, the second branch uses a non-local attention module, and the multi-branch convolutional neural network also includes a third branch, which uses a coordinate attention mechanism to extract the correlation features between defects and the processing texture direction.

3. The online detection method for surface defects in a semiconductor device casing as described in claim 2, characterized in that, The defect categories output in step S4 include scratches, dents, particles, and stains; the defect geometry parameters output in step S4 include the maximum depth and maximum length of the defect.

4. The online detection method for surface defects in a semiconductor device casing as described in claim 3, characterized in that, The grading determination in step S5 specifically refers to: If the defect category is particles, then mark them as repairable or scrapped based on the particle diameter and whether they are located in a preset critical area; If the defect category is stain, it will be marked as repairable or scrapped based on the stain area and whether it is located in a preset critical area; If the defect category is scratch or dent, it is classified according to its maximum depth, maximum length, and whether it is located in a preset critical area.

5. The online detection method for surface defects in a semiconductor device housing as described in claim 4, characterized in that, The preset key areas include the sealing surface and the electrode interface area; wherein, the threshold for graded determination is set as follows: When the defect category is scratch or dent, if the maximum depth is ≤5μm, the maximum length is ≤1mm and it is not located in any critical area, it is marked as repair-free; if the maximum depth is ≤20μm and the maximum length is ≤5mm and it is not located in any critical area, it is marked as repairable; if the maximum depth is >20μm or the maximum length is >5mm or it is located in any critical area, it is marked as scrap. When the defect category is particles, if the particle diameter is ≤30μm and it is not located on the sealing surface or electrode interface area, it is marked as repairable; if the particle diameter is >30μm or it is located in any critical area, it is marked as scrap. When the defect category is stain, if the stain area is ≤0.5mm 2 If the stain is not located on the sealing surface or electrode interface area, it is marked as repairable; if the stain area is >0.5mm... 2 If it is located in any critical area, it is marked as scrap.

6. The online detection method for surface defects in a semiconductor device housing as described in claim 5, characterized in that, After step S1 and before step S2, a multi-angle dark field analysis step is also included, which includes: Step S1a, Acquisition of partitioned dark field images: Set the second light source module as an independently controllable multi-partition ring dark field light source, and divide it into N independently controllable partitions, N≥4; light up each partition in sequence, and acquire the corresponding dark field image through the second camera when each partition is lit, and acquire a total of N partitioned dark field images; Step S1b, Anisotropy Coefficient Calculation: For each pixel position (x,y), based on its gray value sequence I1(x,y), I2(x,y), …, I in the N partitioned dark field images... n Given (x,y), calculate the anisotropy coefficient A(x,y) of this pixel: A(x,y) = (max(I i ) - min(I i )) / (max(I i ) + min(I i ) + ε); Where ε is a preset small positive number to avoid division by zero, max(I i ) and min(I i These represent the maximum and minimum grayscale values ​​of the pixel under different zone lighting conditions; Step S1c, Texture-Defect Differentiation: The anisotropy coefficient A(x,y) is compared with a preset threshold A. th Comparison: If A(x,y) ≥ A th If A(x,y) is a directional texture, then the pixel is determined to belong to the directional texture. th If so, the pixel is determined to be a genuine defect;​ Step S1d, False Defect Filtering: A texture mask is generated based on the determination result of step S1c. When constructing the five-channel fusion feature map in step S3, the feature values ​​corresponding to the processed texture pixels are set to invalid according to the texture mask, so that the multi-branch convolutional neural network in step S4 only detects and classifies the real defect areas.

7. The online detection method for surface defects in a semiconductor device housing as described in claim 6, characterized in that, It also includes a hierarchical feedback-driven continuous learning optimization step, which is performed after step S5: Step S6a, Confidence-Aware Pseudo-Label Generation: For products marked as repair-free in step S5, extract their five-channel fusion feature map, and construct a normal region mask based on the defect detection results output in step S4; use the feature regions covered by the normal region mask as positive sample pseudo-labels, use the regions marked as scrap as hard negative sample pseudo-labels, and calculate the confidence weight w=exp(−D) for the regions marked as repairable based on their defect geometric parameters. max / δ), where δ is a preset scale parameter, used as a pseudo-label for soft negative samples; Step S6b, Multi-level Optimization Pool Management: Construct a three-level optimization pool, including a short-term sliding window pool, a medium-term prototype pool, and a long-term cache pool; add the positive sample pseudo-labels, hard negative sample pseudo-labels, and soft negative sample pseudo-labels generated in step S6a to the three-level optimization pool; adopt a class-incremental learning strategy, and when the number of samples of the newly added defect category exceeds a preset threshold, automatically create a new category prototype and assign a new category ID; Step S6c, Incremental Update of Model Parameters: An incremental update of the model parameters is triggered every preset time interval or when the number of samples accumulated in the short-term sliding window pool described in step S6b reaches a preset batch size; an elastic weight-reinforced regularized loss function is used. ; Where L is the total loss, L t For the detection loss in the current batch, F i For parameter θ i The diagonal elements of the Fisher information matrix, θ i,o These are the parameters of the old model, and λ is the tradeoff coefficient. Step S6d, adaptive adjustment of the threshold for grading determination: Based on the detection results of the verification batch data by the model updated in step S6c, the threshold parameter for grading determination in claim 6 is adjusted using the Bayesian optimization method. Step S6e, Closed-loop optimization of light source parameters: The white light intensity of the first light source module, the ultraviolet light intensity of the second light source module, and the near-infrared structured light intensity of the third light source module in step S1 are used as adjustable parameters. The confidence level of defect detection output in step S4 is used as the optimization target. Bayesian optimization or reinforcement learning methods are used to coordinately adjust the output intensity of the three light source modules.