A method and system for detecting surface quality of semiconductor carrier tape based on machine vision

Abstract

This invention provides a method and system for inspecting the surface quality of semiconductor carrier tape based on machine vision, belonging to the field of machine vision inspection technology. The method includes: driving the surface of the semiconductor carrier tape to be inspected, which is wound between a feed roller and a take-up roller, through an inspection station, and acquiring real-time image information of the semiconductor carrier tape surface at the inspection station; extracting multi-scale texture features to characterize the surface state of the semiconductor carrier tape based on the preprocessed image information; performing classification and recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; determining the position coordinates of the defect on the semiconductor carrier tape according to the defect recognition result and the operating parameters of the feed roller and the take-up roller, and outputting the corresponding defect location information. This invention not only improves the accuracy and sensitivity of semiconductor carrier tape defect detection, but also realizes the mapping of defect location to the actual operating semiconductor carrier tape.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and more specifically to a machine vision-based method and system for inspecting the surface quality of semiconductor carrier tapes. Background Technology

[0002] In the semiconductor device packaging and transport process, the semiconductor carrier tape serves as the carrier for chips, components, and other devices. Its surface quality directly affects the subsequent packaging accuracy and the yield rate of electronic components. However, the current industry-standard methods for inspecting the surface quality of semiconductor carrier tapes mainly involve random visual inspection or using a single-angle industrial camera at the end of the production line for photographic evaluation. These methods not only have a limited scope of inspection but also struggle to cover all types of defects in the production process. Surface defects such as micro-indentations, fine cracks, burrs, bubbles, and film wrinkles are often difficult to accurately identify due to factors such as light reflection, interference from transparent film layers, and semiconductor carrier tape vibration. This results in a large number of minute defects being discovered only after the tape is wound up and stored, leading to the risk of irreversible batch scrap.

[0003] On the other hand, because semiconductor carrier tapes move at high speeds in a roll-to-roll manner during the processing production line, existing online inspection technologies generally suffer from technical weaknesses such as asynchronous image frame acquisition and semiconductor carrier tape movement, and inaccurate mapping of defect locations. Especially when the semiconductor carrier tape's operating speed fluctuates, detection delays, image ghosting, missed detections, or misjudgments are prone to occur, severely affecting the reliability of the identification results. Furthermore, existing defect judgment algorithms mostly rely on fixed templates or statistical thresholds, exhibiting weak adaptability and poor generalization ability for newly emerging subtle defects or defect types with limited samples, failing to meet the needs of practical intelligent inspection.

[0004] In summary, there is an urgent need for a semiconductor carrier surface quality inspection method that can achieve highly sensitive detection of multiple defect types during continuous operation of semiconductor carriers, effectively avoid the interference of specular reflection and motion blur, and has the ability to accurately map defect location, so as to improve the defect monitoring capability and product yield of semiconductor carrier production process. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for inspecting the surface quality of semiconductor carrier tape based on machine vision, so as to at least solve the problems of insensitive defect identification and inaccurate defect location mapping in existing online inspection of semiconductor carrier tape.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for inspecting the surface quality of semiconductor carrier tape based on machine vision. The method is applied to a semiconductor carrier tape inspection device, which includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for winding the semiconductor carrier tape. The feed roller and the take-up roller are connected by a transmission mechanism that drives the semiconductor carrier tape to continuously move along a set running trajectory. An image acquisition device is installed on the running trajectory. The method includes:

[0007] The feeding roller and the receiving roller are driven to rotate, so as to drive the semiconductor carrier tape to move continuously along the set running trajectory and pass through the detection station, and the semiconductor carrier tape surface image information is collected in real time at the detection station;

[0008] The acquired image information is preprocessed, and multi-scale texture features for characterizing the surface state of the semiconductor carrier are extracted based on the preprocessed image information.

[0009] Based on the texture features, classification and identification are performed in the defect identification model to determine whether there are defects on the surface of the semiconductor carrier tape.

[0010] Based on the defect identification results and the operating parameters of the feeding roller and the receiving roller, the position coordinates of the defect on the semiconductor carrier are determined, and the corresponding defect location information is output.

[0011] Optionally, the image acquisition device includes:

[0012] A multi-angle composite polarization light source and a corresponding linear industrial camera array are positioned above the semiconductor carrier tape's running trajectory.

[0013] The real-time acquisition of semiconductor carrier tape surface image information at the inspection station includes:

[0014] While the multi-angle composite polarization light source irradiates the semiconductor carrier surface with oblique light at different polarization angles and emission angles, the linear array of industrial cameras synchronously acquires images of the semiconductor carrier surface according to exposure parameters matched with the running speed of the semiconductor carrier, thereby obtaining real-time semiconductor carrier surface image information.

[0015] Optionally, preprocessing is performed on the acquired image information, including:

[0016] Motion compensation based on optical flow vectors is performed on adjacent image frames to eliminate image ghosting caused by fluctuations in the semiconductor carrier speed;

[0017] The compensated image is then subjected to edge enhancement and background autoencoder filtering processes in sequence to obtain preprocessed image information.

[0018] Optionally, multi-scale texture features for characterizing the surface state of the semiconductor carrier are extracted based on the preprocessed image information, including:

[0019] Multi-scale texture analysis based on Laplacian wavelet decomposition is performed on the preprocessed image information to construct texture coefficient matrices at different scales;

[0020] Based on the fusion of the texture coefficient matrix at different scales with the gray-level co-occurrence matrix and Gabor filtering results, a texture description vector is formed, which serves as a multi-scale texture feature for characterizing the surface state of the semiconductor carrier.

[0021] Optionally, perform multi-scale texture analysis based on Laplacian wavelet decomposition on the preprocessed image information to construct texture coefficient matrices at different scales, including:

[0022] Each image frame is sequentially subjected to Gaussian blur and downsampling operations to construct a Laplacian pyramid structure.

[0023] In each scale level of the Laplacian pyramid structure, residual signals are extracted based on the difference between the corresponding scale image and its previous resolution image to form a texture response matrix characterizing the scale response.

[0024] Based on the texture response matrix at each scale, regional energy features are extracted and statistically fused with the principal direction feature vector to finally obtain a texture coefficient matrix covering different scales.

[0025] Optionally, a texture description vector is formed by fusing the gray-level co-occurrence matrix and Gabor filtering results based on texture coefficient matrices at different scales, including:

[0026] For each scale, a gray-level co-occurrence matrix is ​​constructed according to a preset neighborhood window for the texture coefficient matrix.

[0027] Based on the gray-level co-occurrence matrix, statistical measures including energy, contrast, and entropy are extracted as the first type of texture features;

[0028] Gabor filtering operations are performed on the texture coefficient matrix with different orientation and frequency parameters, and the amplitude features of the filtering response are extracted as the second type of texture features;

[0029] The first type of texture features and the second type of texture features are cascaded and fused in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

[0030] Optionally, based on the texture features, classification and identification are performed in the defect recognition model to determine whether there are defects on the semiconductor carrier surface, including:

[0031] The texture description vector is input into a few-sample defect recognition model built on a prototype network. The feature distance between the texture description vector and the prototype vector of each type of defect is matched and calculated to obtain the corresponding defect category output probability.

[0032] When the output probability exceeds a preset threshold, it is determined that a defect of the corresponding defect category exists on the surface of the semiconductor carrier tape; wherein,

[0033] Defect categories include:

[0034] At least one of the following defects: indentation, scratch, burr, bump, and bubble.

[0035] Optionally, the construction rules for the few-sample defect recognition model based on the prototype network are as follows:

[0036] The mean vector of the texture description vector for each type of defect is calculated using the labeled samples and used as the prototype vector of the corresponding defect type.

[0037] During the training phase, the Euclidean distance between the texture description vector of the labeled sample and the prototype vectors of various types is used as the loss function to optimize the model, so that samples of the same defect category are aggregated to their corresponding prototypes, and samples of different defect categories are far away from their respective prototypes, thus obtaining a small sample prototype recognition model that can complete defect classification based on feature distance.

[0038] Optionally, the position coordinates of the defect on the semiconductor carrier tape are determined based on the defect identification result and the operating parameters of the feed roller and the take-up roller, and the corresponding defect location information is output, including:

[0039] Obtain the real-time rotational speed information of the feeding roller and the receiving roller at the moment of defect identification;

[0040] A mapping model between volume diameter and frame number is constructed based on the real-time rotation speed information;

[0041] Based on the mapping relationship model, the frame number where the defect is located is mapped to the longitudinal displacement length coordinate of the semiconductor carrier tape, and the two-dimensional coordinate position of the defect is determined by combining the set width of the semiconductor carrier tape, thereby generating defect location information that includes the defect type and spatial location.

[0042] A second aspect of the present invention provides a semiconductor carrier tape surface quality inspection system based on machine vision. The system is applied to a semiconductor carrier tape inspection device. The device includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for winding the semiconductor carrier tape. The feed roller and the take-up roller are connected by a transmission mechanism to drive the semiconductor carrier tape to continuously move along a set running trajectory. An image acquisition device is installed on the running trajectory. The system includes: an acquisition unit for driving the feed roller and the take-up roller to rotate, thereby driving the semiconductor carrier tape to continuously move along the set running trajectory and pass through an inspection station, and acquiring real-time image information of the semiconductor carrier tape surface at the inspection station; a processing unit for performing preprocessing on the acquired image information and extracting multi-scale texture features characterizing the surface state of the semiconductor carrier tape based on the preprocessed image information; a defect identification unit for performing classification and identification in a defect identification model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; and an output unit for determining the position coordinates of the defect on the semiconductor carrier tape according to the defect identification result and the running parameters of the feed roller and the take-up roller, and outputting the corresponding defect location information.

[0043] Through the above technical solution, this invention achieves online continuous inspection of semiconductor carriers by constructing a continuously moving trajectory for the semiconductor carrier tape between the feeding roller and the receiving roller, and by acquiring real-time image information of the semiconductor carrier tape surface at the inspection station. Preprocessing and multi-scale texture feature extraction techniques are introduced into the image processing, making the identification of defects such as fine indentations, scratches, and bubbles more sensitive and reliable. Furthermore, by constructing a defect recognition model to classify and judge image texture features, and combining the roller running parameters to map and locate the defect location, the invention can accurately output the defect location coordinate information. Therefore, this invention not only improves the accuracy and sensitivity of semiconductor carrier tape defect detection, but also realizes the mapping of defect location to the actual operating semiconductor carrier tape, providing a reliable basis for subsequent rejection and alarm control, thereby significantly improving the quality monitoring capability of the semiconductor carrier tape production process.

[0044] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0046] Figure 1 This is a flowchart of the steps of a semiconductor carrier tape surface quality detection method based on machine vision provided by one embodiment of the present invention;

[0047] Figure 2 This is a semiconductor carrier testing device provided in one embodiment of the present invention;

[0048] Figure 3 This is a system structure diagram of a semiconductor carrier tape surface quality inspection system based on machine vision provided in one embodiment of the present invention.

[0049] Explanation of reference numerals in the attached figures:

[0050] 10 - Feeding roller; 20 - Receiving roller; 30 - Semiconductor carrier tape; 40 - Image acquisition device;

[0051] 401 - Multi-angle composite polarization light source; 402 - Linear array industrial camera. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0053] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0054] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0055] like Figure 1 As shown, this invention provides a machine vision-based method for inspecting the surface quality of semiconductor carrier tapes, the method comprising:

[0056] Step S1: Drive the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier tape to move continuously along the set running trajectory and pass through the detection station, and collect the surface image information of the semiconductor carrier tape in real time at the detection station.

[0057] Specifically, the machine vision-based surface quality inspection method for semiconductor carrier tape 30 proposed in this invention is applied to semiconductor carrier tape 30 inspection equipment, such as... Figure 2 The device includes a feed roller 10 for releasing the semiconductor carrier tape and a take-up roller 20 for winding the semiconductor carrier tape. The feed roller 10 and the take-up roller 20 are connected by a transmission mechanism to drive the semiconductor carrier tape to move continuously along a set running trajectory. An image acquisition device 40 is provided on the running trajectory.

[0058] The feeding roller 10 is used to release the initial semiconductor carrier tape, and the take-up roller 20 is used to rewind the inspected semiconductor carrier tape. A tension transmission assembly maintains a constant tension on the semiconductor carrier tape, allowing it to pass through the inspection station. An image acquisition device 40 is fixedly installed above the semiconductor carrier tape's running trajectory. It includes a light source assembly and an image acquisition assembly. The light source assembly is matched with a polarizer to achieve illumination at a specific polarization angle, thereby reducing specular reflection interference on the surface of the semiconductor carrier tape 30.

[0059] The image acquisition component is preferably a linear array of industrial cameras 402, used to continuously acquire image frame sequences along the movement direction of the semiconductor carrier tape. Further, the image acquisition device 40 is communicatively connected to the transmission mechanism via a trigger unit. The trigger unit adjusts the exposure time and sampling frequency of the image acquisition according to the actual running speed of the semiconductor carrier tape to ensure that the surface image of the semiconductor carrier tape remains clear and identifiable even at high speeds. By setting the image acquisition device 40 along the running trajectory of the semiconductor carrier tape, continuous and real-time image sampling of the surface of the semiconductor carrier tape 30 can be performed without affecting the transmission process, providing a raw data basis for subsequent defect identification and location, thereby achieving the purpose of online non-contact inspection.

[0060] Preferably, the image acquisition device 40 includes: a multi-angle composite polarization light source 401 disposed above the running trajectory of the semiconductor carrier 30 and a corresponding linear array of industrial cameras 402; real-time acquisition of surface image information of the semiconductor carrier 30 at the inspection station includes: while the multi-angle composite polarization light source 401 irradiates the surface of the semiconductor carrier 30 with oblique light at different polarization angles and emission angles, the linear array of industrial cameras 402 synchronously acquires surface images of the semiconductor carrier 30 according to exposure parameters matched with the running speed of the semiconductor carrier 30, thereby obtaining real-time surface image information of the semiconductor carrier 30.

[0061] In this embodiment of the invention, the multi-angle composite polarization light source 401 includes at least two strip-shaped high-brightness light sources set at different pitch angles. Each light source is equipped with an electrically controlled polarizer. The polarization direction of the irradiated light can be dynamically controlled by adjusting the polarization angle of the polarizer. The oblique light emitted by the light source irradiates the surface of the semiconductor carrier 30 at different incident angles, so that surface scratches, micro-pits, bubbles and thin film defects produce different strong reflection areas under illumination from different directions.

[0062] The linear array of industrial cameras 402 is located above the reflection direction of the multi-angle composite polarization light source 401. Each camera is set with a different viewing angle corresponding to a different incident angle light source, and its frame rate is adjustable. During the acquisition process, the current running speed of the semiconductor carrier 30 is first obtained in real time based on the transmission mechanism. This speed is used as a parameter input into the exposure control program to determine the exposure time, frame interval, and sampling trigger time of each frame of the linear array industrial cameras 402. This enables synchronous triggering of multiple cameras to ensure that the acquired image frame sequence is consistent with the actual displacement of the semiconductor carrier surface, thereby avoiding image ghosting and distortion under high-speed transmission conditions.

[0063] Synchronously, the multi-angle composite polarization light source 401 is controlled to scan the semiconductor carrier surface in turn according to the light source sequence. Each time a set of polarization angles and incident angles are switched, the corresponding industrial camera is triggered to acquire images under the illumination conditions, thereby obtaining multi-channel semiconductor carrier surface images under different illumination angles and different polarization angles.

[0064] Furthermore, each image acquired by the camera is accompanied by an independent frame number and exposure parameter identifier, enabling subsequent multi-channel image fusion and defect feature enhancement processing using a roll-to-roll coordinate mapping model. By employing the aforementioned real-time acquisition method combining composite polarized oblique illumination with a linear industrial camera array, the sensitivity of images to subtle defects on the semiconductor carrier surface can be significantly improved, specular reflection interference can be suppressed, and clear and reliable surface images can be obtained under different motion speeds, providing a stable and sufficient image data foundation for subsequent high-precision defect identification.

[0065] Step S2: Perform preprocessing on the acquired image information, and extract multi-scale texture features to characterize the surface state of the semiconductor carrier based on the preprocessed image information.

[0066] Specifically, motion compensation based on optical flow vectors is performed on adjacent image frames to eliminate image ghosting caused by fluctuations in the semiconductor carrier speed; edge enhancement and background autoencoding filtering are then performed on the compensated images to obtain preprocessed image information.

[0067] Furthermore, multi-scale texture features for characterizing the surface state of semiconductor carriers are extracted based on the preprocessed image information, including: performing multi-scale texture analysis based on Laplacian wavelet decomposition on the preprocessed image information to construct texture coefficient matrices at different scales; and fusing the gray-level co-occurrence matrix and Gabor filtering results based on the texture coefficient matrices at different scales to form a texture description vector, which serves as a multi-scale texture feature for characterizing the surface state of semiconductor carriers.

[0068] In this embodiment of the invention, during the continuous roll-to-roll movement of the semiconductor carrier tape through the inspection station, factors such as transmission speed fluctuations, instantaneous acceleration and deceleration, and tension changes inevitably exist. These factors cause displacement deviations in the corresponding image content between different frames, resulting in ghosting or blurring in the subsequent defect detection stage.

[0069] To address this issue, motion compensation based on optical flow vectors is first performed on adjacent image frames. By estimating the optical flow vector distribution of a specific set of pixels on the semiconductor carrier surface in consecutive frames, the global displacement or local vector field of the image is calculated. This allows for reverse displacement compensation of the subsequent frame, maintaining visual spatial consistency between the two frames. After motion compensation, edge enhancement is performed using a method based on the Sobel operator and morphological gradient enhancement. This enhances the contrast of fine textures such as scratches and pits by highlighting areas with rapid changes in pixel grayscale. Furthermore, an autoencoder is used to construct a baseline background model. The autoencoder reconstruction network is trained on a large number of defect-free images, enabling it to reconstruct the normal surface texture of the semiconductor carrier. The residual between the original image and the reconstruction result is then used as the background filtering result to obtain preprocessed image information, suppressing interference from irrelevant features such as changes in surface gloss, film creases, and dust on the semiconductor carrier.

[0070] Specifically, multi-scale texture analysis based on Laplacian wavelet decomposition is performed on the preprocessed image information to construct texture coefficient matrices at different scales. This includes: performing Gaussian blur and downsampling operations on each image frame sequentially to construct a Laplacian pyramid structure; extracting residual signals based on the difference between the corresponding scale image and its previous resolution image in each scale level of the Laplacian pyramid structure to form a texture response matrix representing the scale response; and statistically fusing regional energy features and principal direction feature vectors extracted from the texture response matrices at each scale to finally obtain texture coefficient matrices covering different scales.

[0071] Furthermore, a texture description vector is formed by fusing the gray-level co-occurrence matrix and Gabor filtering results based on the texture coefficient matrix at different scales. This includes: constructing a gray-level co-occurrence matrix for the texture coefficient matrix at each scale according to a preset neighborhood window; extracting statistics including energy, contrast, and entropy as first-type texture features based on the gray-level co-occurrence matrix; performing Gabor filtering operations on the texture coefficient matrix with different direction and frequency parameters to extract the amplitude features of the filtering response as second-type texture features; and cascading and fusing the first-type texture features and the second-type texture features in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

[0072] In this embodiment of the invention, in order to accurately reflect the texture response of defects such as indentations, fine scratches, local undulations and punching burrs on the semiconductor carrier surface at different spatial scales, Gaussian blur and fixed-multiple downsampling operations are sequentially performed on each frame of the image after motion compensation and background filtering to construct a Laplacian pyramid structure. Gaussian blur is used to preserve the low-frequency structural information of the image, and downsampling is used to progressively reduce the resolution, forming a pyramid-shaped image sequence containing multiple resolution levels.

[0073] In each scale level of the Laplacian pyramid structure, the low-resolution image corresponding to that level is first upsampled to the resolution of the previous scale, and then differentially analyzed with the original image of the previous scale to obtain a residual signal matrix reflecting the high-frequency texture details of the image at that scale. This residual signal matrix is ​​used to construct a texture response matrix characterizing the scale response, thereby preserving the sensitivity to texture changes at different scales. Then, for the texture response matrix at each scale, regional energy features and principal direction feature vectors are extracted using the local energy accumulation rule and the gradient principal direction statistical rule, respectively. These features are then fused to obtain an initial texture feature vector characterizing the surface roughness, directionality, and local texture distribution of the semiconductor carrier strip.

[0074] Repeat the above process to fully acquire the variations in texture energy and directionality across the range from the finest texture scale to the largest texture structure scale, ultimately forming a texture coefficient matrix covering multiple scale levels. This Laplacian pyramid multi-scale analysis method comprehensively reflects the hierarchical distribution characteristics of the semiconductor carrier surface texture, which is beneficial for improving the robust identification capability of defects with different morphologies.

[0075] Furthermore, based on the texture coefficient matrix at different scales, the gray-level co-occurrence matrix and Gabor filtering results are fused to form the final texture description vector used as input in the defect identification stage.

[0076] Specifically, a gray-level co-occurrence matrix is ​​constructed for the texture coefficient matrix at each scale according to a preset neighborhood window. By statistically analyzing the common distribution probability of pixel gray-level pairs in the gray-level co-occurrence matrix in the horizontal direction, vertical direction, and other representative angles, statistical quantities such as energy, contrast, entropy, and correlation are calculated and used as the first type of texture feature to describe the coarseness and structural complexity of the local surface texture. At the same time, a set of Gabor kernel functions with different directions (e.g., 0°, 45°, 90°, and 135°) and different frequency parameters are used to perform Gabor filtering operations on the texture coefficient matrix. A direction-selective filter response map is obtained through convolution operation. The mean amplitude, variance amplitude, and maximum response position are then extracted from the filter response map as the second type of texture feature to reflect the repeatability, periodicity, and detail fluctuations of the texture in different directions on the semiconductor carrier surface.

[0077] Furthermore, following the order of scale hierarchy from low frequency to high frequency in the texture coefficient matrix, the first-class texture features and second-class texture features obtained at each scale are concatenated and combined one by one to construct a multi-scale texture description vector. This texture description vector comprehensively integrates texture statistical features and directional response information at various scales, directions, and frequencies, possessing high-dimensional and multi-angle texture expression capabilities. It can fully characterize the true texture state of the semiconductor carrier surface, providing a more accurate and stable input basis for subsequent detection of subtle defects and category recognition, significantly improving detection sensitivity and accuracy. Especially under complex backgrounds, illumination disturbances, and interference from transparent film structures, it can still maintain good feature discrimination effects.

[0078] Step S3: Based on the texture features, perform classification and recognition in the defect recognition model to determine whether there are defects on the surface of the semiconductor carrier tape.

[0079] Specifically, the texture description vector is input into a small sample defect recognition model built on a prototype network, and the feature distance between the texture description vector and the prototype vector of each type of defect is matched and calculated to obtain the corresponding defect category output probability; when the output probability exceeds a preset threshold, it is determined that there is a defect of the corresponding defect category on the semiconductor carrier surface; wherein, the defect category includes at least one of the following: indentation, scratch, burr, bump and bubble.

[0080] Specifically, the construction rules for the small sample defect recognition model based on the prototype network are as follows: the mean vector of the texture description vector of each type of defect is calculated using labeled samples as the prototype vector of the corresponding defect type; during the training phase, the Euclidean distance between the texture description vector of the labeled samples and the prototype vectors of each type is used as the loss function to optimize the model, so that samples of the same defect type are aggregated to their corresponding prototypes, and samples of different defect types are far away from their respective prototypes, thus obtaining a small sample prototype recognition model that can complete defect classification based on feature distance.

[0081] In this embodiment of the invention, the texture description vector is input into a small sample defect recognition model constructed based on a prototype network, and the feature distance between the texture description vector and the prototype vector of each category of defect is matched and calculated to obtain the corresponding defect category output probability; when the output probability exceeds a preset threshold, it is determined that there is a defect of the corresponding defect category on the surface of the semiconductor carrier; wherein, the defect category includes at least one defect among indentation, scratch, burr, bump and bubble.

[0082] To achieve robust and accurate classification and recognition even with a limited number of defect samples, the model employs a prototype network structure. Specifically, the construction rules for the small-sample defect recognition model based on the prototype network are as follows: For each defect class, a texture description vector is calculated using labeled defect samples. The average of the texture description vectors for each class is used as the prototype vector corresponding to that defect class. During model training, the Euclidean distance is calculated between the texture description vector of each defect sample and the prototype vectors of multiple defect classes. This Euclidean distance is used as the core metric for the loss function to optimize and adjust the network, gradually bringing samples of the same class closer to their corresponding prototype vectors in the feature space, while simultaneously increasing the distance between samples of different classes. After training, a small-sample prototype recognition model is formed that can classify defects based on the distance relationship between the texture description vector and the prototype vector. This model can effectively learn even with only a small number of defect training samples.

[0083] To further improve the stability and generalization of the model under actual semiconductor carrier tape production conditions, the prototype vector update logic adopts a training-while-updating approach to iteratively correct the original prototype vector of each category. That is, if a new sample is found to have a distance deviation from the existing prototype that is lower than the dynamic threshold during the training process, the texture description vector of that sample is added to the prototype mean update according to the weight, so that the constructed defect prototypes have higher representativeness and adaptability.

[0084] In the recognition stage, the texture description vector of the image to be detected is first calculated, and then the Euclidean distance is calculated with the prototype vector of each defect category. The distance value is normalized by the softmax function to obtain the output probability distribution of each category. If the probability value corresponding to a certain defect category exceeds the preset threshold, the recognition result of the defect of that category is output. If the output probability of each category does not reach the threshold, it is determined that there is no significant defect on the semiconductor carrier surface at the current detection position.

[0085] Using the above methods, the surface defect identification task can still be stably completed in production environments where there are very few defect samples or new defects are constantly emerging, significantly improving the detection accuracy and reliability of practical applications. Furthermore, the model parameters can be quickly adjusted according to the differences in texture characteristics of different semiconductor carrier materials to adapt to different semiconductor carrier products.

[0086] Step S4: Determine the position coordinates of the defect on the semiconductor carrier tape based on the defect identification results and the operating parameters of the feeding roller and the receiving roller, and output the corresponding defect location information.

[0087] Specifically, the real-time rotational speed information of the feeding roller and the receiving roller at the moment of defect identification is obtained; a mapping relationship model between roll diameter and frame number is constructed based on the real-time rotational speed information; the frame number where the defect is identified is mapped to the longitudinal displacement length coordinate of the semiconductor carrier tape based on the mapping relationship model, and the two-dimensional coordinate position of the defect is determined by combining the set width of the semiconductor carrier tape, thereby generating defect location information that includes defect type and spatial position.

[0088] In this embodiment of the invention, the rotational speed of the feeding roller and the rotational speed of the receiving roller at the moment of defect identification are obtained, and the sampling trigger time of the corresponding image frame is recorded respectively. At the same time, the image frame number of the industrial camera is timestamped with the number of pulses output by the displacement encoding device of the feeding roller and the receiving roller to establish the correlation basis between the image acquisition frame number and the actual running distance of the semiconductor carrier.

[0089] Subsequently, based on the fixed tension parameters between the feed roller and take-up roller during the online operation of the semiconductor carrier tape in roll-to-roll processing, a mathematical mapping model between roll diameter growth and rotational speed was established. Combined with the real-time rotational speeds of the feed roller and take-up roller, the instantaneous linear velocity of the semiconductor carrier tape at the identification moment was calculated. The image frame number and this linear velocity were input into the mapping model to obtain the cumulative running distance from the image sampling start frame to the defect identification frame, thus accurately mapping the frame number where the defect was identified to the longitudinal (X-direction) displacement length coordinate of the semiconductor carrier tape.

[0090] Furthermore, by combining the known semiconductor carrier tape width value, and through the correspondence between the fixed pixel coordinates of the camera imaging area in the semiconductor carrier tape's transverse (Y direction) direction and the imaging calibration parameters, the transverse pixel position of the identified defect in the image is mapped to the transverse length coordinates of the actual semiconductor carrier tape surface. Finally, based on the X-direction length coordinates and Y-direction length coordinates, a two-dimensional coordinate position (X, Y) describing the defect on the semiconductor carrier tape surface is constructed.

[0091] To enhance the usability of defect output information, the identified defect category (at least one of indentation, scratch, burr, and bubble) is bound to the calculated two-dimensional coordinate position, forming defect location information that includes both "defect type and spatial location attributes." This defect location information is then output in real time to the subsequent rejection mechanism or alarm system. Through this quantitative position mapping method, the defect location can be accurately tracked even under continuous movement and speed variations of the semiconductor carrier tape, ensuring the accuracy of subsequent defect processing and preventing rejection offsets or missed rejections. This improves the reliability and stability of defect tracing throughout the entire inspection process.

[0092] In another possible implementation, when the acquired semiconductor carrier surface image is blurry, lacks contrast, or is difficult to definitively determine whether an anomaly exists, a secondary acquisition and judgment operation based on roller reverse drive can be further performed to ensure the reliability of the detection process. Specifically, if, during the detection process, multiple consecutive frames of images still exhibit blurred edges or distorted texture information after preprocessing, or if the probability values ​​of various defects output by the defect recognition model are all near preset thresholds and difficult to form a clear judgment result, an abnormal judgment state with a "pending confirmation" mark is triggered. In this abnormal judgment state, the feeding roller and the receiving roller are controlled to rotate slowly in the opposite direction, driving the semiconductor carrier to retract a distance in the opposite direction to the original movement direction, so that the pending confirmation area that has passed through the detection station returns to the center of the detection station, and a second image acquisition operation is performed at a lower acquisition frame rate than the original sampling rate.

[0093] In the secondary acquisition stage, the image acquisition device adjusts the exposure time and illumination parameters to sequentially illuminate the oblique light source at different incident angles, ensuring that the re-acquired image has clearer edge information and higher texture contrast compared to the original image. Subsequently, motion compensation, image denoising, texture extraction, and other preprocessing procedures are repeatedly performed on the secondary acquired image, and classification is performed again in the defect recognition model based on the newly obtained texture description vector.

[0094] If the defect probability value output by the secondary judgment is higher than the threshold, an abnormal defect is confirmed in the corresponding area. If the secondary judgment still cannot provide a clear conclusion, the area is moved to the manual review station for further manual assistance. By introducing a "reverse backtracking-secondary acquisition-secondary identification" judgment strategy, the continuous online inspection of semiconductor carrier tape can not only effectively eliminate misjudgments caused by occasional factors such as light fluctuations, vibration trailing, and focal length drift, but also ensure a higher detection rate and identification reliability for subtle and difficult defects, enhancing the robustness and practicality of the detection method.

[0095] Figure 3 This is a system structure diagram of a semiconductor carrier tape surface quality inspection system based on machine vision, provided in one embodiment of the present invention. Figure 3 As shown, this invention provides a semiconductor carrier tape surface quality inspection system based on machine vision. The system includes: a data acquisition unit, used to drive the feeding roller and the receiving roller to rotate, so as to continuously move the semiconductor carrier tape along a set running trajectory and pass through the inspection station, and to acquire semiconductor carrier tape surface image information in real time at the inspection station; a processing unit, used to perform preprocessing on the acquired image information and extract multi-scale texture features to characterize the surface state of the semiconductor carrier tape based on the preprocessed image information; a defect identification unit, used to perform classification and identification in a defect identification model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; and an output unit, used to determine the position coordinates of the defect on the semiconductor carrier tape according to the defect identification result and the running parameters of the feeding roller and the receiving roller, and output the corresponding defect location information.

[0096] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0097] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0098] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for inspecting the surface quality of semiconductor carrier tape based on machine vision, characterized in that, The method is applied to a semiconductor carrier tape inspection device. The device includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for winding it. The feed roller and the take-up roller are connected by a transmission mechanism, which drives the semiconductor carrier tape to continuously move along a set running trajectory. An image acquisition device is installed on the running trajectory. The image acquisition device includes a multi-angle composite polarization light source positioned above the semiconductor carrier tape's running trajectory and a corresponding linear industrial camera array. Real-time acquisition of semiconductor carrier tape surface image information at the inspection station includes: while the multi-angle composite polarization light source irradiates the semiconductor carrier tape surface with oblique light at different polarization angles and emission angles, the linear industrial camera array synchronously acquires images of the semiconductor carrier tape surface according to exposure parameters matched to the semiconductor carrier tape's running speed, thereby obtaining real-time semiconductor carrier tape surface image information. The method includes: The feeding roller and the receiving roller are driven to rotate, so as to drive the semiconductor carrier tape to move continuously along the set running trajectory and pass through the detection station, and the semiconductor carrier tape surface image information is collected in real time at the detection station; The acquired image information is preprocessed, and multi-scale texture features for characterizing the surface state of the semiconductor carrier are extracted based on the preprocessed image information. Based on the texture features, classification and identification are performed in the defect identification model to determine whether there are defects on the surface of the semiconductor carrier tape. Based on the defect identification results and the operating parameters of the feeding roller and the receiving roller, the position coordinates of the defect on the semiconductor carrier tape are determined, and the corresponding defect location information is output; wherein, Based on the defect identification results and the operating parameters of the feed roller and the take-up roller, the position coordinates of the defect on the semiconductor carrier tape are determined, and the corresponding defect location information is output, including: Obtain the real-time rotational speed information of the feeding roller and the receiving roller at the moment of defect identification; A mapping model between volume diameter and frame number is constructed based on the real-time rotation speed information; Based on the mapping relationship model, the frame number where the defect is located is mapped to the longitudinal displacement length coordinate of the semiconductor carrier tape, and the two-dimensional coordinate position of the defect is determined by combining the set width of the semiconductor carrier tape, thereby generating defect location information that includes the defect type and spatial location.

2. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 1, characterized in that, Preprocessing is performed on the acquired image information, including: Motion compensation based on optical flow vectors is performed on adjacent image frames to eliminate image ghosting caused by fluctuations in the semiconductor carrier speed; The compensated image is then subjected to edge enhancement and background autoencoder filtering processes in sequence to obtain preprocessed image information.

3. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 1, characterized in that, Multi-scale texture features for characterizing the surface state of semiconductor carrier strips are extracted based on preprocessed image information, including: Multi-scale texture analysis based on Laplacian wavelet decomposition is performed on the preprocessed image information to construct texture coefficient matrices at different scales; Based on the fusion of the texture coefficient matrix at different scales with the gray-level co-occurrence matrix and Gabor filtering results, a texture description vector is formed, which serves as a multi-scale texture feature for characterizing the surface state of the semiconductor carrier.

4. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 3, characterized in that, Multi-scale texture analysis based on Laplacian wavelet decomposition is performed on the preprocessed image information to construct texture coefficient matrices at different scales, including: Each image frame is sequentially subjected to Gaussian blur and downsampling operations to construct a Laplacian pyramid structure. In each scale level of the Laplacian pyramid structure, residual signals are extracted based on the difference between the corresponding scale image and its previous resolution image to form a texture response matrix characterizing the scale response. Based on the texture response matrix at each scale, regional energy features are extracted and statistically fused with the principal direction feature vector to finally obtain a texture coefficient matrix covering different scales.

5. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 3, characterized in that, A texture description vector is formed by fusing the gray-level co-occurrence matrix and Gabor filtering results based on texture coefficient matrices at different scales, including: For each scale, a gray-level co-occurrence matrix is ​​constructed according to a preset neighborhood window for the texture coefficient matrix. Based on the gray-level co-occurrence matrix, statistical measures including energy, contrast, and entropy are extracted as the first type of texture features; Gabor filtering operations are performed on the texture coefficient matrix with different orientation and frequency parameters, and the amplitude features of the filtering response are extracted as the second type of texture features; The first type of texture features and the second type of texture features are cascaded and fused in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

6. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 5, characterized in that, Based on the texture features, classification and identification are performed in the defect recognition model to determine whether there are defects on the semiconductor carrier surface, including: The texture description vector is input into a few-sample defect recognition model built on a prototype network. The feature distance between the texture description vector and the prototype vector of each type of defect is matched and calculated to obtain the corresponding defect category output probability. When the output probability exceeds a preset threshold, it is determined that a defect of the corresponding defect category exists on the surface of the semiconductor carrier tape; wherein, Defect categories include: At least one of the following defects: indentation, scratch, burr, bump, and bubble.

7. The semiconductor carrier tape surface quality inspection method based on machine vision according to claim 6, characterized in that, The construction rules for the few-sample defect recognition model based on prototype networks are as follows: The mean vector of the texture description vector for each type of defect is calculated using the labeled samples and used as the prototype vector of the corresponding defect type. During the training phase, the Euclidean distance between the texture description vector of the labeled sample and the prototype vectors of various types is used as the loss function to optimize the model, so that samples of the same defect category are aggregated to their corresponding prototypes, and samples of different defect categories are far away from their respective prototypes, thus obtaining a small sample prototype recognition model that can complete defect classification based on feature distance.

8. A semiconductor carrier tape surface quality inspection system based on machine vision, characterized in that, The system is applied to a semiconductor carrier tape inspection device. The device includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for winding the semiconductor carrier tape. The feed roller and the take-up roller are connected by a transmission mechanism to drive the semiconductor carrier tape to continuously move along a set running trajectory. An image acquisition device is installed on the running trajectory. The system is used to perform the method described in any one of claims 1-7. The system includes: The acquisition unit is used to drive the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier tape to move continuously along the set running trajectory and pass through the detection station, and to acquire the semiconductor carrier tape surface image information in real time at the detection station. The processing unit is used to perform preprocessing on the acquired image information and extract multi-scale texture features to characterize the surface state of the semiconductor carrier based on the preprocessed image information. A defect identification unit is used to perform classification and identification based on the texture features in a defect identification model to determine whether there are defects on the surface of the semiconductor carrier tape. The output unit is used to determine the position coordinates of the defect on the semiconductor carrier tape based on the defect identification result and the operating parameters of the feeding roller and the receiving roller, and output the corresponding defect location information.

Images