Semiconductor carrier tape surface quality detection method and system based on machine vision

Abstract

The embodiment of the invention provides a semiconductor carrier tape surface quality detection method and system based on machine vision, and belongs to the technical field of machine vision detection. The method comprises the following steps: driving the surface of a to-be-detected semiconductor carrier tape wound between a feeding roller and a receiving roller to pass through a detection station, and collecting image information of the surface of the semiconductor carrier tape in real time at the detection station; extracting multi-scale texture features for representing the surface state of the semiconductor carrier tape based on the preprocessed image information; performing classification recognition in a defect recognition model based on the texture features, and judging whether the surface of the semiconductor carrier tape has defects or not; and determining position coordinates of the defect on the semiconductor carrier tape according to a defect identification result and the operation parameters of the feeding roller and the material receiving roller, and outputting corresponding defect positioning information. According to the scheme, the accuracy and the sensitivity of defect detection of the semiconductor carrier band are improved, and mapping of defect positioning in the semiconductor carrier band in actual operation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a semiconductor carrier tape surface quality detection method based on machine vision and a semiconductor carrier tape surface quality detection system based on machine vision. Background Art

[0002] During the packaging and transmission process of semiconductor devices, semiconductor carriers serve as carriers for chips, components, and other devices. Their surface quality is directly related to the subsequent packaging accuracy and the factory yield of electronic components. However, the surface quality inspection methods commonly used in the industry at this stage for semiconductor carriers are mainly random visual inspections or photo identification using single-angle industrial cameras at the end of the production line. This type of method not only has a limited detection range, but also has difficulty covering all defect categories in the production process. Surface defects such as tiny indentations, fine cracks, burrs, bubbles, and wrinkles on the film surface are often difficult to accurately identify due to factors such as light reflection, interference from the transparent film layer, and vibration of the semiconductor carrier. As a result, a large number of subtle defects are not discovered until after the tapes are rolled up and stored, thereby causing the risk of irreversible batch scrapping.

[0003] On the other hand, because semiconductor carriers run at high speeds in a roll-to-roll fashion on processing lines, existing online inspection technologies commonly suffer from technical weaknesses such as asynchrony between image frame acquisition and semiconductor carrier motion, making it impossible to accurately map defect locations. Especially when the operating speed of the semiconductor carrier fluctuates, detection delays, image smearing, missed detections, or misjudgments are prone to occur, seriously affecting the reliability of the recognition results. Furthermore, existing defect detection algorithms often rely on fixed templates or statistical thresholds, lacking adaptability and generalization capabilities for newly emerging subtle defects or defect types with fewer samples, and are unable to meet the actual needs of intelligent inspection.

[0004] In summary, the existing technology urgently needs a semiconductor carrier surface quality inspection method that can achieve high-sensitivity and multi-defect type detection during the continuous operation of the semiconductor carrier, effectively avoid the interference of mirror reflection and motion blur, and has the ability to accurately map the defect location, so as to improve the defect monitoring capability and product yield of the semiconductor carrier production process. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a method and system for detecting the surface quality of semiconductor carriers based on machine vision, so as to at least solve the problems of insensitive defect recognition and inaccurate defect position mapping in the existing online detection of semiconductor carriers.

[0006] To achieve the above-mentioned objectives, the present invention provides, in a first aspect, a method for inspecting the surface quality of a semiconductor carrier tape based on machine vision. The method is applied to a semiconductor carrier tape inspection device, wherein the device includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for rewinding the semiconductor carrier tape. The feed roller and the take-up roller drive the semiconductor carrier tape to continuously move along a set running track via a transmission mechanism. An image acquisition device is provided on the running track. The method includes: Driving the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier tape to continuously move along a set running trajectory and pass through a detection station, and collecting image information of the semiconductor carrier tape surface in real time at the detection station; performing preprocessing on the collected image information, and extracting multi-scale texture features for characterizing the surface state of the semiconductor carrier tape based on the preprocessed image information; Performing classification and recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; The position coordinates of the defect on the semiconductor carrier tape are determined according to the defect identification result and the operating parameters of the feed roller and the receiving roller, and the corresponding defect location information is output.

[0007] Optionally, the image acquisition device includes: A multi-angle composite polarized light source and a corresponding linear industrial camera array are arranged above the semiconductor carrier running track; The real-time collection of semiconductor carrier tape surface image information at the inspection station includes: While the multi-angle composite polarized light source irradiates the surface of the semiconductor carrier with oblique light at different polarization angles and luminous angles, the linear industrial camera array is used to synchronously capture images of the semiconductor carrier surface according to exposure parameters matching the operating speed of the semiconductor carrier to obtain real-time image information of the semiconductor carrier surface.

[0008] Optionally, preprocess the collected image information, including: Perform motion compensation based on optical flow vectors on adjacent image frames to eliminate image smear caused by fluctuations in the running speed of the semiconductor carrier; The compensated image is sequentially subjected to edge enhancement and background self-encoding filtering processing to obtain pre-processed image information.

[0009] Optionally, multi-scale texture features for characterizing the surface state of the semiconductor carrier tape are extracted based on the pre-processed image information, including: Perform multi-scale texture analysis based on Laplace wavelet decomposition on the pre-processed image information to construct texture coefficient matrices at different scales; The texture coefficient matrices at different scales are fused with the gray-level co-occurrence matrix and Gabor filter results to form a texture description vector, which serves as a multi-scale texture feature for characterizing the surface state of the semiconductor carrier.

[0010] Optionally, a multi-scale texture analysis based on Laplace wavelet decomposition is performed on the pre-processed image information to construct texture coefficient matrices at different scales, including: Perform Gaussian blur and downsampling operations on each image frame in sequence to construct a Laplacian pyramid structure; At each scale level of the Laplacian pyramid structure, a residual signal is extracted based on the difference between the corresponding scale image and its previous resolution image to form a texture response matrix representing the scale response; Based on the texture response matrix at each scale, the regional energy features are extracted and statistically fused with the main direction feature vectors to finally obtain the texture coefficient matrix covering different scales.

[0011] Optionally, a texture description vector is formed by fusing the gray-level co-occurrence matrix and the Gabor filter result based on the texture coefficient matrices at different scales, including: For the texture coefficient matrix at each scale, a gray-level co-occurrence matrix is ​​constructed according to the preset neighborhood window; Extracting statistics including energy, contrast and entropy as first-class texture features based on the gray-level co-occurrence matrix; Perform Gabor filtering on the texture coefficient matrix with different direction and frequency parameters, and extract the amplitude characteristics of the filter response as the second type of texture features; The first type of texture features and the second type of texture features are cascaded and fused in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

[0012] Optionally, performing classification recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape includes: Input the texture description vector into a small sample defect recognition model built based on a prototype network, and perform matching calculation on the feature distance between the texture description vector and the prototype vector of each category of defects to obtain the corresponding defect category output probability; When the output probability exceeds a preset threshold, it is determined that a defect of a corresponding defect category exists on the surface of the semiconductor carrier tape; wherein, Defect categories include: At least one defect among indentations, scratches, burrs, bumps and bubbles.

[0013] Optionally, the construction rules of the small sample defect recognition model based on the prototype network are: The mean vector of the texture description vector of each type of defect is calculated using the labeled samples as the prototype vector of the corresponding type of defect; During the training phase, the Euclidean distance between the texture description vector of the labeled sample and each type of prototype vector is used as the metric loss function to optimize the model, so that samples of the same defect category are aggregated towards their corresponding prototypes, and samples of different defect categories are far away from their respective prototypes, obtaining a small-sample prototype recognition model that can complete defect classification based on feature distance.

[0014] Optionally, determining the position coordinates of the defect on the semiconductor carrier tape according to the defect identification result and the operating parameters of the feed roller and the receiving roller, and outputting corresponding defect location information, includes: Acquiring real-time rotation speed information of the feeding roller and the receiving roller at the time of defect recognition; Building a mapping relationship model between the coil diameter and the frame number based on the real-time rotation speed information; Based on the mapping relationship model, the frame number where the defect is identified is mapped to the longitudinal displacement length coordinate of the semiconductor carrier, and the two-dimensional coordinate position of the defect is determined in combination with the set width of the semiconductor carrier to generate defect location information including the defect type and spatial position.

[0015] A second aspect of the present invention provides a semiconductor carrier surface quality inspection system based on machine vision, which is applied to semiconductor carrier inspection equipment. The equipment includes a feed roller for releasing the semiconductor carrier and a take-up roller for winding the semiconductor carrier. The feed roller and the take-up roller drive the semiconductor carrier to move continuously along a set running trajectory through a transmission mechanism, and an image acquisition device is provided on the running trajectory. The system includes: an acquisition unit, which is used to drive the feed roller and the take-up roller to rotate so as to drive the semiconductor carrier to move continuously along the set running trajectory and pass through the inspection station, and to collect semiconductor carrier surface image information in real time at the inspection station; a processing unit, which is used to perform preprocessing on the collected image information and extract multi-scale texture features for characterizing the surface state of the semiconductor carrier based on the preprocessed image information; a defect recognition unit, which is used to perform classification recognition in a defect recognition model based on the texture features to determine whether there is a defect on the surface of the semiconductor carrier; and an output unit, which is used to determine the position coordinates of the defect on the semiconductor carrier according to the defect recognition result and the operating parameters of the feed roller and the take-up roller, and output corresponding defect positioning information.

[0016] Through the above technical solution, the solution of the present invention realizes online continuous detection of semiconductor carriers by constructing a continuously moving semiconductor carrier running track between the feeding roller and the receiving roller, and collecting the surface image information of the semiconductor carrier in real time at the detection station. The introduction of pre-processing and multi-scale texture feature extraction means in image processing makes the identification of minor defects such as indentations, scratches, and bubbles more sensitive and reliable. Furthermore, by constructing a defect recognition model to classify and judge the image texture features, and mapping and locating the location of the defects in combination with the roller operation parameters, the defect position coordinate information can be accurately output. Therefore, the present invention not only improves the accuracy and sensitivity of semiconductor carrier defect detection, but also realizes the mapping of defect positioning in the actual running semiconductor carrier, providing a reliable basis for subsequent rejection and alarm control, thereby significantly improving the quality monitoring capability of the semiconductor carrier production process.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings: Figure 1 This is a flowchart of a method for detecting surface quality of a semiconductor carrier tape based on machine vision provided by one embodiment of the present invention; Figure 2 A semiconductor carrier tape detection device provided by one embodiment of the present invention; Figure 3 This is a system structure diagram of a semiconductor carrier surface quality detection system based on machine vision provided by one embodiment of the present invention.

[0019] Description of reference numerals: 10-feed roller; 20-receive roller; 30-semiconductor carrier tape; 40-image acquisition device; 401-Multi-angle composite polarization light source; 402-Linear array industrial camera. DETAILED DESCRIPTION

[0020] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting the surface quality of a semiconductor carrier tape based on machine vision, the method comprising: Step S1: driving the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier to continuously move along a set running trajectory and pass through a detection station, and collecting the surface image information of the semiconductor carrier in real time at the detection station.

[0024] Specifically, the surface quality inspection method of the semiconductor carrier 30 based on machine vision proposed in the present invention is applied to the semiconductor carrier 30 inspection equipment, such as Figure 2 The device includes a feed roller 10 for releasing the semiconductor carrier and a receiving roller 20 for winding the semiconductor carrier. The feed roller 10 and the receiving roller 20 drive the semiconductor carrier to move continuously along a set running track through a transmission mechanism, and an image acquisition device 40 is provided on the running track.

[0025] The feed roller 10 unwinds the initial semiconductor carrier tape, while the take-up roller 20 rewinds the inspected semiconductor carrier tape. A tensioning drive assembly maintains constant tension between the two, allowing the tape to pass through the inspection station. An image acquisition device 40 is fixedly mounted above the semiconductor carrier tape's trajectory and includes a light source assembly and an image acquisition assembly. The light source assembly utilizes a polarizer to achieve illumination at a specific polarization angle, thereby reducing interference from specular reflections on the surface of the semiconductor carrier tape 30.

[0026] The image acquisition component is preferably an array of linear industrial cameras 402, which are used to continuously capture a sequence of image frames in the direction of motion of the semiconductor carrier. Furthermore, the image acquisition device 40 is communicatively connected to the transmission mechanism via a trigger unit. The trigger unit adjusts the exposure duration and sampling frequency of the image acquisition according to the actual operating speed of the semiconductor carrier to ensure that the surface image of the semiconductor carrier is clear and recognizable under high-speed operation. By arranging the above-mentioned image acquisition device 40 on the operating track of the semiconductor carrier, continuous and real-time image sampling of the surface of the semiconductor carrier 30 can be performed without affecting the transmission process of the semiconductor carrier, providing the raw data basis for subsequent defect identification and positioning, thereby achieving the purpose of online non-contact detection.

[0027] Preferably, the image acquisition device 40 includes: a multi-angle composite polarized light source 401 and a corresponding linear array industrial camera 402 array arranged above the running track of the semiconductor carrier 30; real-time acquisition of surface image information of the semiconductor carrier 30 at the inspection station includes: while the multi-angle composite polarized light source 401 irradiates the surface of the semiconductor carrier 30 with oblique light at different polarization angles and luminous angles, the linear array industrial camera 402 array is used to synchronously acquire the surface image of the semiconductor carrier according to exposure parameters matching the running speed of the semiconductor carrier 30, thereby obtaining real-time surface image information of the semiconductor carrier 30.

[0028] In an embodiment of the present invention, the multi-angle composite polarized light source 401 includes at least two strip-shaped high-brightness light sources set at different pitch angles, each light source is equipped with an electrically controlled polarizer, and the polarization direction of the irradiated light can be dynamically controlled by adjusting the polarization angle of the polarizer; the oblique light emitted by the light source is irradiated onto the surface of the semiconductor carrier 30 at different incident angles, so that surface scratches, micro-pits, bubbles and thin film defects produce different strong reflection areas under light illumination from different directions.

[0029] The array of linear industrial cameras 402 is positioned above the reflection direction of the multi-angle composite polarized light source 401. Each camera has a different viewing angle corresponding to different incident angles, and its frame rate is adjustable. During the acquisition process, the current operating speed of the semiconductor carrier 30 is first acquired in real time by the transmission mechanism. This speed is input as a parameter into the exposure control program to determine the exposure time, frame interval, and trigger time for each frame sampling of the linear industrial cameras 402. Synchronous triggering of multiple cameras is then achieved, ensuring that the acquired image frame sequence is consistent with the actual displacement of the semiconductor carrier surface, thereby avoiding image smear and distortion under high-speed transmission conditions.

[0030] Synchronously, the multi-angle composite polarized light source 401 is controlled to scan the surface of the semiconductor carrier in turn in a light source sequence. Each time a set of polarization angles and incident angles is switched, the corresponding industrial camera is triggered to capture images under the lighting conditions, thereby obtaining a surface image of the multi-channel semiconductor carrier 30 under different illumination angles and different polarization angles.

[0031] Furthermore, each camera-captured image is individually numbered and labeled with exposure parameters, enabling subsequent integration with the roll-diameter coordinate mapping model for multi-channel image fusion and defect feature enhancement. By employing this composite polarized oblique illumination combined with real-time acquisition from a linear industrial camera array, the sensitivity of subtle surface defects on semiconductor carriers can be significantly improved, interference from specular reflections can be suppressed, and clear and reliable surface images can be obtained at varying motion speeds. This provides a stable and sufficient image data foundation for subsequent high-precision defect identification.

[0032] Step S2: performing preprocessing on the collected image information, and extracting multi-scale texture features for characterizing the surface state of the semiconductor carrier tape based on the preprocessed image information.

[0033] Specifically, motion compensation operations based on optical flow vectors are performed on adjacent image frames to eliminate image smear caused by fluctuations in the running speed of the semiconductor carrier; edge enhancement and background self-encoding filtering are performed on the compensated images in sequence to obtain pre-processed image information.

[0034] Furthermore, multi-scale texture features for characterizing the surface state of the semiconductor carrier are extracted based on the preprocessed image information, including: performing multi-scale texture analysis based on Laplace wavelet decomposition on the preprocessed image information to construct texture coefficient matrices at different scales; based on the texture coefficient matrices at different scales, the grayscale co-occurrence matrix and the Gabor filtering results are fused to form a texture description vector as a multi-scale texture feature for characterizing the surface state of the semiconductor carrier.

[0035] In an embodiment of the present invention, in the process of the semiconductor carrier continuously moving through the inspection station in a roll-to-roll manner, there are inevitably factors such as transmission speed fluctuations, instantaneous acceleration and deceleration, and tension changes. These factors cause displacement deviations in the corresponding image content between different frame images, thereby forming a ghosting or blurring phenomenon in the subsequent defect detection stage.

[0036] To address this problem, motion compensation based on optical flow vectors is first performed on adjacent image frames. By estimating the optical flow vector distribution of a specific set of pixel blocks on the semiconductor carrier surface in consecutive frames, the global displacement or local vector field of the image is calculated. This allows inverse displacement compensation to be performed on the subsequent frame, maintaining visual spatial consistency between the two frames. After motion compensation, edge enhancement is performed using a method based on the Sobel operator and morphological gradient enhancement. This method enhances the contrast of subtle texture structures such as scratches and pits by highlighting regions with rapid pixel grayscale changes. Furthermore, an autoencoder is used to construct a baseline background model. An autoencoder reconstruction network is trained based on a large number of defect-free images, enabling it to reconstruct the normal surface texture of the semiconductor carrier. The residual between the original image and the reconstruction result is then used as the background filtering result to obtain preprocessed image information, suppressing interference from irrelevant features such as gloss changes on the semiconductor carrier surface, film folds, and dust particles.

[0037] Specifically, a multi-scale texture analysis based on Laplace wavelet decomposition is performed on the preprocessed image information to construct texture coefficient matrices at different scales, including: performing Gaussian blur and downsampling operations on each image frame in sequence to construct a Laplace pyramid structure; in each scale level of the Laplace pyramid structure, a residual signal is extracted based on the difference between the corresponding scale image and its previous resolution image to form a texture response matrix that represents the scale response; based on the texture response matrix at each scale, regional energy features and main direction eigenvectors are extracted and statistically fused to finally obtain a texture coefficient matrix covering different scales.

[0038] Furthermore, a texture description vector is formed based on the fusion of grayscale co-occurrence matrix and Gabor filtering results based on texture coefficient matrices at different scales, including: constructing a grayscale co-occurrence matrix according to a preset neighborhood window for the texture coefficient matrix at each scale; extracting statistics including energy, contrast and entropy as the first type of texture features based on the grayscale co-occurrence matrix; performing Gabor filtering operations on the texture coefficient matrix with different directions and frequency parameters, and extracting the amplitude characteristics of the filtering response as the second type of texture features; and cascading and fusing the first type of texture features with the second type of texture features in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

[0039] In an embodiment of the present invention, in order to accurately reflect the texture response of defects such as indentations, fine scratches, local undulations, and punching burrs on the surface of a semiconductor carrier at different spatial scales, Gaussian blurring and fixed-multiple downsampling operations are sequentially performed on each frame of the image after motion compensation and background filtering to construct a Laplacian pyramid structure. Gaussian blurring is used to retain the low-frequency structural information of the image, and downsampling is used to gradually reduce the resolution to form a pyramidal image sequence containing multiple resolution levels.

[0040] At each scale level of the Laplacian pyramid structure, the corresponding low-resolution image is first upsampled to the resolution of the previous scale and then differentially computed with the original image at the previous scale to obtain a residual signal matrix reflecting the high-frequency texture details of the image at that scale. This residual signal matrix is ​​used to construct a texture response matrix representing the scale response, thereby preserving sensitivity to texture changes at different scales. For each scale, the texture response matrix is ​​then used to extract regional energy features and principal direction eigenvectors using the local energy accumulation rule and the gradient principal direction statistics rule, respectively. These eigenvectors are then fused to obtain an initial texture feature vector representing the surface roughness, directionality, and local texture distribution of the semiconductor tape.

[0041] This process is repeated to fully capture the variations in texture energy and directionality from the finest texture scale to the largest scales, ultimately forming a texture coefficient matrix covering multiple scale levels. This multi-scale Laplacian pyramid analysis method comprehensively captures the hierarchical distribution of semiconductor tape surface textures, improving robust recognition of defects of varying morphologies.

[0042] Furthermore, the gray-level co-occurrence matrix and Gabor filter results are fused based on the texture coefficient matrices at different scales to form the final texture description vector used as input in the defect recognition stage.

[0043] Specifically, a gray-level co-occurrence matrix is ​​constructed for the texture coefficient matrix at each scale according to a preset neighborhood window. By statistically analyzing the common distribution probability of pixel grayscale pairs in the gray-level co-occurrence matrix in the horizontal direction, vertical direction and other representative angles, statistics such as energy, contrast, entropy and correlation are calculated respectively, and used as the first-class texture features to describe the coarseness and structural complexity of the local texture on the surface; at the same time, a set of Gabor kernel functions with different directions (such as 0°, 45°, 90° and 135°) and different frequency parameters are used to perform Gabor filtering operations on the texture coefficient matrix, and a filter response map with direction selectivity is obtained through convolution operation. The amplitude mean, amplitude variance, maximum response position, etc. are extracted from the filter response map as the second-class texture features to reflect the repeatability, periodicity and detail fluctuations of the texture in different directions on the surface of the semiconductor carrier.

[0044] Furthermore, the first-class texture features and second-class texture features obtained at each scale are cascaded and combined one by one according to the order of scale levels in the texture coefficient matrix from low frequency to high frequency, to construct a multi-scale texture description vector. This texture description vector comprehensively integrates the statistical characteristics and directional response information of textures at various scales, multiple directions, and multiple frequencies. It has high-dimensional and multi-angle texture expression capabilities, can fully characterize the true texture state of the semiconductor carrier surface, and provide a more accurate and stable input foundation for subsequent subtle defect detection and category recognition, significantly improving detection sensitivity and accuracy. In particular, it can still maintain good feature differentiation effect under complex backgrounds, light disturbances, and interference from transparent film structures.

[0045] Step S3: performing classification recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape.

[0046] Specifically, the texture description vector is input into a small sample defect recognition model constructed based on a prototype network, and the characteristic distance between the texture description vector and the prototype vector of each category of defects is matched and calculated to obtain the corresponding defect category output probability; when the output probability exceeds a preset threshold, it is determined that a defect of the corresponding defect category exists on the surface of the semiconductor carrier; wherein the defect categories include: at least one defect among indentations, scratches, burrs, bumps and bubbles.

[0047] Specifically, the construction rules of the small-sample defect recognition model based on the prototype network are as follows: the mean vector of the texture description vector of each type of defect is calculated using the labeled samples as the prototype vector of the corresponding type of defect; in the training stage, the Euclidean distance between the texture description vector of the labeled samples and each type of prototype vector is used as the metric loss function to optimize the model, so that samples of the same defect category are aggregated to their corresponding prototypes, and samples of different defect categories are far away from their respective prototypes, thereby obtaining a small-sample prototype recognition model that can complete defect classification and judgment based on feature distance.

[0048] In an embodiment of the present invention, the texture description vector is input into a small sample defect recognition model constructed based on a prototype network, and a matching calculation is performed on the characteristic distance between the texture description vector and the prototype vector of each category of defects to obtain the corresponding defect category output probability; when the output probability exceeds a preset threshold, it is determined that a defect of the corresponding defect category exists on the surface of the semiconductor carrier; wherein the defect category includes at least one defect selected from indentations, scratches, burrs, bumps and bubbles.

[0049] In order to achieve robust and accurate classification and recognition even when the number of defect samples is limited, the model is constructed using a prototype network structure. Specifically, the construction rules of the small sample defect recognition model based on the prototype network are as follows: the texture description vector of each type of defect is calculated using labeled defect samples, and the texture description vectors of each type of sample are averaged as the prototype vector corresponding to that type of defect; during the model training phase, the Euclidean distance is calculated between the texture description vector of each defect sample and the prototype vectors of multiple defect categories, and the Euclidean distance is used as the core indicator of the metric loss function to optimize and adjust the network, so that samples of the same category gradually approach the corresponding prototype vector in the feature space, while the distance between samples of different categories gradually increases. After the training is completed, a small sample prototype recognition model is formed that can complete defect classification based on the distance relationship between the texture description vector and the prototype vector. This model can complete effective learning with only a small number of defect training samples.

[0050] In order to further improve the stability and generalization of the model under actual semiconductor carrier production conditions, the prototype vector update logic adopts a training and updating method to iteratively correct the original prototype vector of each category. That is, during the training process, if the distance deviation between a new sample and an existing prototype is found to be lower than the dynamic threshold, the texture description vector of the sample is added to the prototype mean update according to the weight, so that the constructed defect prototypes of various types have higher representativeness and adaptability.

[0051] In the recognition stage, the texture description vector of the image to be detected is first calculated, and then the Euclidean distance with the prototype vector of each defect category is calculated respectively. The distance value is normalized by the softmax function to obtain the output probability distribution of each category; if the probability value corresponding to a certain defect category exceeds the preset threshold, the recognition result of the defect of this category is output; if the output probability of each category does not reach the threshold, it is determined that there is no significant defect on the surface of the semiconductor carrier at the current detection position.

[0052] Through the above method, the surface defect identification task can still be stably completed in a production environment where there are very few defect samples or new defects continue to appear, significantly improving the detection accuracy and practical application reliability. The model parameters can also be quickly adjusted according to the texture feature differences of different semiconductor carrier materials for adaptation to different semiconductor carrier products.

[0053] Step S4: determining the position coordinates of the defect on the semiconductor carrier tape according to the defect identification result and the operating parameters of the feed roller and the receiving roller, and outputting corresponding defect location information.

[0054] Specifically, the real-time rotational speed information of the feed roller and the receiving roller at the moment of defect identification is obtained; a mapping relationship model between the roll diameter and the frame number is constructed based on the real-time rotational speed information; the frame number where the identified defect is located is mapped to the longitudinal displacement length coordinate of the semiconductor carrier based on the mapping relationship model, and the two-dimensional coordinate position of the defect is determined in combination with the set width of the semiconductor carrier, and defect location information including the defect type and spatial position is generated.

[0055] In an embodiment of the present invention, the speed of the feed roller and the speed of the receiving roller at the time of defect recognition are obtained in rpm, and the sampling trigger time of the corresponding image frame is recorded respectively; at the same time, the image frame number of the industrial camera is timestamped and aligned with the number of pulses output by the displacement encoding device of the feed roller and the receiving roller to establish a correlation basis between the image acquisition frame number and the actual running distance of the semiconductor carrier.

[0056] Subsequently, a mathematical mapping model between roll diameter growth and rotational speed was established based on the fixed tension parameter between the feed and take-up rollers during roll-to-roll processing of the semiconductor carrier tape. The instantaneous linear velocity of the semiconductor carrier tape at the moment of identification was calculated by combining the real-time rotational speeds of the feed and take-up rollers. The image frame number and this linear velocity were input into the mapping model to obtain the cumulative travel distance from the image sampling start frame to the defect identification frame. This allows the defect identification frame number to be accurately mapped to the longitudinal (X-direction) displacement coordinate of the semiconductor carrier tape.

[0057] Furthermore, based on the known width of the semiconductor tape, the camera's imaging area maps the horizontal pixel position of the defect in the image to the horizontal length coordinate of the actual semiconductor tape surface, using the correspondence between the fixed pixel coordinates in the camera's imaging area in the lateral (Y) direction of the semiconductor tape and the imaging calibration parameters. Finally, based on the X- and Y-direction length coordinates, the two-dimensional coordinate position (X, Y) of the defect on the semiconductor tape surface is constructed.

[0058] To increase the usability of defect output information, the identified defect category (at least one of indentations, scratches, burrs, and bubbles) is bound to the calculated two-dimensional coordinate position to form defect location information containing the dual attributes of "defect type and spatial location." This defect location information is then output in real time to the subsequent rejection mechanism or alarm process. This quantitative position mapping method allows for precise tracking of defect locations even under conditions of continuous motion and speed variations of the semiconductor carrier, ensuring the accuracy of subsequent defect processing and avoiding rejection offsets or missed rejections, thereby improving the reliability and stability of defect tracing throughout the entire inspection process.

[0059] In another possible embodiment, when the collected surface image of the semiconductor carrier is blurred, lacks contrast, or it is difficult to clearly determine whether there is an abnormality, a secondary acquisition and judgment operation based on the reverse drive of the roller can be further performed to ensure the reliability of the detection process. Specifically, during the detection process, if multiple consecutive frames of images still have blurred edges and distorted texture information after pre-processing, or if the probability values ​​of various defects output by the defect recognition model are all near the preset threshold and it is difficult to form a clear judgment result, an abnormal judgment state with a "pending confirmation" mark is triggered. In this abnormal judgment state, the feed roller and the receiving roller are controlled to rotate slowly in the opposite direction synchronously, driving the semiconductor carrier to retreat a distance in the direction opposite to the original movement direction, so that the area to be confirmed that has passed the detection station re-runs back to the center of the detection station, and at the same time, a second image acquisition operation is performed at an acquisition frame rate lower than the original sampling rate.

[0060] During the secondary acquisition phase, the image acquisition device adjusts the exposure time and lighting parameters, sequentially illuminating the oblique light source at different angles of incidence. This ensures that the recaptured image has clearer edge information and higher texture contrast than the original image. Preprocessing processes such as motion compensation, image denoising, and texture extraction are then repeated on the secondary acquired image. Classification is then performed again in the defect recognition model based on the recovered texture description vector.

[0061] If the defect probability value output by the secondary judgment result is higher than the threshold, the corresponding area is confirmed to have an abnormal defect. If the secondary judgment still cannot reach a clear conclusion, the area is moved back to the manual review station for manual assisted judgment. By introducing the "reverse retreat-secondary collection-secondary recognition" judgment strategy, during the continuous online inspection of semiconductor carriers, not only can misjudgments caused by occasional factors such as lighting fluctuations, vibration smear, and focus drift be effectively eliminated, but it can also ensure a higher detection rate and recognition reliability for subtle and difficult defects, enhancing the robustness and practicality of the detection method.

[0062] Figure 3 This is a system structure diagram of a semiconductor carrier surface quality detection system based on machine vision provided by an embodiment of the present invention. Figure 3As shown, an embodiment of the present invention provides a semiconductor carrier surface quality inspection system based on machine vision, and the system includes: an acquisition unit, used to drive the feed roller and the receiving roller to rotate, so as to drive the semiconductor carrier to continuously move along a set running trajectory and pass through the inspection station, and to collect semiconductor carrier surface image information in real time at the inspection station; a processing unit, used to perform preprocessing on the collected image information, and extract multi-scale texture features for characterizing the surface state of the semiconductor carrier based on the preprocessed image information; a defect recognition unit, used to perform classification recognition in a defect recognition model based on the texture features, and determine whether there are defects on the surface of the semiconductor carrier; an output unit, used to determine the position coordinates of the defect on the semiconductor carrier according to the defect recognition result and the operating parameters of the feed roller and the receiving roller, and output corresponding defect positioning information.

[0063] Those skilled in the art will appreciate that all or part of the steps in the methods described in the aforementioned embodiments can be performed by instructing the relevant hardware through a program. The program, stored in a storage medium, includes instructions for causing a microcontroller, chip, or processor to execute all or part of the steps in the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] The above describes in detail the optional embodiments of the present invention in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, a variety of simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the scope of protection of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will no longer describe the various possible combinations separately.

[0065] In addition, the various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A method for detecting the surface quality of a semiconductor carrier tape based on machine vision, characterized in that: The method is applied to a semiconductor carrier tape inspection device, the device comprising a feed roller for releasing the semiconductor carrier tape and a take-up roller for rewinding the semiconductor carrier tape, wherein a transmission mechanism is provided between the feed roller and the take-up roller to drive the semiconductor carrier tape to continuously move along a set running track, and an image acquisition device is provided on the running track. The method comprises: Driving the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier tape to continuously move along a set running trajectory and pass through a detection station, and collecting image information of the semiconductor carrier tape surface in real time at the detection station; performing preprocessing on the collected image information, and extracting multi-scale texture features for characterizing the surface state of the semiconductor carrier tape based on the preprocessed image information; Performing classification and recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; The position coordinates of the defect on the semiconductor carrier tape are determined according to the defect identification result and the operating parameters of the feed roller and the receiving roller, and the corresponding defect location information is output.

2. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 1, characterized in that: The image acquisition device comprises: A multi-angle composite polarized light source and a corresponding linear industrial camera array are arranged above the semiconductor carrier running track; The real-time collection of semiconductor carrier tape surface image information at the inspection station includes: While the multi-angle composite polarized light source irradiates the surface of the semiconductor carrier with oblique light at different polarization angles and luminous angles, the linear industrial camera array is used to synchronously capture images of the semiconductor carrier surface according to exposure parameters matching the operating speed of the semiconductor carrier to obtain real-time image information of the semiconductor carrier surface.

3. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 1, characterized in that: Perform pre-processing on the acquired image information, including: Perform motion compensation based on optical flow vectors on adjacent image frames to eliminate image smear caused by fluctuations in the running speed of the semiconductor carrier; The compensated image is sequentially subjected to edge enhancement and background self-encoding filtering processing to obtain pre-processed image information.

4. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 1, characterized in that: Based on the pre-processed image information, multi-scale texture features are extracted to characterize the surface state of the semiconductor carrier, including: Perform multi-scale texture analysis based on Laplace wavelet decomposition on the pre-processed image information to construct texture coefficient matrices at different scales; The texture coefficient matrices at different scales are fused with the gray-level co-occurrence matrix and Gabor filter results to form a texture description vector, which serves as a multi-scale texture feature for characterizing the surface state of the semiconductor carrier.

5. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 4, characterized in that: Perform multi-scale texture analysis based on Laplace wavelet decomposition on the pre-processed image information to construct texture coefficient matrices at different scales, including: Perform Gaussian blur and downsampling operations on each image frame in sequence to construct a Laplacian pyramid structure; At each scale level of the Laplacian pyramid structure, a residual signal is extracted based on the difference between the corresponding scale image and its previous resolution image to form a texture response matrix representing the scale response; Based on the texture response matrix at each scale, the regional energy features are extracted and statistically fused with the main direction feature vectors to finally obtain the texture coefficient matrix covering different scales.

6. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 4, characterized in that: Based on the texture coefficient matrix at different scales, the gray-level co-occurrence matrix and Gabor filter results are fused to form a texture description vector, including: For the texture coefficient matrix at each scale, a gray-level co-occurrence matrix is ​​constructed according to the preset neighborhood window; Extracting statistics including energy, contrast and entropy as first-class texture features based on the gray-level co-occurrence matrix; Perform Gabor filtering on the texture coefficient matrix with different direction and frequency parameters, and extract the amplitude characteristics of the filter response as the second type of texture features; The first type of texture features and the second type of texture features are cascaded and fused in scale order to form a texture description vector for characterizing the surface state of the semiconductor carrier.

7. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 1, characterized in that: Performing classification and recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape includes: Input the texture description vector into a small sample defect recognition model built based on a prototype network, and perform matching calculation on the feature distance between the texture description vector and the prototype vector of each category of defects to obtain the corresponding defect category output probability; When the output probability exceeds a preset threshold, it is determined that a defect of a corresponding defect category exists on the surface of the semiconductor carrier tape; wherein, Defect categories include: At least one defect among indentations, scratches, burrs, bumps and bubbles.

8. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 7, characterized in that: The construction rules of the small sample defect recognition model based on the prototype network are: The mean vector of the texture description vector of each type of defect is calculated using the labeled samples as the prototype vector of the corresponding type of defect; During the training phase, the Euclidean distance between the texture description vector of the labeled sample and each type of prototype vector is used as the metric loss function to optimize the model, so that samples of the same defect category are aggregated towards their corresponding prototypes, and samples of different defect categories are far away from their respective prototypes, obtaining a small-sample prototype recognition model that can complete defect classification based on feature distance.

9. The method for detecting surface quality of semiconductor carrier tape based on machine vision according to claim 1, characterized in that: Determine the position coordinates of the defect on the semiconductor carrier tape based on the defect identification result and the operating parameters of the feed roller and the receiving roller, and output corresponding defect location information, including: Acquiring real-time rotation speed information of the feeding roller and the receiving roller at the time of defect recognition; Building a mapping relationship model between the coil diameter and the frame number based on the real-time rotation speed information; Based on the mapping relationship model, the frame number where the defect is identified is mapped to the longitudinal displacement length coordinate of the semiconductor carrier, and the two-dimensional coordinate position of the defect is determined in combination with the set width of the semiconductor carrier to generate defect location information including the defect type and spatial position.

10. A semiconductor carrier surface quality detection system based on machine vision, characterized in that: The system is applied to semiconductor carrier tape inspection equipment. The equipment includes a feed roller for releasing the semiconductor carrier tape and a take-up roller for rewinding the semiconductor carrier tape. The feed roller and the take-up roller drive the semiconductor carrier tape to continuously move along a set running track via a transmission mechanism. An image acquisition device is provided on the running track. The system includes: A collection unit is used to drive the feeding roller and the receiving roller to rotate, so as to drive the semiconductor carrier tape to continuously move along a set running trajectory and pass through the inspection station, and to collect image information of the surface of the semiconductor carrier tape in real time at the inspection station; a processing unit, configured to preprocess the collected image information and extract multi-scale texture features for characterizing the surface state of the semiconductor carrier tape based on the preprocessed image information; a defect recognition unit, configured to perform classification recognition in a defect recognition model based on the texture features to determine whether there are defects on the surface of the semiconductor carrier tape; The output unit is used to determine the position coordinates of the defect on the semiconductor carrier tape according to the defect identification result and the operating parameters of the feed roller and the receiving roller, and output corresponding defect location information.

Images