Chip packaging positioning method, system device and storage medium

Abstract

The invention provides a chip packaging positioning method, a system device and a storage medium, which are applied to the technical field of semiconductor packaging. The method comprises the following steps: acquiring a physical image of a chip, extracting outline and pin information of the chip, and obtaining an initial position of the chip; matching the initial position with the substrate template, inputting a convolutional neural network model when a threshold value is not met, and outputting a compensation adjustment vector; generating a driving instruction to adjust the position angle of the chip, feeding back and cyclically judging the positioning precision, and performing iterative adjustment; according to the scheme, the precision and robustness of chip positioning can be improved, and the sensitivity to ambient light and chip surface defects is reduced.

Description

Technical Field

[0001] This invention relates to the field of semiconductor manufacturing technology, and in particular to a chip packaging positioning method, system device, and storage medium. Background Technology

[0002] Chip packaging is a crucial step in semiconductor manufacturing that ensures the electrical connectivity and physical protection of chips. Its task is to precisely mount the chip onto a designated location on a substrate or lead frame. With increasing semiconductor integration, chip positioning accuracy requirements have reached the micrometer level; any deviation can lead to subsequent process failures, affecting product yield and reliability.

[0003] Current mainstream chip packaging positioning technologies include mechanical positioning and visual positioning. Mechanical positioning achieves positioning through precision mechanical structures, but its flexibility and accuracy are limited, and it is prone to damaging the chip. Visual positioning utilizes industrial cameras and image processing algorithms. For example, Normalization Cross Correlation (NCC) compares real-time images with templates, or edge detection operators such as Canny and Sobel extract edges, and then fits the contour using Hough Transform. These methods perform well under ideal conditions.

[0004] However, in actual production, factors such as lighting fluctuations, equipment vibrations, and chip surface defects lead to a decline in image quality. Traditional vision algorithms have poor robustness to complex environments, resulting in a significant reduction in accuracy. The combined translational and rotational deviations generated during chip transport and pickup are difficult to compensate for accurately using simple linear models. Furthermore, lens distortion and thermal expansion and contraction introduce nonlinear deviations that are difficult for traditional geometric algorithms to handle. Chips contain various features that can be used for positioning; when a feature is unreliable, traditional systems lack intelligent feature fusion mechanisms. To improve accuracy, advanced algorithms increase computational complexity, making it difficult to meet the requirements of high-speed production cycles. Therefore, there is an urgent need in this field for a technical solution to achieve fast, stable, and high-precision chip positioning in complex environments.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a positioning method, system device and storage medium for chip packaging, aiming to solve the problems of poor robustness, difficulty in handling compound nonlinear deviations and inability to balance speed and accuracy of traditional visual positioning methods in the complex and dynamic industrial environment of chip packaging, such as uneven illumination, background interference and physical defects on the chip surface. The present invention provides a technical solution that can achieve fast, high-precision and high-stability automated chip positioning.

[0007] This invention provides a positioning method for chip packaging, comprising: Acquire physical images containing the chip; The physical image is processed using the first image processing algorithm to extract the chip's outline and pin arrangement information, thereby obtaining the chip's preliminary position coordinates and angle values. The initial position coordinates and angle values ​​are matched and compared with the preset substrate standard template. When the similarity of the match is lower than the preset threshold, the physical image is input into the pre-trained convolutional neural network model. The convolutional neural network model analyzes the symbols and background interference factors in the physical image to output a compensation adjustment vector that characterizes the nonlinear deviation of the chip. Drive instructions are generated based on the compensation adjustment vector to drive the precision actuator to adjust the physical position and physical angle of the chip; Through a feedback loop, the adjusted chip position coordinates and angle values ​​are obtained from the adjusted physical image, and it is determined whether the positioning accuracy meets the preset accuracy standard to decide whether to iteratively adjust the chip.

[0008] In some optional embodiments, processing the physical image using the first image processing algorithm includes: converting the physical image into a grayscale image and smoothing the grayscale image using a Gaussian filter to reduce noise.

[0009] In some optional embodiments, processing the physical image using the first image processing algorithm further includes: processing the smoothed grayscale image using the Canny edge detection algorithm to extract strong edges and obtain contour lines.

[0010] In some optional embodiments, processing the physical image using the first image processing algorithm further includes: performing line detection on strong edges using Hough transform to obtain pin arrangement information.

[0011] In some alternative embodiments, the method further includes, before inputting the physical image into the convolutional neural network model: Based on the contour lines and pin arrangement information, multiple feature fusion processing is performed on the two to determine the nonlinear deviation parameters of the chip. The convolutional neural network model is then used for analysis based on the nonlinear deviation parameters.

[0012] In some optional embodiments, the multi-feature fusion process includes: Technical quality scores were given for the imaging quality of the contour lines and pin arrangement information, respectively. Based on the technical quality score, a fusion weight is dynamically assigned to the contour lines and pin arrangement information, with information with a higher quality score being assigned a higher weight. Weighting is performed based on fusion weights to determine the nonlinear deviation parameters.

[0013] In some alternative embodiments, the convolutional neural network model is a semantic segmentation network that includes an encoder-decoder structure and an attention mechanism module.

[0014] In some optional embodiments, the compensation adjustment vector output by the convolutional neural network model is obtained through the following steps: First, the convolutional neural network model outputs a deviation contribution heatmap, which shows the direction and magnitude of the contribution of each pixel region in the physical image to the overall physical deviation. Then, by calculating the vector integral or weighted centroid of the deviation contribution heatmap, the compensation adjustment vector is analytically generated.

[0015] In some alternative embodiments, background interference factors include physical defects on the chip surface or optical interference during the imaging process; the convolutional neural network model analyzes the physical image including: Identify abnormal interference areas in physical images; Based on the type, shape, and location of the abnormal interference area, the positioning deviation caused by the abnormal interference area is inferred, so as to generate a compensation adjustment vector.

[0016] In some optional embodiments, the nonlinear deviation is a composite deviation that includes rotational and translational deviations; the convolutional neural network model analyzes the physical image by including: The composite bias is decoupled to output independent translation compensation components and independent rotation compensation components in parallel, and a compensation adjustment vector is constructed based on the translation compensation components and rotation compensation components.

[0017] In some alternative embodiments, iterative adjustments via a feedback loop include: If it is determined that the positioning accuracy does not meet the preset accuracy standard, the adjusted physical image and the remaining deviation information will be used as the input for a new round of iteration, and the analysis and subsequent adjustment steps of the convolutional neural network model will be repeated.

[0018] In some optional embodiments, the iterative adjustment is set with a maximum number of iterations. When the number of adjustments is reached, the iterative adjustment is terminated even if the positioning accuracy still does not meet the preset accuracy standard.

[0019] In some optional embodiments, the method further includes: After the positioning operation is completed, if the positioning accuracy meets the preset accuracy standard, the physical image used in this operation and the final output compensation adjustment vector are stored as a data pair.

[0020] In some optional embodiments, the method further includes: When the number of stored data pairs reaches a preset data volume threshold, the online optimization process is automatically triggered. In the online optimization process, the stored data pairs are used to incrementally train or fine-tune the convolutional neural network model to generate an updated model for subsequent localization operations.

[0021] In some optional embodiments, the substrate standard template includes the ideal position coordinates of the chip on the substrate, the ideal angle value, the contour template, and the pin arrangement template; the matching comparison includes calculating the deviation between the preliminary position coordinates and the angle value and the ideal position coordinates and the ideal angle value, and calculating the normalized cross-correlation coefficient between the contour lines and the pin arrangement information and the contour template and the pin arrangement template.

[0022] This invention provides a positioning system device for chip packaging, comprising: The image acquisition module is configured to acquire physical images containing the chip; The preliminary positioning processing module is configured to process the physical image using a first image processing algorithm to extract the chip's outline lines and pin arrangement information, and obtain the chip's preliminary position coordinates and angle values. The deviation analysis and processing module is configured to match and compare the initial position coordinates and angle values ​​with the preset substrate standard template. When the similarity of the match is lower than the preset threshold, the module analyzes the symbols and background interference factors in the physical image through a pre-trained convolutional neural network model to output a compensation adjustment vector that characterizes the nonlinear deviation of the chip. The execution control module is configured to generate drive instructions based on the compensation adjustment vector to drive the precision actuator to adjust the physical position and physical angle of the chip; The feedback iteration control module is configured to obtain the adjusted chip position coordinates and angle values ​​from the adjusted physical image through a feedback loop, and determine whether the positioning accuracy meets the preset accuracy standard in order to decide whether to perform iterative adjustment.

[0023] In some optional embodiments, the deviation analysis processing module is further configured as follows: When surface physical defects or optical interference exist in a physical image, abnormal interference areas can be identified. Based on the type and location of the abnormal interference area, the positioning deviation caused by the abnormal interference area is inferred, so as to generate a compensation adjustment vector.

[0024] In some optional embodiments, the deviation analysis processing module is further configured as follows: When the nonlinear deviation is a composite deviation including rotational and translational deviations, decoupling analysis is performed on the composite deviation to output independent translational compensation components and independent rotational compensation components in parallel, and a compensation adjustment vector is constructed based on the translational compensation components and rotational compensation components.

[0025] In some optional embodiments, it also includes: The online model optimization module is configured to collect and store data pairs consisting of physical images and the final determined compensation adjustment vectors after multiple positioning operations. When the number of stored data pairs reaches a preset threshold, the module uses the data pairs to incrementally train the convolutional neural network model to generate an updated model for use by the deviation analysis and processing module.

[0026] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described above.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention.

[0028] The chip packaging positioning method, system device, and storage medium of the present invention have the following beneficial effects: Through the aforementioned iterative adjustments, this invention can continuously reduce the deviation between the chip pose and the ideal state, thereby stably improving the chip positioning accuracy to the micrometer level. Even under the influence of interference factors such as uneven illumination and defects on the chip surface, this method can accurately identify and compensate for various nonlinear deviations through the deep analysis capabilities of convolutional neural networks, significantly improving the robustness of positioning. Furthermore, while ensuring accuracy, this method can effectively shorten the overall processing time, meeting the demands of high-paced industrial production. Attached Figure Description

[0029] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart of a chip packaging positioning method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the positioning system for chip packaging according to an embodiment of the present invention. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.

[0034] Chip packaging and positioning requires precise control of the chip's pose, involving multiple disciplines such as image processing, pattern recognition, and servo control. The basic process is as follows: First, an image acquisition system is used to obtain an image of the chip. Then, image processing algorithms are used to extract chip features, such as edges, corners, or specific markings. These features are then used to calculate the chip's initial position and angle. Due to variations in lighting, noise interference, and chip surface defects in real-world industrial environments, the extracted features may contain deviations, leading to inaccurate initial positioning results. To achieve high-precision positioning, pattern recognition algorithms, such as template matching or deep learning-based target detection, are often used to perform detailed analysis of the chip image to identify and correct these deviations. The precise positioning results are converted into control commands, driving a precision actuator to adjust the chip's pose, ultimately achieving precise alignment of the chip on the packaging substrate. The deviation correction process can be viewed as a closed-loop feedback system, gradually approaching the target accuracy through iterative measurement and adjustment. Deep learning, trained on massive amounts of data, can effectively extract robust features from images and model complex deviations, thereby achieving high-precision positioning under harsh conditions.

[0035] like Figure 1As shown, this embodiment of the invention provides a chip packaging positioning method, which includes the following steps: Step S100: Acquire a physical image containing the chip.

[0036] An image acquisition device is used to acquire images of the chip. In one embodiment, the image acquisition device includes an industrial camera with a resolution of 1920x1080 pixels and an LED light source for providing uniform illumination. The industrial camera communicates with a central processing unit (CPU) via a gigabit Ethernet interface and transmits the acquired image data to the CPU. The optical axis of the camera lens is perpendicular to the chip surface.

[0037] In some other alternative implementations, different image sensor types (e.g., CMOS or CCD) or different illumination methods (e.g., backlighting or structured light) may be used.

[0038] Step S200: The physical image is processed using the first image processing algorithm to extract the chip's outline and pin arrangement information, and to obtain the chip's preliminary position coordinates and angle values.

[0039] Chip features are extracted from images using image processing algorithms. In one embodiment, the first image processing algorithm includes an edge detection operator and a shape fitting algorithm. First, the Sobel operator is used to detect edge information in the image, and then the Hough transform algorithm is used to fit straight lines to the detected edges, thereby extracting the chip's contour lines. Simultaneously, by analyzing the grayscale value variation patterns in the image, the chip pin arrangement information is identified. Based on the extracted contour lines and pin arrangement information, the preliminary position coordinates and angle values ​​of the chip are calculated using the least squares method.

[0040] In some other alternative implementations, different edge detection operators (e.g., the Prewitt operator or the Laplacian operator) may be used, or different shape fitting algorithms (e.g., the RANSAC algorithm) may be employed.

[0041] Step S300: The initial position coordinates and angle values ​​are matched and compared with the preset substrate standard template. When the similarity of the matching is lower than the preset threshold, the physical image is input into the pre-trained convolutional neural network model.

[0042] The extracted chip pose is compared with a pre-defined standard template to determine if fine-tuning is needed. In one embodiment, the matching comparison process includes calculating the Euclidean distance between the initial position coordinates and the target position coordinates in the substrate standard template, and calculating the difference between the initial angle value and the target angle value in the substrate standard template. Simultaneously, the similarity score between the extracted contour lines and the contour lines in the substrate standard template is calculated. If the Euclidean distance is greater than 5 pixels, the angle difference is greater than 0.5 degrees, and the similarity score is less than 0.8, the similarity of the match is determined to be below a preset threshold. In this case, the acquired physical image is passed to a pre-trained convolutional neural network model for further analysis.

[0043] In some other alternative implementations, different distance metrics (e.g., Manhattan distance or Chebyshev distance) or different similarity evaluation metrics (e.g., the structural similarity index SSIM) may be used.

[0044] Step S400: The convolutional neural network model analyzes the symbols and background interference factors in the physical image to output a compensation adjustment vector that characterizes the nonlinear deviation of the chip.

[0045] A convolutional neural network (CNN) model is used to perform depth analysis on images to obtain precise pose adjustment information. In one embodiment, the pre-trained CNN model is configured to recognize identifiers on the chip surface (e.g., manufacturer logos or model codes) and detect background interference factors in the image (e.g., uneven lighting or surface contaminants). By analyzing the degree of deformation of the identifiers and the distribution of background interference factors, the model can calculate a compensation adjustment vector that includes the translational deviation of the chip in the X and Y axes, as well as the rotational deviation about the Z axis.

[0046] In some other alternative implementations, different neural network architectures (e.g., ResNet or DenseNet) or different training methods (e.g., transfer learning or adversarial training) may be used.

[0047] Step S500: Generate driving instructions based on the compensation adjustment vector to drive the precision actuator to adjust the physical position and physical angle of the chip.

[0048] The compensation adjustment vector is converted into control commands to drive the actuator to adjust the chip's position and angle. In one embodiment, the drive commands include a set of control signals sent to a six-axis robotic arm. Based on the received control signals, the robotic arm precisely adjusts the chip's position and orientation in three-dimensional space.

[0049] In some other alternative implementations, different actuator types (e.g., stepper motors or piezoelectric ceramics) or different control strategies (e.g., PID control or fuzzy control) may be used.

[0050] Step S600: Through feedback loop, obtain the adjusted chip position coordinates and angle values ​​from the adjusted physical image, and determine whether the positioning accuracy meets the preset accuracy standard, so as to decide whether to iteratively adjust the chip.

[0051] Continuous optimization of alignment accuracy is achieved through closed-loop control. In one embodiment, the feedback loop includes re-acquiring the adjusted chip image and recalculating the chip's position coordinates and angle values ​​using a first image processing algorithm. The calculated position coordinates and angle values ​​are then compared with a standard template on the substrate to determine if the positioning accuracy meets a preset accuracy standard (e.g., position deviation less than 1 micrometer, angle deviation less than 0.01 degrees). If the positioning accuracy does not meet the requirements, steps S400 to S600 are repeated until the accuracy requirements are met or the maximum number of iterations is reached.

[0052] In some other alternative implementations, different accuracy metrics (e.g., root mean square error RMSE) may be used, or different iteration termination conditions (e.g., early stopping strategies) may be employed.

[0053] Through the above scheme, this embodiment can perform rapid preliminary positioning using the first image processing algorithm, and perform precise nonlinear deviation compensation through a convolutional neural network model, while continuously optimizing the alignment accuracy by combining a feedback loop. This multi-stage, closed-loop feedback positioning method can effectively solve the problem of chip packaging positioning in complex industrial environments, and achieve rapid, high-precision automated positioning.

[0054] Compared with existing technologies, this embodiment can achieve high-precision positioning that meets preset accuracy standards by using precise analysis of convolutional neural networks and adjustment of precision actuators, even when there is a large deviation in the initial positioning. Furthermore, the closed-loop feedback mechanism ensures the stability and reliability of the positioning process and can adaptively adjust the positioning strategy according to actual conditions, thereby improving the overall performance of the system.

[0055] In one specific implementation, based on the above embodiments, the physical image is processed using a first image processing algorithm, further comprising: converting the acquired physical image from its original color format (e.g., RGB format) to a grayscale image. This conversion can employ a weighted average method commonly used in the art, for example, weighting the pixel values ​​of the three RGB channels according to a ratio of 0.299:0.587:0.114 to obtain the grayscale value of each pixel.

[0056] The converted grayscale image is smoothed using a Gaussian filter to reduce image noise and improve the accuracy of subsequent edge detection. The Gaussian filter uses a two-dimensional Gaussian kernel to convolve each pixel in the image. The size and standard deviation of the Gaussian kernel are adjustable parameters used to control the degree of smoothing. In one example, a kernel of size [missing value] can be used. Standard deviation The Gaussian kernel. In some other alternative implementations, Gaussian kernels of different sizes can be used (e.g., or ), or adjust the standard deviation of the Gaussian kernel. This allows for adaptation to images with varying noise levels. Additionally, other optional image smoothing methods, such as median filtering and bilateral filtering, can be used as alternatives to Gaussian filtering.

[0057] Through the above solution, this embodiment can effectively reduce noise in the physical image, provide clearer input for subsequent image processing steps, and thus improve the overall positioning accuracy and robustness.

[0058] In one specific implementation, based on the above embodiments, the Canny edge detection algorithm is used to process the smoothed grayscale image to extract strong edges and obtain contour lines. Specifically, the Canny edge detection algorithm first uses the Sobel operator to calculate the gradient components of the image in the horizontal and vertical directions, and then calculates the gradient magnitude and gradient direction of each pixel. Pixels with gradient magnitudes exceeding a set threshold are considered candidate edge points. Next, the Non-Maximum Suppression method is used to refine the edges, retaining the pixels with the largest gradient magnitude in the gradient direction and suppressing other pixels. Then, a dual threshold method is used for edge connection. Pixels with gradient magnitudes above the high threshold are considered strong edge points and are directly retained; pixels with gradient magnitudes below the low threshold are directly suppressed; pixels with gradient magnitudes between the high and low thresholds are also retained if they are adjacent to strong edge points, otherwise they are suppressed. Through this step, clear and continuous edge contour lines in the image can be effectively extracted. In some optional implementations, other edge detection operators, such as the Prewitt operator or the Laplacian operator, can be used to extract image edges and obtain the chip's contour lines.

[0059] Through the above scheme, this embodiment can accurately extract the edge contour of the chip using the Canny edge detection algorithm, providing high-quality image features for subsequent positioning and deviation analysis.

[0060] In one specific implementation, based on the above embodiments, the Hough transform is used to perform line detection on strong edges to obtain pin arrangement information. The specific implementation includes: first, setting the parameter space quantization level of the Hough transform. 1 pixel The degree is 1. Then, for each non-zero pixel in the strong edge image obtained after Canny edge detection, in the parameter space... Accumulate the sums. Specifically, for each edge point in the image... Calculate all possible line parameters satisfy For each calculated Add 1 to the corresponding position in the accumulator matrix. Next, search the accumulator matrix for local maxima, i.e., accumulated values ​​exceeding a preset threshold. For example, if a threshold of 100 is set, values ​​greater than this threshold... Combinations are considered potential straight lines. Finally, the detected straight lines are filtered and merged, with a minimum segment length of 20 pixels and a maximum allowed discontinuity of 5 pixels. Line segments that are close in distance and at similar angles are merged into a single line. Through this process, a set of straight line segments characterizing the chip pin arrangement can be obtained, containing the precise position and orientation information of each pin edge. In some alternative implementations, the Radon transform can be used instead of the Hough transform for line detection.

[0061] Through the above scheme, this embodiment can extract the pin arrangement information of the chip more accurately, providing a more reliable data basis for subsequent deviation judgment and compensation. Especially when the chip is tilted or rotated, the deviation of the pin can be identified more accurately, thereby improving the positioning accuracy.

[0062] In one specific implementation, based on the above embodiments, before inputting the physical image into the convolutional neural network model, the method further includes: First, a multi-feature fusion process based on traditional features is performed. Specifically, this multi-feature fusion process extracts and integrates various visual features related to chip positioning from the physical image, and generates a set of nonlinear deviation parameters to assist subsequent deep learning model analysis. First, various visual features of the chip are extracted. These visual features may include, but are not limited to: chip contour edge information, points on the chip surface (such as corners, intersections, etc.), chip pin arrangement information, and texture features of specific areas on the chip surface. Then, a feature fusion algorithm is used to integrate the extracted visual features. The feature fusion algorithm can be a weighted average algorithm, which weights different features according to preset weights to generate a fused feature vector. Alternatively, the feature fusion algorithm can be a feature concatenation algorithm, which directly concatenates different feature vectors into a longer feature vector. Next, based on the fused feature vector, the nonlinear deviation parameters of the chip are calculated. The nonlinear deviation parameters describe the deviation of the chip from its ideal position and may include translational deviation, rotational deviation, scaling deviation, and distortion deviation. These deviation parameters can be estimated using methods such as least squares and RANSAC algorithms. Finally, the calculated nonlinear bias parameters are passed to the convolutional neural network model as one of its inputs. In some alternative implementations, the nonlinear bias parameters may not be used as direct inputs to the model, but rather as a constraint during model training to guide the model in learning more accurate feature representations.

[0063] Through the above scheme, this embodiment can utilize complementary information from multiple visual features to improve the accuracy and robustness of chip positioning, while providing more effective input for subsequent deep learning models, accelerating model training and optimization.

[0064] In one specific implementation, based on the above embodiments, the multi-feature fusion processing includes: First, for the extracted contour lines, calculating their edge strength, continuity, and closure. Specifically, the Sobel operator can be used to calculate the gradient magnitude of each pixel on the contour line, and the average of the gradient magnitudes is used as the edge strength score. Then, the number of consecutive pixels on the contour line is counted, and the proportion of consecutive pixels to the total number of pixels is calculated as the continuity score. Next, the ratio of the area enclosed by the contour line to the square of the contour line's perimeter is calculated as the closure score. The formula is used... Calculate the profile quality score ,in, Indicates the edge strength score. Indicates continuous scoring. The score indicates the degree of closure. , and These are the corresponding weight coefficients, and .

[0065] For the extracted pin arrangement information, firstly, the number of detected pins is compared with the expected number of pins in the substrate standard template. Then, the spacing between all adjacent pins is calculated, and the standard deviation of these spacings is also calculated. The closeness of the pin number to the expected number and the reciprocal of the spacing standard deviation are used as the pin number score and spacing consistency score, respectively. The formula is used... Calculate pin quality score ,in, The score indicates the number of pins. Indicates the spacing consistency score. and These are the corresponding weight coefficients, and .

[0066] Then, based on the contour quality score and pin quality rating The system dynamically assigns fusion weights to the contour lines and pin arrangement information. The weight calculation formula is as follows: Next, two independent sets of deviation parameters were calculated based on the contour lines and pin arrangement information, respectively. The deviation parameters calculated based on the contour lines include positional deviations. and angle deviation The deviation parameters calculated based on pin arrangement information include positional deviation. and angle deviation Then, a weighted average is calculated using the aforementioned dynamic weights to determine a more robust and reliable set of nonlinear deviation parameters. The formula for calculating the weighted average is as follows: In some other alternative implementations, edge strength scoring Continuity scoring can also be obtained by calculating the gradient variance of local regions along the contour line. The degree of closure can be obtained by calculating the rate of change of curvature of the contour line. This can be obtained by calculating the texture complexity within the contour line. In some alternative implementations, obviously faulty pins, such as those significantly farther from other pins than the average distance, can be excluded when calculating pin spacing. In some alternative implementations, other factors can be considered when assigning fusion weights, such as the illumination intensity of the chip surface; when the illumination intensity is weak, the weight of the contour line can be appropriately reduced. In some alternative implementations, the nonlinear deviation parameter can also include a scaling factor and a shearing factor.

[0067] Through the above scheme, this embodiment can adaptively adjust the proportion of contour lines and pin arrangement information in the positioning process according to their quality, thereby improving the robustness and accuracy of positioning.

[0068] In one specific implementation, based on the above embodiments, the convolutional neural network model is specifically a U-Net network structure. This U-Net network includes an encoder and a decoder, as well as skip connections between the encoder and decoder.

[0069] First, the encoder consists of multiple convolutional layers and max-pooling layers to extract multi-scale features from the input physical image. For example, the encoder may contain 5 coding blocks, each containing two... Each convolutional layer is followed by a ReLU activation function, and a... The encoder uses max pooling layers. Through layer-by-layer convolution and pooling, the encoder progressively transforms the original image into a series of abstract feature maps.

[0070] Specifically, within each coding block, convolutional layers perform feature extraction, the ReLU activation function introduces non-linearity, and max pooling layers are used to reduce the resolution of the feature map, increase the receptive field, and reduce computational cost.

[0071] Then, the decoder, symmetrical to the encoder structure, consists of multiple deconvolutional and convolutional layers. These layers progressively restore the feature maps extracted by the encoder to the resolution of the original image, generating pixel-level predictions. For example, the decoder can contain five decoding blocks, each containing one... The deconvolutional layer is used to upsample the feature map, and two... The convolutional layers are followed by a ReLU activation function.

[0072] Next, skip connections located between the encoder and decoder are used to directly pass the feature maps of each layer in the encoder to the corresponding layer in the decoder. These skip connections combine low-level detailed information with high-level semantic information, which helps to improve the accuracy of segmentation.

[0073] Next, an attention mechanism module was added to the skip connections of the U-Net network. This attention mechanism module is used to automatically learn which features are more relevant to the current task and assign higher weights to these features.

[0074] Specifically, the attention mechanism module is the Squeeze-and-Excitation (SE) module. This SE module first performs global average pooling on the input feature map to obtain a global feature vector. Then, this global feature vector passes through two fully connected layers to generate a set of weight coefficients. Finally, these weight coefficients are applied to the original feature map to weight the features of each channel.

[0075] In some alternative implementations, the attention mechanism module can also be a Convolutional Block Attention Module (CBAM), which learns attention weights in both channel and spatial dimensions to capture information in the image more comprehensively. In other alternative implementations, the U-Net network structure can be replaced with other semantic segmentation networks such as DeepLabv3+ or Mask R-CNN.

[0076] Through the above scheme, this embodiment can extract multi-scale features using the encoder-decoder structure and fuse features at different levels using skip connections and attention mechanisms, thereby improving the accuracy and robustness of chip positioning.

[0077] In one specific implementation, the compensation adjustment vector output by the convolutional neural network model is obtained through the following steps: First, a convolutional neural network model is configured to output a deviation contribution heatmap. Specifically, this deviation contribution heatmap is a single-channel grayscale image of the same size as the physical image, where the grayscale value of each pixel represents the contribution of the corresponding image region to the overall chip positioning deviation. A higher grayscale value indicates a greater influence of the region on the deviation, and vice versa. The directional relationship between the grayscale value and the deviation is obtained through neural network training. The deviation contribution heatmap can be understood as a visual representation of the "deviation sensitivity" of each region on the chip image; for example, regions with sharp chip edges and correct positioning have grayscale values ​​close to zero, while regions with blurred edges or deviations exhibit higher grayscale values. Then, a heatmap parsing module is configured to receive the deviation contribution heatmap and calculate its vector integral to generate a compensation adjustment vector. Specifically, this heatmap parsing module first divides the deviation contribution heatmap into multiple small regions (e.g., The process involves first dividing the pixel grid, then calculating the average or sum of the grayscale values ​​of all pixels within each region, which is used as the bias contribution intensity for that region. Next, according to a pre-defined directional encoding rule, the bias contribution intensity of each region is converted into a two-dimensional vector, representing the region's contribution to the chip's bias in the horizontal and vertical directions. For example, if the horizontal direction to the right is defined as the positive X direction and the vertical direction downwards as the positive Y direction, the contribution vector for each region can be represented as... Finally, the contribution vectors of all regions are weighted and averaged or summed to obtain the compensation adjustment vector. .in, and These represent the pixel offsets of the chip in the horizontal and vertical directions, respectively. This indicates the rotation angle of the chip. In some other alternative implementations, the deviation contribution heatmap can be a multi-channel image, with different channels representing different types or directions of deviation contribution. The heatmap analysis module needs to analyze each channel separately and fuse the results to generate a compensation adjustment vector.

[0078] Through the above scheme, this embodiment can visualize the analysis results of the convolutional neural network model in the form of a deviation contribution heatmap, and use the heatmap to accurately calculate the chip's compensation adjustment vector, thereby improving the chip positioning accuracy and robustness.

[0079] In one specific implementation, after the image acquisition module acquires a physical image containing the chip, the physical image may contain physical defects such as scratches and stains on the chip surface, or optical interference such as reflections and shadows caused by uneven lighting.

[0080] First, the image preprocessing unit in the central processing module analyzes the physical image and uses an image segmentation algorithm (e.g., Mask R-CNN or DeepLabv3+) to identify abnormal interference regions in the image. Specifically, this image segmentation algorithm is trained to identify and segment different types of interference. For example, scratched areas are labeled "scratches," stained areas are labeled "stains," and reflective areas are labeled "reflections." The segmentation result is the generation of a segmentation mask with the same size as the original image, where different pixel values ​​represent different interference regions.

[0081] Then, for each identified anomalous interference region, the deviation analysis processing module evaluates its type, shape, and location. The type is directly derived from the label of the segmentation mask. The shape is described by calculating the geometric features of the segmented region, such as its area, perimeter, and aspect ratio. The location is determined by calculating the centroid coordinates of the segmented region.

[0082] Next, based on this information, the deviation analysis and processing module calls a pre-trained deviation inference model. This deviation inference model is a regression model whose input is the type, shape, and location information of the interference region, and whose output is the positioning deviation caused by the interference (a two-dimensional vector representing the pixel offset in the X and Y directions). For example, a large scratch located on the edge of a chip may cause the chip's center position to shift by -0.2 pixels in the X direction and by 0.1 pixels in the Y direction.

[0083] Specifically, this deviation inference model can be trained as follows: First, a large number of chip images containing various interferences are collected, and the true positional deviation of the chips in these images is obtained manually or through high-precision equipment. Then, a regression model is trained using this data, enabling the model to learn the mapping relationship between interference features and positional deviations. The regression model can be linear regression, support vector regression, or a neural network, etc.

[0084] Finally, the positioning deviation inferred from each abnormal interference region is converted into a deviation contribution vector, which is then superimposed on the deviation contribution heatmap and used to generate a compensation adjustment vector. In other optional implementations, different image segmentation algorithms, such as FCN or U-Net, can be employed, and different regression models can be selected based on the specific application scenario. For example, when dealing with a large number of interference types, a deep neural network can be chosen as the deviation inference model to achieve higher accuracy.

[0085] Through the above scheme, this embodiment can accurately identify and quantify the positioning deviation caused by physical defects on the chip surface and optical interference during the imaging process, thereby effectively improving positioning accuracy and robustness.

[0086] In one specific implementation, the nonlinear deviation is a composite deviation comprising rotational and translational deviations; the convolutional neural network model analyzes the physical image including: First, the network structure is designed to include two parallel output branches: a translational deviation prediction branch and a rotational deviation prediction branch.

[0087] Specifically, in the decoder part of the model, after several layers of deconvolution and upsampling operations, the feature map is separated into two independent channel groups. The first group of channels is connected to a fully connected layer containing two neurons, used to predict the translation compensation component. ,in and These represent pixel offsets along the X and Y axes, respectively. The second set of channels is connected to a fully connected layer containing a single neuron to predict the rotation compensation component. ,in This indicates the rotational angle deviation of the chip, in degrees.

[0088] Then, during the training phase, the model uses a custom loss function consisting of two parts: translation loss. and rotational loss Translation loss Mean squared error (MSE) is used to measure the difference between the predicted and actual translation. Rotational loss. Smooth L1 loss is used to reduce the impact of outliers. Overall loss function. for: in, It is a weighting factor used to balance translation and rotation losses. For example, in one embodiment, It was set to 0.7 to emphasize the accuracy of translation deviation correction.

[0089] Next, during the inference phase, the model outputs translation compensation components in parallel. and rotational compensation components The execution control module combines these two components into the final compensation adjustment vector. This information is then transmitted to a precision actuator, enabling independent correction of translational and rotational deviations.

[0090] In other alternative implementations, different network structures can be used to achieve bias decoupling. For example, two independent convolutional neural networks can be used to predict translational and rotational biases separately; or a shared feature extractor can be used, followed by separate fully connected layers to predict translational and rotational biases. Furthermore, the choice of loss function can also vary; for example, Huber loss or Tukey loss can be used instead of MSE loss or Smooth L1 loss to improve the robustness of the model.

[0091] Through the above solution, this embodiment can accurately decouple and correct the combined translation and rotation deviations that exist in the chip packaging process, thereby improving positioning accuracy and stability.

[0092] In one specific implementation, based on the above embodiments, if it is determined that the positioning accuracy does not meet the preset accuracy standard, the central processing module first fuses the newly acquired physical image after adjustment with the remaining deviation information calculated in the above steps. Specifically, the remaining deviation information may include the remaining positional deviation. and remaining angular deviation The residual deviation information is calculated in the above steps by comparing the adjusted chip position coordinates and angle values ​​with a preset substrate standard template. Then, the fused information is packaged into a data packet as input for a new iteration. Next, the above steps are repeated: the convolutional neural network model receives the data packet and analyzes it based on the physical image and residual deviation information to generate a new compensation adjustment vector. The convolutional neural network model can use the residual deviation information to adjust its internal attention mechanism, thereby focusing more on feature regions in the image related to the residual deviation. In some alternative implementations, the residual deviation information can be converted into a spatial attention mask and superimposed on the physical image to guide the convolutional neural network model to focus on specific image regions. Finally, step 205 is repeated, and based on the new compensation adjustment vector, the precision actuator is driven again to make more precise adjustments to the chip's position and angle.

[0093] Through the above approach, this embodiment can make full use of the feedback information from the previous adjustment to guide the neural network to analyze the remaining deviation more accurately, thereby accelerating the iteration process and improving positioning efficiency.

[0094] In one specific implementation, iterative adjustment via a feedback loop includes: Preset a maximum number of iterations Maximum number of iterations It is a positive integer, and its value is determined comprehensively based on factors such as the specific chip type, positioning accuracy requirements, and production line cycle time. For example, for chips with high positioning accuracy requirements, or in cases where the initial deviation may be large, it can be... Set it to a slightly larger size (e.g.) This allows the system ample opportunity for adjustments. For chips with lower positioning difficulty, or production lines with stricter cycle time requirements, [the following can be done]: Set it a little smaller (e.g.) ).

[0095] Before each physical adjustment operation in the above steps, the system checks an internal iteration counter. The counter records the number of iterations the chip has undergone. Initially, the counter is set to 0.

[0096] Next, after completing one physical adjustment and before returning to the step of reacquiring images, the iteration counter... The value will increment by 1, that is... .

[0097] Specifically, before determining whether the positioning accuracy meets the preset accuracy standard in the above steps, the system will first update the current iteration counter value. With the preset maximum number of iterations Compare. If If the maximum allowed number of iterations has been reached, the system will skip the accuracy judgment step and directly terminate the positioning process for the chip.

[0098] Finally, the system marks the chip as "positioning failed" and logs the event in an error log. Simultaneously, the system can send an alarm signal to the higher-level control system to alert the operator. Even if the chip's positioning accuracy still fails to meet the preset accuracy standard at this point, the system will not perform any further iterative adjustments.

[0099] In some other alternative implementations, the maximum number of iterations It doesn't have to be a fixed constant; instead, it can be dynamically adjusted based on the chip type, batch, or current production line status. For example, a database could be created that stores the values ​​corresponding to different chip types. Before starting the localization process, the system first queries this database to obtain the value corresponding to the current chip type. Value. Alternatively, the system can adjust the value based on the real-time status of the production line (such as equipment load, temperature, etc.). Dynamic adjustments can be made; for example, when the equipment load is high, the speed can be appropriately reduced. This reduces the positioning time of individual chips and alleviates system pressure.

[0100] Through the above solution, this embodiment can ensure that the positioning process will not fall into an infinite loop due to unexpected situations, while also avoiding wasting valuable production line time due to excessive adjustments, thereby achieving a balance between positioning efficiency and stability.

[0101] In one specific implementation, the method further includes: Once the positioning accuracy meets the preset accuracy standard, meaning the chip has successfully completed high-precision positioning, the system first acquires the original physical image used in this positioning operation from the image acquisition module, and then extracts the compensation adjustment vector finally generated in this positioning process from the central processing module. Specifically, the physical image is stored in uncompressed TIFF format to retain the most image detail information. The compensation adjustment vector, containing three floating-point values ​​representing the position compensation amount in the X-axis direction, the position compensation amount in the Y-axis direction, and the angle compensation amount, is serialized into a JSON format text string. Subsequently, the system packages the physical image (TIFF) and the compensation adjustment vector (JSON) to form a complete data pair and assigns it a globally unique identifier (UUID). This data pair is then written to a dedicated data table in the database. The database can be a relational database (e.g., PostgreSQL) or a non-relational database (e.g., MongoDB) to ensure the reliability and scalability of data storage. In other optional implementations, the data pair can also be stored in a distributed file system (e.g., HDFS) to support larger-scale data storage needs. After the data is stored, the system will record the metadata information of this data storage operation, such as storage time, data size, and storage location, for subsequent data management and maintenance.

[0102] Through the above approach, this embodiment can transform each successful positioning operation into valuable data assets, providing a data foundation for subsequent machine learning model training and system self-optimization.

[0103] In one specific implementation, when the system detects that the number of stored data pairs has reached a preset data volume threshold, an online optimization process is automatically triggered. Specifically, the data volume threshold is set to 500, a value that comprehensively considers both the effectiveness of model updates and the consumption of computing resources.

[0104] First, the system's task scheduler (e.g., the Python-based Celery framework) periodically (e.g., every 24 hours) checks the number of stored data pairs. This check is performed by querying a database, where a dedicated table records the storage information for each data pair.

[0105] Then, when the task scheduler detects that the number of data pairs exceeds 500, it automatically triggers the online optimization process. This process is executed by a separate background training program that is pre-configured with the hyperparameters required for model training (e.g., learning rate, batch size).

[0106] Next, the background training program reads 500 newly accumulated data pairs from the database and merges them with the dataset previously used for initial training.

[0107] Specifically, incremental training uses the Adam optimizer with a learning rate of 0.0001 and 10 training epochs. Fine-tuning fixes the parameters of most layers of the model and trains only the last few layers (e.g., fully connected layers) to prevent overfitting.

[0108] After training is complete, an updated model file is generated.

[0109] Then, atomic operations (such as file renaming) are used to replace the old model file currently in use with the updated model file, thereby achieving a smooth model switch and avoiding service interruption. At the same time, the log of this model update is recorded in the database for subsequent tracking and analysis.

[0110] In other alternative implementations, the task scheduler can also perform checks irregularly, instead adopting an event-driven approach, triggering a check whenever a new data pair is generated. Furthermore, the optimizer and hyperparameters used for model training can be adjusted according to specific circumstances; for example, an SGD optimizer can be used, or a smaller learning rate can be employed. The conditions for triggering the online optimization process can also be a combination of factors, such as the number of data pairs exceeding a threshold and the time interval since the last model update exceeding a certain duration.

[0111] Through the above solution, this embodiment enables the system to automatically adapt to differences in new chip batches on the production line, minor changes caused by equipment aging, and continuously maintain optimal positioning performance.

[0112] This invention provides a positioning system device for chip packaging, such as... Figure 2 As shown, the device includes: an image acquisition module M100, a preliminary positioning processing module M200, a deviation analysis processing module M300, an execution control module M400, and a feedback iteration control module M500.

[0113] The image acquisition module M100 includes an industrial camera and a coaxial light source. The industrial camera is a Basler acA1920-155um camera from Germany, with a resolution of 1920x1200 pixels and a telecentric lens with a focal length of 50mm to reduce perspective distortion. The coaxial light source is an LED light source with an emission wavelength of 630nm, providing uniform and stable illumination. The image acquisition module M100 is configured to acquire physical images of the chip. In one application scenario, the image acquisition module M100 is located above the chip packaging equipment, acquiring real-time images of the chip by shooting downwards. In other alternative implementations, the image acquisition module M100 can also use a line scan camera or an area scan camera, and the light source can be a backlight or a ring light source.

[0114] The preliminary positioning processing module M200, connected to the image acquisition module M100, receives the physical image acquired by the image acquisition module M100. The preliminary positioning processing module M200 consists of a central processing unit (CPU) and a graphics processing unit (GPU), such as an Intel Core i7-9700K CPU and an NVIDIA GeForce RTX 2080 Ti GPU. The preliminary positioning processing module M200 is configured to process the physical image using a first image processing algorithm to extract the chip's contour lines and pin arrangement information, obtaining the chip's preliminary position coordinates. With angle value Specifically, the first image processing algorithm includes the Canny edge detection algorithm and the Hough transform algorithm. In one specific implementation, the low threshold of the Canny edge detection algorithm is set to 50, the high threshold is set to 150, and the minimum line segment length of the Hough transform algorithm is set to 20 pixels, and the maximum allowed discontinuity is set to 5 pixels. In some other optional implementations, the first image processing algorithm may also be the Sobel operator, the Harris corner detection algorithm, or the template matching algorithm.

[0115] The deviation analysis processing module M300 is connected to the preliminary positioning processing module M200 and receives the preliminary position coordinates output by the preliminary positioning processing module M200. With angle value The deviation analysis processing module M300 includes a memory and a pre-trained convolutional neural network model running on the memory. The memory stores a preset substrate standard template, which includes the ideal position coordinates of the chip on the substrate. Ideal angle value Contour template and pin arrangement template. The deviation analysis processing module M300 is configured to process the initial position coordinates. With angle value The system compares the physical image with a preset substrate standard template. When the similarity of the match is lower than a preset threshold (e.g., 0.95), a pre-trained convolutional neural network model analyzes the identifiers and background interference factors in the physical image to output a compensation adjustment vector characterizing the nonlinear deviation of the chip. In one implementation, the convolutional neural network model is a U-Net network structure, which takes a 256x256 pixel grayscale image as input and outputs a deviation contribution heatmap of the same size as the input image. In other alternative implementations, the convolutional neural network model can also be a network structure such as ResNet, DenseNet, or Mask R-CNN.

[0116] The execution control module M400 is connected to the deviation analysis and processing module M300 and receives the compensation adjustment vector output by the deviation analysis and processing module M300. The execution control module M400 is implemented using an embedded processor, such as an ARM Cortex-M7. The execution control module M400 is configured to perform adjustments based on the compensation vector. The system generates drive instructions to instruct a precision actuator to adjust the physical position and angle of the chip. For example, the execution control module M400 converts pixel deviations into micrometers in the physical world and angular deviations into pulse signals. In some optional implementations, the execution control module M400 can optimize the adjustment process using a PID control algorithm to improve the accuracy and stability of the adjustment.

[0117] The feedback iteration control module M500 is connected to the image acquisition module M100, the preliminary positioning module 140, and the execution control module M400, respectively. The feedback iteration control module M500 is configured to obtain the adjusted chip position coordinates and angle values ​​from the adjusted physical image through a feedback loop, and determine whether the positioning accuracy meets a preset accuracy standard to decide whether to perform iterative adjustment. In one embodiment, the preset accuracy standard is a position deviation of less than 0.5 pixels and an angle deviation of less than 0.1 degrees. The feedback iteration control module M500 is also configured to set the maximum number of iterations. (For example, When the number of adjustments performed reaches Even if the positioning accuracy still does not meet the preset accuracy standard, the iterative adjustment will terminate. In other optional implementations, the feedback iterative control module M500 can also dynamically adjust the preset accuracy standard and the maximum number of iterations according to the actual situation.

[0118] The aforementioned image acquisition module M100, preliminary positioning processing module M200, deviation analysis processing module M300, execution control module M400, and feedback iteration control module M500 work together in a coordinated manner. First, the image acquisition module M100 acquires a physical image of the chip. Then, the preliminary positioning processing module M200 uses traditional image processing algorithms to quickly obtain the chip's preliminary position coordinates and angle values. Next, the deviation analysis processing module M300 analyzes the chip's nonlinear deviations and outputs a compensation adjustment vector. Finally, the execution control module M400 drives a precision actuator to adjust the chip's position, and the feedback iteration control module M500 iteratively adjusts the position until the preset accuracy standard is met. Through the coordinated work of these modules, complex and dynamic industrial environments such as uneven lighting, background interference, and physical defects on the chip surface can be overcome, achieving rapid, high-precision, and highly stable automated chip positioning. This solves the technical bottlenecks of existing technologies, such as poor robustness, difficulty in handling complex nonlinear deviations, and the inability to balance speed and accuracy.

[0119] Through the above solution, this embodiment can achieve rapid and accurate capture of chip pose in complex environments. Under harsh working conditions with surface defects and optical interference, it can stably achieve micron-level positioning accuracy, effectively solve the problem of accurate decoupling and compensation of translation-rotation composite deviation, and can balance speed and accuracy to meet the high-cycle production requirements of modern semiconductor packaging production lines.

[0120] In one specific implementation, the deviation analysis processing module of the central processing module includes a preprocessing submodule and a deviation inference submodule. The preprocessing submodule is configured to perform pixel-level classification on the acquired physical image using an image segmentation algorithm, identifying regions belonging to the chip surface, background, and interference factors. The image segmentation algorithm used can be based on deep learning like Mask R-CNN, or a traditional threshold-based image segmentation method, such as the `cv::threshold` function in the OpenCV library. The preprocessing submodule is also configured to determine the type of the identified interference regions, for example, judging whether they are scratches, stains, or reflections based on color and texture features. The deviation inference submodule is configured to look up a pre-established deviation mapping table based on the output of the preprocessing submodule. This deviation mapping table stores the impact of different types and locations of interference factors on positioning accuracy. The deviation inference submodule obtains a deviation adjustment amount corresponding to the interference factor by looking up the table and uses this deviation adjustment amount for subsequent compensation adjustment vector generation. The deviation mapping table can be obtained in advance through experimental calibration or simulation. In some alternative implementations, the bias inference submodule does not use a lookup table approach, but instead uses a regression model to predict the bias adjustment. The regression model takes the type and location of the confounding factors as input and outputs the corresponding bias adjustment. The regression model is trained on a large amount of experimental data using machine learning algorithms such as Support Vector Regression (SVR) or Gaussian Process Regression (GPR).

[0121] Through the above solution, this embodiment can accurately identify the interference area for physical defects on the chip surface or optical interference in the imaging process, and infer the positioning deviation caused by the interference area based on the type and location of the interference area, thereby improving the positioning accuracy and robustness.

[0122] In one specific implementation, based on the above embodiments, the deviation analysis processing module includes a decoupling analysis unit. This decoupling analysis unit is configured to receive intermediate feature maps from a convolutional neural network model, the intermediate feature maps containing encodings of various deviation information in the chip image. The decoupling analysis unit includes two parallel sub-networks: a translational deviation estimation sub-network and a rotational deviation estimation sub-network.

[0123] The translational bias estimation subnetwork receives intermediate feature maps and extracts features related to translational bias using several convolutional and pooling layers. The output layer of this subnetwork contains two neurons, used to predict the translational bias Δx in the X direction and Δy in the Y direction, respectively. A linear activation function is used to ensure that the predicted values ​​are not limited in range. The structure of the rotational bias estimation subnetwork is similar to that of the translational bias estimation subnetwork, but the size and number of its convolutional kernels can be adjusted according to actual needs to better extract features related to rotational bias. The output layer of this subnetwork contains one neuron for predicting the rotational bias Δθ. The output activation function is also a linear function.

[0124] The translation compensation component is directly composed of the output of the translation bias estimation subnetwork: Similarly, the rotation compensation component is directly constituted by the output of the rotation bias estimation subnetwork: The final compensation adjustment vector is composed of these two components: In other alternative implementations, the translation bias estimation subnetwork and the rotation bias estimation subnetwork can share some convolutional layers to reduce the number of parameters and computational cost. Furthermore, an attention mechanism can be introduced to enable the subnetworks to focus more on image regions associated with their respective biases.

[0125] Through the above scheme, this embodiment can more accurately decouple and estimate the translational and rotational deviations of the chip, thereby improving positioning accuracy and robustness.

[0126] In one specific embodiment, the positioning system device further includes an online model optimization module. This online model optimization module includes a data acquisition unit, a data storage unit, an online training trigger unit, and a model update unit. The data acquisition unit is configured to determine whether the positioning accuracy in the above steps meets a preset accuracy standard after each positioning operation. If it does, the data acquisition unit collects the original physical image acquired by the image acquisition module M100 and the verified correct compensation adjustment vector output by the deviation analysis processing module as an "image-label" data pair. The data storage unit is coupled to the data acquisition unit and is used to store the "image-label" data pairs. The data storage unit can be implemented using a relational database (e.g., MySQL) or a non-relational database (e.g., MongoDB). The online training trigger unit is configured to monitor the number of data pairs stored in the data storage unit. When this number reaches a preset data volume threshold (e.g., 1000 pairs), or when an external instruction is received (e.g., a production line shutdown maintenance signal), the online training trigger unit triggers the model update unit to execute an online incremental training process. The model update unit contains one or more program modules for incrementally training or fine-tuning convolutional neural network models. These modules are implemented based on deep learning frameworks such as TensorFlow or PyTorch. The model update unit reads newly accumulated data pairs from the data storage unit and uses these data pairs to incrementally update the parameters of the convolutional neural network model currently deployed in the bias analysis processing module. After the incremental update is complete, the model update unit deploys the updated model to the bias analysis processing module, replacing the old model for subsequent chip localization operations. In some alternative implementations, the model update unit may employ federated learning training to achieve continuous model optimization without centrally storing the original data.

[0127] Through the above solution, this embodiment enables the positioning system to have online adaptive learning capabilities. By continuously learning new data pairs, it continuously optimizes the positioning model, thereby adapting to minor changes caused by differences in new chip batches or equipment aging on the production line, and maintaining optimal positioning performance in the long term.

[0128] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the chip packaging and positioning method described above.

[0129] Through the above solution, this embodiment can significantly improve the accuracy and robustness of chip packaging positioning. Even under poor lighting conditions or with defects on the chip surface, it can still achieve micron-level positioning accuracy, effectively solving the problem of poor robustness of traditional visual positioning methods in complex industrial environments.

[0130] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A positioning method for a chip package, characterized in that, include: Acquire physical images containing the chip; The physical image is processed using a first image processing algorithm to extract the outline lines and pin arrangement information of the chip, and to obtain the preliminary position coordinates and angle values ​​of the chip. The preliminary position coordinates and angle values ​​are matched and compared with the preset substrate standard template. When the similarity of the matching is lower than the preset threshold, the physical image is input into the pre-trained convolutional neural network model. The convolutional neural network model analyzes the symbols and background interference factors in the physical image to output a compensation adjustment vector that characterizes the nonlinear deviation of the chip. Based on the compensation adjustment vector, drive instructions are generated to drive the precision actuator to adjust the physical position and physical angle of the chip; Through a feedback loop, the adjusted chip position coordinates and angle values ​​are obtained from the adjusted physical image, and it is determined whether the positioning accuracy meets the preset accuracy standard, so as to decide whether to iteratively adjust the chip.

2. The method according to claim 1, characterized in that, Processing the physical image using the first image processing algorithm includes: converting the physical image into a grayscale image, and smoothing the grayscale image using Gaussian filtering to reduce noise; The processing of the physical image using the first image processing algorithm further includes: processing the smoothed grayscale image using the Canny edge detection algorithm to extract strong edges and obtain the contour lines; The processing of the physical image using the first image processing algorithm further includes: performing line detection on the strong edges using Hough transform to obtain the pin arrangement information.

3. The method according to claim 1, characterized in that, Before inputting the physical image into the convolutional neural network model, the method further includes: Based on the contour lines and the pin arrangement information, multiple feature fusion processing is performed on the two to determine the nonlinear deviation parameters of the chip, and the convolutional neural network model performs analysis based on the nonlinear deviation parameters; The multi-feature fusion process includes: The imaging quality of the contour lines and the pin arrangement information are respectively scored using technical quality assessment. Based on the technical quality score, fusion weights are dynamically assigned to the contour lines and the pin arrangement information, wherein the fusion weights assigned to the contour lines and the pin arrangement information are positively correlated with their respective technical quality scores; The nonlinear deviation parameter is determined by weighting the fusion weights.

4. The method according to claim 1, characterized in that, The convolutional neural network model is a semantic segmentation network that includes an encoder-decoder structure and an attention mechanism module; The compensation adjustment vector output by the convolutional neural network model is obtained through the following steps: First, the convolutional neural network model outputs a deviation contribution heatmap, which shows the direction and magnitude of the contribution of each pixel region in the physical image to the overall physical deviation. Then, the compensation adjustment vector is generated analytically by calculating the vector integral or weighted centroid of the deviation contribution heatmap.

5. The method according to claim 1, characterized in that, The background interference factors include physical defects on the chip surface or optical interference during the imaging process; the convolutional neural network model analyzes the physical image by including: Identify abnormal interference areas in the physical image; Based on the type, shape, and location of the abnormal interference area, the positioning deviation caused by the abnormal interference area is inferred to generate the compensation adjustment vector.

6. The method according to claim 1, characterized in that, The nonlinear deviation is a composite deviation including rotational and translational deviations; the convolutional neural network model analyzes the physical image including: The composite deviation is decoupled and analyzed to output independent translation compensation components and independent rotation compensation components in parallel, and the compensation adjustment vector is constructed based on the translation compensation components and the rotation compensation components.

7. The method according to claim 1, characterized in that, The iterative adjustment via the feedback loop includes: If it is determined that the positioning accuracy does not meet the preset accuracy standard, the adjusted physical image and the remaining deviation information are used as the input for a new round of iteration, and the analysis and subsequent adjustment steps of the convolutional neural network model are repeated. The iterative adjustment is set with a maximum number of iterations. When the number of adjustments reaches the maximum number of iterations, the iterative adjustment is terminated even if the positioning accuracy still does not meet the preset accuracy standard.

8. The method according to claim 1, characterized in that, Also includes: After the positioning operation is completed, if the positioning accuracy meets the preset accuracy standard, the physical image of the positioning process that meets the accuracy standard and the final output compensation adjustment vector are stored as a data pair: When the number of stored data pairs reaches a preset data volume threshold, an online optimization process is automatically triggered. In the online optimization process, the stored data pairs are used to perform incremental training or fine-tuning on the convolutional neural network model to generate an updated model for subsequent localization operations.

9. A positioning system device for chip packaging, characterized in that, include: The image acquisition module is configured to acquire physical images containing the chip; The preliminary positioning processing module is configured to process the physical image using a first image processing algorithm to extract the outline lines and pin arrangement information of the chip, and obtain the preliminary position coordinates and angle values ​​of the chip. The deviation analysis and processing module is configured to match and compare the preliminary position coordinates and the angle value with a preset substrate standard template, and when the similarity of the match is lower than a preset threshold, analyze the identifiers and background interference factors in the physical image through a pre-trained convolutional neural network model to output a compensation adjustment vector characterizing the nonlinear deviation of the chip. The execution control module is configured to generate drive instructions based on the compensation adjustment vector to drive the precision actuator to adjust the physical position and physical angle of the chip; The feedback iteration control module is configured to obtain the adjusted chip position coordinates and angle values ​​from the adjusted physical image through a feedback loop, and determine whether the positioning accuracy meets the preset accuracy standard in order to decide whether to perform iterative adjustment.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images