Chip 3d angle correction method and system

Abstract

The application discloses a kind of chip 3D angle correction method and system, through the chip image sequence in the process of chip patching is collected;Based on the optical flow field of classical TV-L1 variation model calculation chip image sequence, obtain estimated flow field, median filter is carried out to estimated flow field, obtain auxiliary flow field, based on estimated flow field and auxiliary flow field, construct mutual guide optical flow estimation model, based on mutual guide optical flow estimation model, calculate the offset angle of chip and according to the angle of deviation of chip, angle correction is carried out to chip, the technical problem that present technical cannot accurately detect the angle deviation of small chip is solved, accurate detection and correction of fast and small angle deviation in the process of automatic chip assembly can be realized, so as to improve the stability and reliability of entire assembly process.

Description

Technical Field

[0001] This invention mainly relates to the field of eutectic bonding technology, specifically a method and system for 3D angle correction of chips. Background Technology

[0002] Today, with the continuous development of semiconductor technology and microelectronics manufacturing technology, the requirements for the precision and reliability of chip mounting processes are becoming increasingly stringent. Especially in certain high-precision, high-performance application fields, such as aerospace and precision instruments, the accuracy of parameters such as chip mounting position and angle directly affects the performance and reliability of the final product.

[0003] Eutectic bonding technology, as a crucial technology in modern microelectronic packaging, demands high accuracy and stability. Traditional bonding methods are insufficient for certain high-precision applications, while new bonding technologies must consider various chip parameters, such as position and angle. Among these, chip angle correction is particularly critical. Due to various reasons, such as chip manufacturing deviations and mechanical errors in the bonding machine, angular deviations can occur during the bonding process, thus affecting chip performance and reliability.

[0004] To achieve high-precision chip mounting, researchers have explored various methods. Among them, optical inspection technology, due to its advantages such as high precision and non-contact operation, has gradually been widely used for angle correction in the chip mounting process. However, traditional optical inspection methods are often based on fixed models, making it difficult to adapt to various complex scenarios and requirements. Therefore, finding a more intelligent and flexible angle correction method has become a research hotspot.

[0005] Optical flow field technology, as a core technology in the field of computer vision, has been widely used in various scenarios, such as video analysis and object tracking. Optical flow field technology can accurately detect and describe motion information in images, providing new possibilities for applications such as angle correction. However, applying it to angle correction in chip placement machines requires consideration of the special and complex nature of the chip placement process, such as the stability, real-time performance, and accuracy of the optical flow field.

[0006] Against this background, a chip 3D angle correction method and system based on optical flow field technology is developed. This method can fully utilize the advantages of optical flow field technology to achieve high-precision and high-efficiency chip angle correction, meet the high requirements of modern microelectronics manufacturing technology, and has important practical value and research significance.

[0007] Chinese patent application CN116429025A discloses a chip tilt detection method, system, computer, and readable storage medium. This method detects the chip tilt angle by pre-calibrating a camera, calculating the theoretical chip position, detecting the actual grayscale value, identifying the chip region, calculating the average grayscale value, and inputting it into a tilt angle algorithm. This can shorten chip detection time to some extent and improve the efficiency of chip tilt angle detection. However, this method relies on pre-calibration, grayscale threshold settings, and tilt angle algorithm expressions, which may limit its adaptability to different types of chips, and the system has high complexity. Therefore, it may lead to reduced accuracy of detection results, misjudgments or omissions, insufficient adaptability, and increased implementation and maintenance difficulty.

[0008] The invention patent with publication number CN111220621A provides a method for detecting chip tilted surfaces. This method obtains the correspondence between the image intensity ratio of the tilted surface and the horizontal surface and the tilt angle by calibrating the chip. Then, it uses a surface detection system to obtain the image intensity of the chip surface under test and calculates the tilt angle of the chip surface. However, environmental factors can affect the accuracy of the calibration results, and the method is dependent on optical performance, calibration time, reduced accuracy when processing chips with complex surface structures, and the need for additional calibration and adjustment when processing chips with different materials and surface characteristics. Therefore, it may lead to inaccurate detection results, misjudgments, or missed detections, affecting production efficiency and product quality.

[0009] Chinese patent application CN110017814A discloses a method and system for detecting chip tilt. This method uses a laser displacement sensor for non-contact, high-precision measurement. It determines whether the chip is tilted by measuring the distance from the chip's upper surface to the laser displacement sensor, thereby improving detection accuracy and avoiding the missed and false detections common in traditional photoelectric sensor-based through-beam detection methods. However, this method has a limited measurement range and is affected by environmental factors and the limited detection efficiency of high-speed production lines. Therefore, it may lead to reduced detection accuracy, reduced detection efficiency, and an increase in false or missed detections, thus affecting the quality and efficiency of the chip manufacturing process. Summary of the Invention

[0010] The present invention provides a chip 3D angle correction method and system, which solves the technical problem that the prior art cannot accurately detect the angle deviation of micro chips.

[0011] To solve the above-mentioned technical problems, the present invention proposes a chip 3D angle correction method, which includes:

[0012] Acquire chip image sequences during the chip mounting process.

[0013] The optical flow field of the chip image sequence is calculated based on the classic TV-L1 variational model to obtain the estimated flow field.

[0014] Median filtering is applied to the estimated flow field to obtain the assist flow field.

[0015] Based on the estimated flow field and the assisted flow field, a mutually guided optical flow estimation model is constructed.

[0016] The chip offset angle is calculated based on the mutual guided optical flow estimation model.

[0017] The chip is angle-corrected based on its deviation angle.

[0018] Furthermore, based on the estimated flow field and the assisted flow field, a mutually guided optical flow estimation model is constructed, including:

[0019] Based on the consistent and inconsistent structures between the estimated flow field and the collaborating flow field, a relative structure expression is constructed. The specific formula for the relative structure expression is as follows:

[0020]

[0021] in, and Let ψ and ψ represent the first and second expressions in the relative structure expression, respectively. Let ψ represent the estimated flow field and the assisting flow field, respectively. p and These represent the optical flow at the corresponding location in the estimated flow field and the assisted flow field for pixel p in the input image I, respectively. This represents the gradient of the estimated flow field ψ at pixel p in the input image I in the x and y directions. and Indicates assisting flow field The gradient of pixel p in the x and y directions in the input image I. and Where υ∈{x,y}, and Ω represents the optical flow domain.

[0022] Based on the relative structure expression, construct the mutual guiding constraint terms.

[0023] Based on the mutual guidance constraint term, a mutual guidance optical flow estimation model is constructed.

[0024] Furthermore, based on the relative structure expression, the specific form of the mutually guiding constraint term is constructed as follows:

[0025]

[0026] in, The terms represent mutual guiding constraints, where α1, α2, β3, and β4 represent the first, second, third, and fourth equilibrium constants, respectively. and Representing ψ p and The initial optical flow, Represents ψ p and The square of the L2 norm of the difference express and The square of the L2 norm of the difference.

[0027] Furthermore, based on the classic TV-L1 variational model and according to the mutual guidance constraint term, the mutual guidance optical flow estimation model is constructed as follows:

[0028]

[0029] in, E represents the construction function of the mutually guided optical flow estimation model. d (ψ), E s (ψ) and These represent matching terms, constraint terms, nonlocal constraint terms, coupling terms, and mutual guidance constraint terms, respectively. T(p) and T(p+ψ) represent robust penalty functions with the general Charbonnier function, where the symbol δ is a constant, usually set to 0.001. p ) represent pixels p and p+ψ in the texture image, respectively. p The brightness at a certain point, where the texture image is an image obtained based on the texture of the input image I, and p' represents a specified neighborhood D centered at pixel p. p The neighboring pixels adjacent to pixel p, (u p ,v p ) represents the motion vector of pixel p. and Let θp and θ1 represent the gradient values ​​of pixel p in the horizontal and vertical directions, respectively. λ is the weight balancing the matching and constraint terms. θ1 and θ2 represent the first and second scalar weights, respectively. ωp p,p' This represents the probability that pixel p and pixel p' belong to the same surface. and These represent the optical flow components in the horizontal and vertical directions at the corresponding position of the assisted flow field corresponding to pixel p, respectively. p and v p These represent the optical flow components in the horizontal and vertical directions at the corresponding position in the estimated flow field for pixel p, respectively. and These represent the optical flow components in the horizontal and vertical directions at the corresponding positions of the assisted flow field corresponding to adjacent pixels p'.

[0030] Furthermore, based on the mutual-guided optical flow estimation model, the chip offset angle is calculated as follows:

[0031] Based on the mutual guided optical flow estimation model, the accurate optical flow field of the chip image sequence is obtained.

[0032] Based on the precise optical flow field of the chip image sequence, the offset angle of the chip from a single viewpoint is calculated, including the horizontal offset angle and the vertical offset angle.

[0033] The actual offset angle of the chip is obtained by weighted calculation of the offset angles of multiple single-view chips.

[0034] Furthermore, after obtaining the precise optical flow field of the chip image sequence based on the mutual guided optical flow estimation model, the process also includes visualizing the precise optical flow field, specifically including:

[0035] The direction of the optical flow velocity vector in the precise optical flow field is mapped to the H hue in the HSV space, using the following formula:

[0036]

[0037] Where H(u,v) represents the optical flow velocity vector in the precise optical flow field. The direction is mapped to the hue value on the H hue of the HSV space, where θ represents the mapping angle, and u and v represent the optical flow velocity vectors, respectively. Components in the horizontal and vertical directions.

[0038] The magnitude of the optical flow velocity vector in the precise optical flow field is mapped to the S-saturation in the HSV space, using the following formula:

[0039]

[0040] Where S(u,v) represents the saturation value of the magnitude of the optical flow velocity vector in the precise optical flow field mapped to the S-saturation value in the HSV space, and v max This indicates the maximum speed observed.

[0041] Furthermore, based on the chip's deviation angle, angle correction of the chip includes:

[0042] The actual deviation angle of the chip is input into the PID controller, and the PID controller controls the position of the rotating nozzle to align with the chip's orientation.

[0043] The chip 3D angle correction system provided by this invention includes:

[0044] The present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the chip 3D angle correction method provided by the present invention.

[0045] The chip 3D angle correction method and system proposed in this invention acquires chip image sequences during the chip mounting process; calculates the optical flow field of the chip image sequence based on the classical TV-L1 variational model to obtain an estimated flow field; performs median filtering on the estimated flow field to obtain an assisting flow field; constructs a mutually guided optical flow estimation model based on the estimated flow field and the assisting flow field; calculates the chip offset angle based on the mutually guided optical flow estimation model; and corrects the chip angle based on the chip offset angle. This solves the technical problem that existing technologies cannot accurately detect the angle offset of tiny chips, and can achieve rapid and accurate detection and correction of tiny angle offsets in automated chip assembly processes, thereby improving the stability and reliability of the entire assembly process.

[0046] The purpose of this invention is to design a method and system capable of real-time detection and correction of the spatial attitude angle of microchips, achieving precise and rapid chip positioning and accurate control of tilt angles. This provides technical support for the efficient and stable assembly of microchips.

[0047] The purpose of this invention is to provide a method for rapidly measuring and estimating the 3D angle of a chip in mid-air based on a mutually guided optical flow estimation model, aiming to achieve high-precision real-time positioning of microchips. This provides technical support for the efficient and stable assembly of microchips.

[0048] The beneficial effects of this invention specifically include:

[0049] (1) In the classic TV-L1 model, the constant brightness assumption and constant gradient assumption can cause many outliers due to changes in illumination and motion occlusion, thus affecting the accuracy of the optical flow vector. Therefore, this invention removes outliers in the flow field by using nonlocal constraint terms based on median smoothing, thereby obtaining an optical flow field with significantly higher energy and higher accuracy. Furthermore, by acquiring optical flow information of the chip surface from different perspectives through multiple optical flow acquisition modules, and combining the reliability weights of different perspectives, an accurate chip offset angle can be calculated.

[0050] (2) This invention constructs a relative structure between the un-median-smoothed flow field and the median-smoothed flow field, and then constructs an optical flow cross-guided filter based on this relative structure. This optical flow cross-guided filter can remove outliers in the flow field while preserving boundary information. Furthermore, due to the randomness of texture and noise, the estimated flow field ψ and the assisted flow field... Inconsistent structures often appear in the texture or noise regions of the flow field. Therefore, the optical flow guide filter can also smooth the texture and noise in the flow field, thereby obtaining a more accurate flow field vector, i.e. a more accurate chip offset angle.

[0051] (3) This invention constructs a color mapping function for the optical flow field, combining multiple mathematical transformations and normalization processes to express the mapping from velocity vectors to color space. By mapping the velocity and direction of motion at each point in the optical flow field to the hue and saturation of colors, a holistic and intuitive expression of the optical flow field is achieved. Compared to observing optical flow vector diagrams, this color mapping method can more comprehensively reflect the overall motion pattern of the scene. Attached Figure Description

[0052] Figure 1 This is a flowchart of the chip 3D angle correction method according to Embodiment 2 of the present invention;

[0053] Figure 2 This is an overall system diagram of the chip 3D angle correction method according to Embodiment 2 of the present invention;

[0054] Figure 3 This is a schematic diagram of the chip 3D motion process of the chip 3D angle correction method according to Embodiment 2 of the present invention;

[0055] Figure 4 This is a visualized dense flow field diagram of the chip 3D angle correction method according to Embodiment 2 of the present invention;

[0056] Figure 5 This is a structural block diagram of the chip 3D angle correction system according to an embodiment of the present invention.

[0057] Figure label:

[0058] U1, Nozzle module; U2, Image sequence acquisition unit; U3, Optical flow modeling and processing unit; P1, Optical flow field dynamics characterization; P2, Optical flow vector field component mapping; 10, Memory; 20, Processor. Detailed Implementation

[0059] To facilitate understanding of the present invention, the present invention will be described more fully and in detail below with reference to the accompanying drawings and preferred embodiments, but the scope of protection of the present invention is not limited to the following specific embodiments.

[0060] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0061] Example 1

[0062] The chip 3D angle correction method provided in Embodiment 1 of the present invention includes:

[0063] Step S101: Acquire chip image sequence during the chip mounting process.

[0064] Step S102: Calculate the optical flow field of the chip image sequence based on the classic TV-L1 variational model to obtain the estimated flow field.

[0065] Step S103: Perform median filtering on the estimated flow field to obtain the assist flow field.

[0066] Step S104: Based on the estimated flow field and the assisted flow field, construct a mutual guided optical flow estimation model.

[0067] Step S105: Calculate the chip offset angle based on the mutual guided optical flow estimation model.

[0068] Step S106: Correct the chip angle according to the chip's deviation angle.

[0069] The chip 3D angle correction method provided in this invention acquires chip image sequences during the chip mounting process; calculates the optical flow field of the chip image sequences based on the classic TV-L1 variational model to obtain an estimated flow field; performs median filtering on the estimated flow field to obtain an assisting flow field; constructs a mutually guided optical flow estimation model based on the estimated flow field and the assisting flow field; calculates the chip offset angle based on the mutually guided optical flow estimation model; and corrects the chip angle based on the chip offset angle. This solves the technical problem that existing technologies cannot accurately detect the angle offset of tiny chips, and can achieve rapid and accurate detection and correction of tiny angle offsets in automated chip assembly processes, thereby improving the stability and reliability of the entire assembly process.

[0070] Example 2

[0071] This invention proposes an innovative method for detecting and correcting angular offsets in microchips, based on multi-view optical flow field modeling and mutual guided filtering technology. First, a high-resolution camera array captures image sequences containing the chip from different angles, and advanced image preprocessing algorithms are used to improve image quality, ensuring the accuracy of subsequent analysis. Next, an optical flow algorithm is used to calculate the motion vector field between the image sequences, constructing an optical flow field model of the chip's motion. This model can monitor and analyze the chip's angular offset in the horizontal and Z-axis directions on the nozzle in real time. When the system detects an angular deviation, it immediately feeds the offset information back to the control system, which then adjusts the specific position of the rotating nozzle to correct the chip's posture and ensure precise alignment. Finally, the system can determine whether the adjusted chip posture is normal or if an offset still exists. This closed-loop control detection and control method, based on optical flow visual perception and feedback adjustment, can achieve rapid and accurate detection and correction of minute angular offsets during automated chip assembly, thereby improving the stability and reliability of the entire assembly process.

[0072] like Figure 1 As shown in the accompanying drawings, the specific implementation method will be described below with reference to the accompanying drawings.

[0073] 1. Figure 2In the process, when the suction nozzle in the suction nozzle module U1 is about to adsorb the chip from the material tray, the image sequence acquisition unit U2 and the optical flow modeling and processing unit U3 are started and initialized. The image sequence acquisition unit U2 is responsible for acquiring image sequence data of the chip during the adsorption process, and the optical flow modeling and processing unit U3 is responsible for analyzing and processing the image data.

[0074] 2. The image sequence acquisition unit U2 is responsible for capturing a continuous image sequence of the chip during the suction process using a high-resolution and high-frame-rate camera. It uses a filtering algorithm to reduce image noise and a contrast enhancement algorithm to improve image visibility and feature recognition. Finally, the processed image sequence Γ={I1,I2,…,I… k The signal is transmitted to the optical flow modeling and processing unit U3.

[0075] 3. The optical flow modeling and processing unit U3 inputs image I into the mutual guided optical flow estimation model of this embodiment of the invention based on the image sequence data. It calculates the horizontal and vertical optical flows u and v of each pixel in the chip image, i.e., the optical flow vectors, through the constructor (10). Then, it performs a weighted average of the optical flow vectors of all relevant pixels to obtain the horizontal offset angle θ of the chip from a single viewpoint. x and the vertical offset angle θ z Furthermore, considering N perspectives, each perspective i provides an angular offset and a reliability weight ω. i The actual offset angle Θ of the chip can be calculated using equation (1).

[0076]

[0077] By evaluating the offset angle Θ, if Θ z If the deviation exceeds a certain threshold, it indicates potential damage to the nozzle or unstable airflow. In this case, an alarm signal should be issued, and the equipment should be paused for inspection and maintenance. If the horizontal offset exceeds a certain threshold, it indicates that the chip's pose needs adjustment. The chip's offset angle Θ should be adjusted accordingly. x The signal is sent to the precisely controllable rotary nozzle module U1. A dynamic model of the rotary nozzle is established, with its moment of inertia as J, its drag coefficient as b, and it subjected to a control torque M. The dynamic equations of the system can be expressed as:

[0078]

[0079] Where θ(t) represents the rotation angle at time t. To achieve precise angle control, a PID controller is designed to calculate the required control torque M. The PID controller operates based on the angle error ε(t) = Θ. x -θ(t) is used to adjust the torque to ensure that the target angle Θ is achieved. xThe control torque M can be calculated using the following equation:

[0080]

[0081] Where K p K i and K d These represent the proportional, integral, and derivative gains of the PID controller, respectively. These parameters need to be optimized through system feedback and adjustments. The precisely controllable rotary nozzle module U1 processes motion information using the aforementioned dynamic equations and controls the nozzle's rotation angle to achieve precise adjustment of the chip's pose.

[0082] 4. To more clearly display the chip's offset, this embodiment of the invention proposes a color mapping function, which combines multiple mathematical transformations and normalization processes to express the mapping from the velocity vector to the color space. The definition is as follows:

[0083] set up It is a velocity vector on a two-dimensional plane, where u(x,y) and v(x,y) are its components in the horizontal and vertical directions, respectively. The direction is mapped onto the hue H, and the direction is expressed through angles. To represent this, normalizing θ to the [0,1] interval maps it to the hue range H [0°, 360°]:

[0084]

[0085] Mapping velocity magnitude to saturation S, velocity magnitude Applying a logarithmic scale to the speed magnitude enhances the distinction between low and high speeds:

[0086]

[0087] Where v max This is the observed maximum velocity, used for normalization. For example... Figure 4 The diagram illustrates a densely visualized flow field. This color map clearly presents the motion information of each point in the optical flow field.

[0088] The following will be combined with the appendix Figure 3 The mutual guidance optical flow estimation model is further explained in detail.

[0089] To obtain accurate optical flow vectors, this embodiment of the invention uses a nonlocal flow field smoothing model based on the variational principle, consisting of a matching term, a constraint term, a nonlocal constraint term, and a coupling term. The matching term measures the consistency or matching degree of pixel brightness in an image sequence; the constraint term imposes constraints on the spatial continuity or smoothness of the optical flow field; and the nonlocal constraint term is used to supplement the flow field. A specific smoothing assumption is imposed within a specified region; the coupling term is used to make the calculated estimated flow field ψ and the auxiliary flow field... Approaching the same.

[0090] The objective function for nonlocal regularization of the flow field based on the variational method is in the form of:

[0091]

[0092] in, E represents the construction function of the mutually guided optical flow estimation model. d (ψ), E s (ψ) and These represent matching terms, constraint terms, nonlocal constraint terms, coupling terms, and mutual guidance constraint terms, respectively. T(p) and T(p+ψ) represent robust penalty functions with the general Charbonnier function, where the symbol δ is a constant, usually set to 0.001. p ) represent pixels p and p+ψ in the texture image, respectively. p The brightness at a certain point, where the texture image is an image obtained based on the texture of the input image I, and p' represents a specified neighborhood D centered at pixel p. p The neighboring pixels adjacent to pixel p, (u p ,v p ) represents the motion vector of pixel p. and Let θ1 and θ2 represent the gradient values ​​of pixel p in the horizontal and vertical directions, respectively. λ is the weight balancing the matching and constraint terms. θ1 and θ2 represent the first and second scalar weights, respectively. and These represent the optical flow components in the horizontal and vertical directions at the corresponding position of the assisted flow field corresponding to pixel p, respectively. p and v p These represent the optical flow components in the horizontal and vertical directions at the corresponding position in the estimated flow field for pixel p, respectively. and ω represents the optical flow components in the horizontal and vertical directions at the corresponding positions of the assisted flow field corresponding to adjacent pixels p', respectively. p,p' ω represents the probability that pixels p and p' belong to the same surface. The weights ω can be defined based on the spatial and color distances between the two pixels to estimate the probability. p,p' ,Right now:

[0093]

[0094] Where I(x,y) represents the input image I at pixel p=(x,y) T The color vectors are set, and ζ1 and ζ2 are set to 7 and 2 respectively.

[0095] In the field of optical flow, traditional methods based on the TV-L1 variational principle can estimate dense flow fields. However, the assumptions of constant brightness and gradient upon which this model relies lack robustness to outliers caused by illumination variations, large-scale displacements, and motion occlusion. This is because these assumptions often do not hold true in complex, dynamically changing real-world scenarios. To address this issue, researchers employed a weighted median filtering strategy to optimize the optical flow field. This method effectively removes outliers by assigning different weights to different pixels, thereby improving the robustness of optical flow estimation. Nevertheless, weighted median filtering may blur edge details in the neighborhood of moving edges, reducing boundary accuracy. To overcome this limitation, this method proposes a novel framework that achieves optimization by constructing both consistent and inconsistent structures between the flow field and the assisting flow field. This strategy allows the flow field and the assisting flow field to maintain overall consistency while suppressing inconsistencies through mutual constraints, resulting in more accurate and robust optical flow estimation. Ultimately, this method significantly improves the performance of optical flow detection while preserving key motion boundaries in the flow field.

[0096] The above variational method-based objective function for nonlocal regularization of the flow field constructs two flow fields of the same size: the estimated flow field ψ and the assisting flow field. The estimated flow field ψ is derived from the classical TV-L1 variational model without median smoothing, thus containing optical flow anomalies but also more information about the moving boundaries, which assists in the flow field estimation. It is generated through median smoothing, so it can remove optical flow anomalies in the estimated flow field ψ, but it also loses motion boundary information. Therefore, embodiments of the present invention define the estimated flow field ψ and the assisted flow field. Consistent and inconsistent structures. ① Consistent structure: At a pixel position p = (x, y) in the optical flow domain Ω. T , and The amplitudes are either sufficiently large or relatively small. ② Inconsistent structure: at a pixel position p = (x, y) under the optical flow domain Ω. T , and One amplitude is large (greater than the preset threshold) while the other is small. It includes (horizontal direction) and First-order partial derivative (in the vertical direction).

[0097]

[0098] in, and Let ψ and ψ represent the first and second expressions in the relative structure expression, respectively. Let ψ represent the estimated flow field and the assisting flow field, respectively. p and These represent the optical flow at the corresponding location in the estimated flow field and the assisted flow field for pixel p in the input image I, respectively. This represents the gradient of the estimated flow field ψ at pixel p in the input image I in the x and y directions. and Indicates assisting flow field The gradient of pixel p in the x and y directions in the input image I. and Where υ∈{x,y}, and Ω represents the optical flow domain.

[0099] For assisting flow field Edge position, relative structure The response amplitudes are all relatively weak; conversely, in the assist flow field... Smooth regions, relative structure The response amplitudes are all quite strong. Based on the relative structure Based on the response characteristics, this method constructs a global objective function to preserve the estimated flow field ψ and the assist flow field. Maintaining a consistent structure and suppressing incompatible structures:

[0100]

[0101] in, The terms represent mutual guiding constraints, where α1, α2, β3, and β4 represent the first, second, third, and fourth equilibrium constants, respectively. and Representing ψ p and The initial optical flow, Represents ψ p and The square of the L2 norm of the difference express and The square of the L2 norm of the difference.

[0102] Authenticity and By constraining ψ and respectively To prevent it from deviating too much from the input estimated flow field ψ 0 and assist flow field This avoids generating trivial solutions. The regularization term uses two different relative structures. and To achieve estimation of the flow field ψ and the assisting flow field The purpose of mutual guidance filtering is to achieve the desired effect, hence the method is called optical flow mutual guidance filter. When estimating the flow field ψ and the assisting flow field... When the structure is consistent at position p, the relative structure and The applied penalty is relatively mild, and if it is at a common location on the edge, it will preserve the edge intensity, giving the optical flow guide filter the ability to preserve edges; when estimating the flow field ψ and the assisting flow field When the two relative structures are incompatible at position p, and One of them will impose a larger penalty. After smoothing with an optical flow guide filter, a flow field without outliers and retaining more information about the motion boundaries will be obtained.

[0103] In summary, this invention first proposes a nonlocal flow field smoothing model based on the variational principle, thereby constructing two flow fields ψ and Secondly, by constructing a relative structure Thus, a global objective function (9) is constructed to preserve the estimated flow field ψ and the assist flow field. The model establishes a consistent structure between components and suppresses incompatible structures. Finally, by introducing the objective function of the mutual guided optical flow filter as a constraint term into the nonlocal flow field smoothing model based on the variational principle, the constructor of the mutual guided optical flow estimation model can be obtained.

[0104]

[0105] To estimate the optical flow field ψ, a multi-scale approach can be used to construct an image pyramid, applying an internal and external fixed-point iterative strategy at each layer of the pyramid. Starting from the top layer (low resolution), a coarse optical flow estimate is first obtained, and this estimate is then used as the initial condition for the next layer (higher resolution). At each layer, the optical flow increment is solved through internal iteration, continuously refining the result until the bottom layer (original resolution) is reached. During this process, the optimal iterative scheme is selected to ensure both computational efficiency and the accuracy of the optical flow estimate. Through this top-down, layer-by-layer refinement method, a precise optical flow field is finally obtained.

[0106] This invention removes outliers in the flow field by using nonlocal constraint terms based on median smoothing, resulting in an optical flow field with significantly higher energy and accuracy. Multiple optical flow acquisition modules acquire chip surface optical flow information from different perspectives, and combining the reliability weights of these perspectives allows for the calculation of a precise chip offset angle. Furthermore, this invention constructs a relative structure between the unsmoothed and median-smoothed flow fields, and then uses this relative structure to build an optical flow cross-guided filter. This cross-guided filter removes outliers while preserving boundary information in the flow field. Moreover, due to the randomness of texture and noise, the estimated flow field ψ and the assisted flow field... Inconsistent structures often appear in the texture or noise regions of the flow field. Therefore, the optical flow guide filter can also smooth the texture and noise in the flow field, thereby obtaining a more accurate flow field vector, i.e. a more accurate chip offset angle.

[0107] Reference Figure 5 The chip 3D angle correction system proposed in this embodiment includes a memory 10, a processor 20, and a computer program stored on the memory 10 and executable on the processor 20. When the processor 20 executes the computer program, it implements the steps of the chip 3D angle correction method proposed in this embodiment.

[0108] The specific working process and working principle of the chip 3D angle correction system in this embodiment can be referred to the working process and working principle of the chip 3D angle correction method in this embodiment.

[0109] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for 3D angle correction of a chip, characterized in that, The method includes: Acquire chip image sequences during the chip mounting process; The optical flow field of the chip image sequence is calculated based on the classical TV-L1 variational model to obtain the estimated flow field. Median filtering is applied to the estimated flow field to obtain the assist flow field; Based on the estimated flow field and the assisted flow field, a mutually guided optical flow estimation model is constructed. This model includes the following steps: Based on the consistent and inconsistent structures between the estimated flow field and the assisting flow field, a relative structure expression is constructed, the specific formula of which is: ， in, and These represent the first and second expressions in the relative structure expression, respectively. and They represent the estimated flow field and the assisting flow field, respectively. and These represent the input images respectively. medium pixel Corresponding to the estimated optical flow at the corresponding location in the assisted flow field, Indicates the estimated flow field Input image medium pixel exist and gradient in direction and , Indicates assisting flow field Input image medium pixel exist and gradient in direction and ,in , Indicates the optical flow domain; Based on the relative structure expression, construct the mutual guiding constraint term, where the specific form of the mutual guiding constraint term is as follows: ， in, This indicates a mutual guiding constraint. , , and These represent the first equilibrium constant, the second equilibrium constant, the third equilibrium constant, and the fourth equilibrium constant, respectively. and They represent and The initial optical flow, express and The square of the L2 norm of the difference express and The square of the L2 norm of the difference; Based on the mutual guidance constraint term, a mutual guidance optical flow estimation model is constructed, wherein the mutual guidance optical flow estimation model is constructed as follows: ， in, This represents the construction function for the mutual guided optical flow estimation model. , , , and These represent matching terms, constraint terms, nonlocal constraint terms, coupling terms, and mutual guidance constraint terms, respectively. Denotes a robust penalty function with a Charbonnier function, denoted by [symbol]. It is a constant, set to 0.

001. and These represent the pixels in the texture image. and The brightness at the location, where the texture image is based on the input image. The image obtained from the texture, Represented in pixels The designated neighborhood centered on Inner and pixel Adjacent adjacent pixels, Represents pixels The motion vector, and Representing pixels The gradient values ​​in the horizontal and vertical directions, It is about balancing the weights of matching terms and constraint terms. and These represent the first scalar weight and the second scalar weight, respectively. Represents pixels With pixels The probability of belonging to the same surface. and Representing pixels The corresponding optical flow components in the horizontal and vertical directions at the corresponding positions of the assisted flow field. and Representing pixels The corresponding optical flow components at the corresponding locations in the flow field are estimated in the horizontal and vertical directions. and Representing adjacent pixels respectively The corresponding optical flow components in the horizontal and vertical directions at the corresponding positions of the assist flow field; The chip offset angle is calculated based on the mutual guided optical flow estimation model. The chip is angle-corrected based on its offset angle.

2. The chip 3D angle correction method according to claim 1, characterized in that, Based on the mutual guided optical flow estimation model, the chip offset angle is calculated as follows: Based on the mutual guided optical flow estimation model, the accurate optical flow field of the chip image sequence is obtained; Based on the precise optical flow field of the chip image sequence, the offset angle of the chip from a single viewpoint is calculated, and the offset angle includes the horizontal offset angle and the vertical offset angle. The actual offset angle of the chip is obtained by weighted calculation of the offset angles of multiple single-view chips.

3. The chip 3D angle correction method according to claim 2, characterized in that, Based on the mutual guided optical flow estimation model, after obtaining the accurate optical flow field of the chip image sequence, the process also includes visualizing the accurate optical flow field, specifically including: The direction of the optical flow velocity vector in the precise optical flow field is mapped to the H hue in the HSV space, using the following formula: ， in, Represents the optical flow velocity vector in a precise optical flow field The direction is mapped to the hue value on the H hue of the HSV space. Indicates the mapped angle, and , and These represent optical flow velocity vectors respectively. Components in the horizontal and vertical directions; The magnitude of the optical flow velocity vector in the precise optical flow field is mapped to the S-saturation in the HSV space, using the following formula: ， in, This represents the saturation value of the S-saturation in the HSV space, which represents the magnitude of the optical flow velocity vector in the precise optical flow field. This indicates the maximum speed observed.

4. The chip 3D angle correction method according to claim 3, characterized in that, Based on the chip's offset angle, angle correction of the chip includes: The actual offset angle of the chip is input into the PID controller, and the PID controller controls the position of the rotating nozzle to be registered with the chip's orientation.

5. A chip 3D angle correction system, the system comprising: The memory (10), the processor (20), and the computer program stored in the memory (10) and executable on the processor (20) are characterized in that the processor (20) implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.

Images