Glass panel pressing multiple positioning and deviation correction method and system based on machine vision

Abstract

The application discloses a glass panel pressing multiple positioning and deviation rectifying method and system based on machine vision, relates to the technical field of pressing positioning and deviation rectifying, and collects a global image, a local enlarged image, a downlink microtexture sequence image and a contact instantaneous optical deformation image to construct a quadruple positioning system: firstly, an initial offset matrix is generated by using global feature matching; then, sub-pixel level edge extraction is realized by frequency domain phase consistency processing to generate a fine deviation rectifying matrix; in the pressing downlink stage, a rotation compensation amount is calculated by using optical flow divergence analysis to generate a rotation compensation matrix; in the moment when a pressing head contacts a glass panel, a dynamic compensation matrix is generated by analyzing the eccentric distance of interference fringes; and a self-adaptive fusion algorithm based on confidence evaluation is used to fuse the quadruple deviation rectifying matrices, and an optimal pose estimation is realized by using a weighted filtering gain formula. The application realizes six-degree-of-freedom high-precision deviation rectifying from macro to micro and from geometry to deformation, and improves the reliability of glass panel pressing.

Description

Technical Field

[0001] This application relates to the field of lamination positioning and correction technology, and in particular to a multi-positioning correction method and system for glass panel lamination based on machine vision. Background Technology

[0002] In the glass panel lamination process, high-precision alignment is crucial for ensuring product yield and preventing screen breakage and bubbles. Existing technologies typically employ a single camera for one-time edge comparison or simple multi-camera shooting at fixed points. However, these industrial vision-based inspection solutions suffer from the following technical problems:

[0003] Features of transparent materials are difficult to extract: Glass panels have high light transmittance and reflectivity. Traditional edge extraction algorithms based on grayscale thresholds are prone to failure under complex lighting conditions, leading to positioning drift.

[0004] Error accumulation and dimension loss: Traditional visual positioning is mostly two-dimensional positioning, which cannot perceive the micro tilt angle of the Z-axis during the pressing process and the dynamic deviation caused by mechanical vibration.

[0005] The lag in static positioning: Existing solutions are mostly open-loop models that go from taking pictures to calculating and then pressing down, which cannot make real-time dynamic adjustments based on physical feedback (such as light spot deformation caused by uneven force) the moment the pressure head contacts the glass panel.

[0006] Data fusion is crude: Existing multi-sensor fusion uses simple averages or fixed weights, lacking real-time evaluation of data source quality. If a link fails due to missing texture, the entire system will collapse. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-positioning and correction method and system for glass panel lamination based on machine vision, so as to overcome the problems of unstable feature extraction of transparent materials, inability to perceive multi-dimensional micro-tilt angles and lack of dynamic closed-loop feedback in the prior art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a machine vision-based multi-positioning and correction method for glass panel lamination, comprising: Step S10: The visual sensor is used to acquire global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base. Step S20: Extract and match feature points from the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. ; Step S30, according to Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. ; Step S40: During the pressing process, calculate the motion vector field of the texture pixels in the micro-texture sequence image, and calculate the rotation compensation amount of the pressure head around the X-axis and around the Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix ; Step S50: At the instant the pressure head contacts the glass panel, calculate the eccentricity of the Newton's rings interference fringes in the optical deformation diagram, and generate the fourth dynamic compensation matrix through the controller. ; Step S60: Construct a state vector containing translation, rotation, head translation speed, and rotation speed, and calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

[0009] Furthermore, step S20 includes the following detailed steps: Step S201: Acquire a global image that simultaneously includes the glass panel to be pressed and the base, and obtain a pre-stored standard template image of the base, in which the origin of the reference coordinate system and the reference direction are marked. Step S202: Extract the corner feature vectors of the base region in the global image, match the corner feature vectors with the feature vectors in the base standard template image using the FLANN algorithm, generate an initial set of matching point pairs, remove mismatched points, and establish the actual reference pose of the base in the global image. Step S203: Extract the contour corner points of the glass panel in the global image, calculate the center coordinates and attitude angle of the glass panel in the global image, subtract the actual reference pose of the base from the center coordinates and attitude angle of the glass panel to obtain the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. .

[0010] Furthermore, step S30 includes the following detailed steps: Step S301, according to Guide the camera to capture a single magnified image containing the corner points of the glass panel and the reference marks on the base. ; Step S302, for Perform a Fourier transform to the frequency domain, and use a Log-Gabor filter bank to calculate the local energy distribution of the image at at least three scales and four directions to generate a phase consistency map. Step S303: Extract local energy peak points in the phase consistency map and determine pixels with peak values ​​greater than a preset consistency threshold as physical edge points; Step S304: Perform least-squares fitting on the physical edge points to generate continuous edge lines, and correct the intersection coordinates of the edge lines using bilinear interpolation to obtain sub-pixel edge coordinates with an accuracy of 0.1 pixels. Step S305: Calculate the rate of change of curvature of continuous edge lines. If the curvature abrupt change exceeds a preset range, it is determined to be noise and removed. Valid sub-pixel edge coordinates are retained for affine transformation to generate... .

[0011] Furthermore, step S40 includes the following detailed steps: Step S401: During the pressing process until the gap is less than 1mm, an image sequence consisting of multiple consecutive frames of micro-texture images is acquired using a camera. ; Step S402, build The image pyramid is constructed by using the Lucas-Kanade optical flow method to calculate the displacement vectors of texture pixels in the X and Y axes layer by layer from the top to the bottom of the image pyramid, forming a motion vector field. Step S403: Calculate the divergence value of the motion vector field. ,in and These are the optical flow vectors in the horizontal and vertical directions, respectively, representing the displacement of a pixel along the X and Y axes; yes right The partial derivatives describe the optical flow in the horizontal direction. With horizontal position The rate of change; yes right The partial derivatives describe the optical flow in the vertical direction. With vertical position The rate of change; Step S404: If the divergence values ​​exhibit a radially divergent distribution originating from the contact center, then the rotational compensation of the indenter around the X and Y axes is calculated using the least squares method. and Generate the third rotation compensation matrix .

[0012] It should be emphasized that after step S202 uses the random sampling consensus algorithm to remove mismatched points, the proportion of the retained inner point set to the initial matching point pair set is calculated; when the proportion of inner points is lower than the preset threshold, the base reference pose matching is determined to be invalid and the global image is reacquired.

[0013] Furthermore, step S50 includes the following detailed steps: Step S501, process the single-frame optical deformation map Adaptive threshold segmentation is performed to extract the boundaries between bright and dark fringes of Newton's rings interference fringes; Step S502: Perform ellipse fitting on the boundary between the bright and dark stripes, and calculate the center coordinates of the fitted ellipse. With the ideal center The offset is used to obtain the eccentricity. ; Step S503, based on the eccentricity A fourth dynamic compensation matrix is ​​generated by using a PID controller to correct the pressure distribution along the Z-axis of the pressure head. ; Step S504: When no closed stripe is detected or the centroid offset of the light spot exceeds the preset limit, the anomaly is determined to be resolved and a re-acquisition is performed, or the following steps are taken: As final compensation.

[0014] Furthermore, step S60 includes the following detailed steps: Step S601: Define the 12-dimensional state vector at the current moment, which includes the complete six-degree-of-freedom pose of the glass panel in three-dimensional space, as well as the sliding and rotational rates in three-dimensional space during the downward movement of the pressure head. Step S602, set the first initial offset matrix The numerical mapping is converted into a state vector. The prior initial values ​​for translation and rotation are determined, and calculations are performed respectively. , and Confidence weight , and : according to Feature point reprojection error during generation Perform calculations when At pixel level, Corresponding confidence weight Otherwise, it decays linearly. Based on optical flow residual norm calculate Corresponding confidence weight ,when hour Otherwise, set to zero; Calculated based on the eccentricity of Newton's rings Corresponding confidence weight When the offset is less than the preset pixel threshold Otherwise, it decays exponentially. Step S603, will , and Substitute into the gain formula of the extended Kalman filter:

[0015] in, This represents the Kalman gain matrix, used to calculate the posterior estimate. Let represent the prior pose covariance matrix, indicating only the previous time step. When the information is received, the system's uncertainty regarding the current pose of the glass panel (the first 6 dimensions of the state vector); Represents the observation matrix. This is a matrix transpose operation; Represents the observation noise covariance matrix. Represents the inverse of a matrix. A diagonal matrix weighted by confidence level; Using the adjusted gain The prior state of the glass panel is corrected and fused. to The optimal posterior pose estimate is calculated and output, and then converted into the final six-degree-of-freedom pressure pose command. Step S604, if , and If any weight is reset to zero, the missing degrees of freedom information of the pose matrix is ​​completed using the singular value decomposition algorithm based on the remaining correction matrix, and a degradation compensation command is output.

[0016] The present invention also provides a machine vision-based multi-positioning and correction system for glass panel lamination, comprising: Multiple image acquisition module: used to acquire global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base through a vision sensor; Initial offset matrix generation module: used to extract and match feature points in the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. ; Sub-pixel correction matrix generation module: used to generate sub-pixel correction matrices based on... Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. ; Rotation compensation matrix generation module: Used to calculate the motion vector field of texture pixels in the micro-texture sequence map during the pressing process, and to calculate the rotation compensation amount of the indenter around the X-axis and Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix ; Dynamic compensation matrix generation module: used to calculate the eccentricity of the Newton's rings interference fringes in the optical deformation pattern at the instant the indenter contacts the glass panel, and generate the fourth dynamic compensation matrix through the controller. ; The multi-positioning and correction calculation module is used to construct a state vector containing translation, rotation, indenter translation speed, and rotation speed, and to calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

[0017] The present invention discloses the following technical effects: This invention provides a machine vision-based multi-positioning correction method and system for glass panel lamination. Addressing the problem of traditional grayscale edge extraction failure caused by glass panel reflection and light transmission in industrial visual inspection, this invention employs a frequency domain consistency algorithm. It calculates local energy peaks in the frequency domain using a Log-Gabor filter bank to extract physical edges unaffected by light intensity, and combines this with bilinear interpolation to improve accuracy. This solves the problem of feature loss in transparent materials in industrial settings and enhances the robustness of glass panel lamination positioning. To address the issue that existing machine vision-based methods can only correct two-dimensional deviations and cannot perceive Z-axis posture, this invention employs optical flow divergence analysis. It calculates the motion vector field of micro-textures using the Lucas-Kanade algorithm and uses divergence formulas to calculate the rotation compensation of the pressure head around the X and Y axes. This enables the system to perceive and compensate for micro-tilt angles during the pressing process in real time, realizing the transformation from planar alignment to full-space six-degree-of-freedom posture control and avoiding glass panel breakage caused by uneven pressing stress. To address system instability caused by conflicts between different types of data, this invention constructs an adaptive extended Kalman filter architecture. By calculating reprojection error and optical flow residuals in real time, confidence weights are dynamically generated and injected into the gain formula for dynamic adjustment. When a certain type of data (such as optical flow) fails, its weights are automatically reset to zero, and singular value decomposition is used to complete the pose information based on the residual matrix. This mechanism achieves intelligent decision-making and improves the system's fault tolerance. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of a machine vision-based multi-positioning and correction method for glass panel lamination, provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a machine vision-based glass panel lamination multi-positioning and correction system provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Example 1: This application provides a machine vision-based multi-positioning and correction method for glass panel lamination, such as... Figure 1 As shown, the method includes: Step S10: The visual sensor acquires global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base.

[0022] In this embodiment, four types of image data are acquired using multiple visual sensors: Global Image A panoramic image is captured using a top-down camera mounted at a high position, simultaneously showing the glass panel to be pressed (held and suspended by a robotic arm) and the base (fixed to the worktable); this image is used for the initial glass panel pressing and alignment.

[0023] magnified image Based on the preset region of interest, a high-precision microscope camera is used to capture close-up images of the corner points of the glass panel and the reference marks on the base.

[0024] Microtexture sequence images: During the lamination process, multiple consecutive images of the microtexture (AG anti-glare coating particles) on the glass panel surface are continuously acquired using a high-speed camera, forming an image sequence. .

[0025] Optical Deformation Map: At the instant of contact between the indenter and the contact surface, the Newton's rings interference fringes are captured by a side-view camera to generate a single-frame optical deformation map. .

[0026] Step S20: Extract and match feature points from the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. .

[0027] In this embodiment, the global image is extracted. The ORB corner feature vectors of the mid-base region are represented in the following form:

[0028] in, Represents the first in the global image The pixel coordinates of the corner points of the base area Indicates the base area. yes The abbreviated form; express The corresponding binary feature descriptor.

[0029] The Fast Nearest Neighbor (FLANN) algorithm is used to compare these ORB corner feature vectors with the pre-stored base standard template image features. Perform a match. This represents a standard template image of the base. The set of feature vectors extracted from the binary description above. Indicates the first image on the template image Descriptors for each base area corner point; for each Search The nearest neighbor is used to obtain the initial set of matching point pairs through a distance ratio test. , This indicates the template image on the first... The pixel coordinates of the corner points of the base area, and , This represents the number of point pairs in the initial set of matched point pairs; the homogeneous form of each point pair is... , , and These represent the homogeneous coordinates of corner points in the global image and the template image, respectively.

[0030] by As input, the 3×3 homography matrix under the local plane approximation of the base is estimated using Random Sample Consensus (RANSAC). To satisfy:

[0031] in, Representation matrix The element at the corresponding position; based on the reprojection residual: The criteria for determining the set of interior points are as follows: , This represents the preset reprojection parameter threshold, which is set to 2 pixels in this embodiment; thus, the set of interior points is obtained. The number of interior points is ;When the interior point ratio (In this implementation) When the baseline mismatch is 30%, the base reference mismatch is determined and a re-acquisition is triggered. .

[0032] Homography matrix Approximate decomposition yields the rotation matrix. Translation components :

[0033] in, The result is approximately an identity matrix; This indicates the reference attitude angle of the base in the current drawing (relative to the reference direction of the template); and These represent the x- and y-direction offsets of the base reference origin in the global image, respectively; further calculations yield... , The arctangent function is defined in four quadrants, and the pixel coordinates of the base reference origin in the global image are calculated. :

[0034] in, Represents the origin pixel coordinates (homogeneous form) in the reference template image; and These represent the x and y coordinates of the base reference origin in the global image, respectively.

[0035] In the same sheet In the process, extract the set of points representing the outer contour of the glass panel within the non-base area. The center coordinates of the glass panel are calculated from the profile. With attitude angle (Calculated from the principal axis of the glass panel outline); The pixel offset is obtained by translating and rotating the coordinates relative to the base reference origin to the base reference coordinate system. and :

[0036] Multiply by the calibration factor Convert to physical offset and : Relative rotation Recorded as Finally, the first initial offset matrix is ​​obtained by assembly. : and send it to the next step as , and The guidance and prior reference input of the state vector.

[0037] Step S30, according to Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. .

[0038] In this embodiment, according to The provided initial offset guides the high-precision camera to acquire a single magnified image. ,right Perform a Fast Fourier Transform to the frequency domain to obtain the complex spectrum. , and It is a frequency domain unit; secondly, it utilizes Log-Gabor filter banks at three scales. and four directions The spectrum is then subjected to convolution filtering: ,in Indicated in scale and direction Local energy response on This represents the inverse fast Fourier transform. It is a Log-Gabor filter core.

[0039] Calculate the integrated local energy distribution across all scales and directions to generate a phase-consistent spectrum. This value reflects the phase alignment of local waveform features and is insensitive to illumination intensity. The formula for generating the phase consistency map is as follows:

[0040] in, Indicates the corresponding scale and direction phase angle, For weighted average phase, To prevent small constants from being divided by zero, pixels with peak values ​​greater than a preset consistency threshold in the graph are extracted and identified as physical edge point sets. .

[0041] For physical edge point set The points in the graph are fitted with least squares to generate continuous edge lines. In order to obtain a precision beyond the pixel level, bilinear interpolation is used to correct the coordinates of the intersection points of the edge lines to obtain corrected sub-pixel coordinates. The coordinate precision is corrected to 0.1 pixels. At the same time, the curvature of the edge lines is calculated, and abrupt noise points with a curvature greater than a preset threshold are removed.

[0042] Corrected subpixel edge coordinates coordinates with pre-stored CAD standard template To perform the correspondence, the affine transformation matrix is ​​calculated using the least squares method. This matrix describes the mapping from the template coordinate system to the locally magnified image coordinate system (including translation, rotation, shearing, and non-uniform scaling); from linear part Extract the rotation component and calculate the minute rotation relative to the base reference. : ; Simultaneously, subpixel edge coordinates Subtract the base reference origin And via the base reference rotation matrix After alignment, multiply by the calibration factor. The local translation amount relative to the base reference is obtained. Finally, the second sub-pixel correction matrix is ​​generated. .

[0043] Step S40: During the pressing process, calculate the motion vector field of the texture pixels in the micro-texture sequence image, and calculate the rotation compensation amount of the pressure head around the X-axis and around the Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix .

[0044] In this embodiment, as the pressure head descends to a distance of less than 1 mm, the system uses a high-speed camera to acquire a sequence of micro-texture images consisting of multiple consecutive frames. To handle motion at different scales, the micro-texture images are segmented, and an image pyramid containing micro-texture images of various resolutions is constructed. An improved Lucas-Kanade optical flow method is then used to calculate the displacement vectors of texture pixels layer by layer from the top to the bottom of the pyramid. This forms the motion vector field of the current frame relative to the reference frame, where and These are the optical flow vectors in the horizontal and vertical directions, respectively, representing the displacement of the pixel along the X and Y axes, and:

[0045] in, This represents the image resolution index in the image pyramid. Indicates the first The generalized inverse of the gradient matrix of an image at various resolutions. and These represent the image gradients in the x and y directions, respectively; This represents the grayscale difference between two adjacent frames.

[0046] Perform partial differential operations on the moving vector field to calculate the divergence of the field in two-dimensional space. Divergence reflects whether the texture diffuses outward (positive divergence, indicating that the indenter tilt causes the contact area to expand) or contracts inward:

[0047] in, express right The partial derivatives, express right The partial derivatives of .

[0048] If the divergence field exhibits a contact center Radial divergence distribution starting from (i.e.) If the value increases with increasing radius, then the pressure head is determined to have a slight tilt angle; by fitting the relationship between the divergence distribution and spatial coordinates using the least squares method, the rotational compensation of the pressure head around the X and Y axes is calculated. and :

[0049] in, As calibration coefficients, pixel divergence is converted into physical rotation angles. The calculated rotation compensation amounts are combined into a third rotation compensation matrix. Simultaneously calculate optical flow residuals norm If the value exceeds the preset threshold (0.1 in this embodiment), the confidence weight is reset to zero in the subsequent step S60, and a downgrade process is triggered.

[0050] Step S50: At the instant the pressure head contacts the glass panel, calculate the eccentricity of the Newton's rings interference fringes in the optical deformation diagram, and generate the fourth dynamic compensation matrix through the controller. .

[0051] In this embodiment, a single frame of optical deformation image is captured the instant the pressure head contacts the glass panel. ,right Adaptive threshold segmentation is performed to divide the image into bright and dark fringe regions. The boundary point set of the bright and dark fringe is extracted from the Newton's rings interference fringes. Least square ellipse fitting is then performed on this boundary point set to obtain the center coordinates of the fitted ellipse. Calculate the distance between the center and the ideal circle center. The Euclidean distance, i.e., the eccentricity. :

[0052] Based on the magnitude of the eccentricity, a PID controller calculates the dynamic compensation amount used to correct the pressure distribution or posture of the indenter along the Z-axis. This is manifested in the nonlinear increase of the compensation amount as the eccentricity increases:

[0053] in, Indicates a time index; , and These are proportional gain, integral gain, and derivative gain, respectively. The magnitude of the compensating thrust is determined by how much the pressure head deviates. This represents the cumulative penalty for a prolonged period where the eccentricity is not zero. This indicates the rate of response to changes in eccentricity. This represents the rate of change of eccentricity, i.e., the error derivative; the compensation amounts are combined into a fourth dynamic compensation matrix. .

[0054] Simultaneously, calculate the optical deformation map of a single frame. offset of the centroid of the light spot If no closed fringes are detected (interference failure) or If the preset limit is exceeded, an abnormal contact is determined; The confidence weights are forced to zero, and only fusion is performed in step S60. , and .

[0055] Step S60: Construct a state vector containing translation, rotation, head translation speed, and rotation speed, and calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

[0056] In this embodiment, a 12-dimensional state vector containing translation, rotation, and velocity at the current moment is constructed. and will The provided macroscopic offset mapping is the prior initial value of the pose component in the state vector, while the remaining velocity components are set to zero; the state vector is represented in the following form:

[0057] The first 6 dimensions of the state vector represent the pose vector, and the last 6 dimensions represent the velocity components. , and These represent the horizontal and vertical positions of the glass panel center relative to the base reference, as well as the pressing depth of the pressure head. and These represent the slight tilt angles of the glass panel in the head-to-tail and left-to-right directions, respectively. In and Dynamic correction For the in-plane corner of the glass panel, by of initialization, of Fine-tuning; , and These represent the translational and downward velocities in the X, Y, and Z directions, respectively. , and These represent the angular velocities of the indenter around the X, Y, and Z axes, respectively. The prior injection process is represented by the following formula:

[0058] in, and It is the orientation mapping coefficient calibrated by the camera intrinsic parameters that converts image offset into world-frame translation.

[0059] Calculate separately , and The confidence weights include: according to Feature point reprojection error during generation Perform calculations when Pixels )hour, Corresponding confidence weight Otherwise, it decays linearly, as expressed by the formula:

[0060] in, This represents the function that takes the maximum value. This represents a preset threshold constant, which is set empirically.

[0061] Based on optical flow residual norm calculate Corresponding confidence weight ,when hour Otherwise, set to zero, expressed by the formula:

[0062] Based on the eccentricity of Newton's rings calculate Corresponding confidence weight When the eccentricity Less than the preset pixel threshold hour Otherwise, it decays exponentially, as expressed by the formula:

[0063] in, A value greater than 0 indicates an exponential decay coefficient, which is obtained through calibration.

[0064] Will , and Substituting the gain formula of the extended Kalman filter, and using... , and Dynamically scale the observation noise covariance:

[0065] in, This represents the Kalman gain matrix, used to calculate the posterior estimate. Let represent the prior pose covariance matrix, indicating only the previous time step. When the information is received, the system checks the current pose of the glass panel ( The uncertainty of the first 6 dimensions; Represents the observation matrix. This is a matrix transpose operation; Represents the observation noise covariance matrix. Represents the inverse of a matrix. A confidence-weighted diagonal matrix; using the adjusted Kalman gain. For prior states Make corrections:

[0066] in For the reason , and The mapped observation vector is estimated from the optimal posterior. The first 6 dimensions are calculated and the final six-degree-of-freedom pressing pose command is output, completing the quadruple positioning and correction of the glass panel pressing.

[0067] like , and If any weight is reset to zero, the missing degrees of freedom information of the pose components are completed using the singular value decomposition (SVD) algorithm based on the remaining correction matrix, and a degradation compensation command is output.

[0068] Example 2: The machine vision-based glass panel lamination multi-positioning correction system provided in this embodiment of the invention can execute the machine vision-based glass panel lamination multi-positioning correction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method, such as... Figure 2 As shown, it includes: Multiple image acquisition module: used to acquire global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base through a vision sensor; Initial offset matrix generation module: used to extract and match feature points in the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. ; Sub-pixel correction matrix generation module: used to generate sub-pixel correction matrices based on... Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. ; Rotation compensation matrix generation module: Used to calculate the motion vector field of texture pixels in the micro-texture sequence map during the pressing process, and to calculate the rotation compensation amount of the indenter around the X-axis and Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix ; Dynamic compensation matrix generation module: used to calculate the eccentricity of the Newton's rings interference fringes in the optical deformation pattern at the instant the indenter contacts the glass panel, and generate the fourth dynamic compensation matrix through the controller. ; The multi-positioning and correction calculation module is used to construct a state vector containing translation, rotation, indenter translation speed, and rotation speed, and to calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

[0069] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0070] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A machine vision-based multi-positioning and correction method for glass panel lamination, characterized in that, The method includes: Step S10: The visual sensor is used to acquire global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base. Step S20: Extract and match feature points from the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. ; Step S30, according to Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. ; Step S40: During the pressing process, calculate the motion vector field of the texture pixels in the micro-texture sequence image, and calculate the rotation compensation amount of the pressure head around the X-axis and around the Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix ; Step S50: At the instant the pressure head contacts the glass panel, calculate the eccentricity of the Newton's rings interference fringes in the optical deformation diagram, and generate the fourth dynamic compensation matrix through the controller. ; Step S60: Construct a state vector containing translation, rotation, head translation speed, and rotation speed, and calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

2. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 1, characterized in that, Step S20 includes the following detailed steps: Step S201: Acquire a global image that simultaneously includes the glass panel to be pressed and the base, and obtain a pre-stored standard template image of the base, in which the origin of the reference coordinate system and the reference direction are marked. Step S202: Extract the corner feature vectors of the base region in the global image, match the corner feature vectors with the feature vectors in the base standard template image using the FLANN algorithm, generate an initial set of matching point pairs, remove mismatched points, and establish the actual reference pose of the base in the global image. Step S203: Extract the contour corner points of the glass panel in the global image, calculate the center coordinates and attitude angle of the glass panel in the global image, subtract the actual reference pose of the base from the center coordinates and attitude angle of the glass panel to obtain the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. .

3. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 2, characterized in that, In step S202, after removing mismatched points using the random sampling consensus algorithm, the proportion of the retained set of interior points to the initial set of matched points is calculated. When the proportion of interior points is lower than a preset threshold, the base reference pose matching is determined to be invalid, and global image re-acquisition is triggered.

4. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 1, characterized in that, Step S30 includes the following detailed steps: Step S301, according to Guide the camera to capture a single magnified image containing the corner points of the glass panel and the reference marks on the base. ; Step S302, for Perform a Fourier transform to the frequency domain, and use a Log-Gabor filter bank to calculate the local energy distribution of the image at at least three scales and four directions to generate a phase consistency map. Step S303: Extract local energy peak points in the phase consistency map and determine pixels with peak values ​​greater than a preset consistency threshold as physical edge points; Step S304: Perform least-squares fitting on the physical edge points to generate continuous edge lines, and correct the intersection coordinates of the edge lines using bilinear interpolation to obtain sub-pixel edge coordinates with an accuracy of 0.1 pixels. Step S305: Calculate the rate of change of curvature of continuous edge lines. If the curvature abrupt change exceeds a preset range, it is determined to be noise and removed. Valid sub-pixel edge coordinates are retained for affine transformation to generate... .

5. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 1, characterized in that, Step S40 includes the following detailed steps: Step S401: During the pressing process until the gap is less than 1mm, an image sequence consisting of multiple consecutive frames of micro-texture images is acquired using a camera. ; Step S402, build The image pyramid is constructed by using the Lucas-Kanade optical flow method to calculate the displacement vectors of texture pixels in the X and Y axes layer by layer from the top to the bottom of the image pyramid, forming a motion vector field. Step S403: Calculate the divergence value of the motion vector field. ,in and These are the optical flow vectors in the horizontal and vertical directions, respectively, representing the displacement of a pixel along the X and Y axes; yes right The partial derivatives describe the optical flow in the horizontal direction. With horizontal position The rate of change; yes right The partial derivatives describe the optical flow in the vertical direction. With vertical position The rate of change; Step S404: If the divergence values ​​exhibit a radially divergent distribution originating from the contact center, then the rotational compensation of the indenter around the X and Y axes is calculated using the least squares method. and Generate the third rotation compensation matrix .

6. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 1, characterized in that, Step S50 includes the following detailed steps: Step S501, process the single-frame optical deformation map Adaptive threshold segmentation is performed to extract the boundaries between bright and dark fringes of Newton's rings interference fringes; Step S502: Perform ellipse fitting on the boundary between the bright and dark stripes, and calculate the center coordinates of the fitted ellipse. With the ideal center The offset is used to obtain the eccentricity. ; Step S503, based on the eccentricity A fourth dynamic compensation matrix is ​​generated by using a PID controller to correct the pressure distribution along the Z-axis of the pressure head. ; Step S504: When no closed stripe is detected or the centroid offset of the light spot exceeds the preset limit, the anomaly is determined to be resolved and a re-acquisition is performed, or the following steps are taken: As final compensation.

7. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 1, characterized in that, Step S60 includes the following detailed steps: Step S601: Define the 12-dimensional state vector at the current moment, which includes the complete six-degree-of-freedom pose of the glass panel in three-dimensional space, as well as the sliding and rotational rates in three-dimensional space during the downward movement of the pressure head. Step S602, set the first initial offset matrix The numerical mapping is converted into a state vector. The prior initial values ​​for translation and rotation are determined, and calculations are performed respectively. , and Confidence weight , and ; Step S603, will , and Substituting the gain formula of the extended Kalman filter, and using the adjusted gain... The prior state of the glass panel is corrected and fused. to The optimal posterior pose estimate is calculated and output, and then converted into the final six-degree-of-freedom pressure pose command. Step S604, if , and If any weight is reset to zero, the missing degrees of freedom information of the pose matrix is ​​completed using the singular value decomposition algorithm based on the remaining correction matrix, and a degradation compensation command is output.

8. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 7, characterized in that, The gain formula for the extended Kalman filter is expressed as: in, This represents the Kalman gain matrix, used to calculate the posterior estimate. Let represent the prior pose covariance matrix, indicating only the previous time step. When providing information, the system's uncertainty regarding the current pose of the glass panel; Represents the observation matrix. This is a matrix transpose operation; Represents the observation noise covariance matrix. Represents the inverse of a matrix. A diagonal matrix weighted by confidence level.

9. The machine vision-based multi-positioning and correction method for glass panel lamination as described in claim 7, characterized in that, In step S602, calculate respectively , and The confidence weights include: according to Feature point reprojection error during generation Perform calculations when At pixel level, Corresponding confidence weight Otherwise, it decays linearly; based on the optical flow residual. norm calculate Corresponding confidence weight ,when hour Otherwise, set to zero; calculate based on the eccentricity of the Newton's rings. Corresponding confidence weight When the offset is less than the preset pixel threshold Otherwise, it decays exponentially.

10. A multi-positioning and correction system for glass panel lamination based on machine vision, characterized in that, The system is used to implement the machine vision-based multi-positioning and correction method for glass panel lamination as described in any one of claims 1-9, and the system comprises: Multiple image acquisition module: used to acquire global images, magnified local images, micro-texture sequence images during the downward process, and optical deformation images at the moment of contact between the glass panel to be pressed and the base through a vision sensor; Initial offset matrix generation module: used to extract and match feature points in the global image, calculate the translation and rotation of the glass panel relative to the base, and generate the first initial offset matrix. ; Sub-pixel correction matrix generation module: used to generate sub-pixel correction matrices based on... Guided acquisition of locally magnified images, frequency domain phase consistency processing is performed on the magnified images to extract sub-pixel edge coordinates, and a second sub-pixel correction matrix is ​​generated through affine transformation. ; Rotation compensation matrix generation module: Used to calculate the motion vector field of texture pixels in the micro-texture sequence map during the pressing process, and to calculate the rotation compensation amount of the indenter around the X-axis and Y-axis based on the divergence distribution of the motion vector field. and Generate the third rotation compensation matrix ; Dynamic compensation matrix generation module: used to calculate the eccentricity of the Newton's rings interference fringes in the optical deformation pattern at the instant the indenter contacts the glass panel, and generate the fourth dynamic compensation matrix through the controller. ; The multi-positioning and correction calculation module is used to construct a state vector containing translation, rotation, indenter translation speed, and rotation speed, and to calculate the following respectively. , and The corresponding confidence weights are substituted into the extended Kalman filter formula to dynamically adjust the gain and fuse the results. to Solve and output the final six-degree-of-freedom compression pose command.

Images