A method for high-precision calibration of rapid recognition of camera torsion

Abstract

The present invention discloses a method for high-precision calibration of rapid identification of camera torsion, and the present invention relates to the technical field of industrial camera calibration. The method for high-precision calibration of rapid identification of camera torsion specifically includes the following steps: S1 Layout of calibration points and dynamic perception; S2 Expansion of mathematical model and error compensation; S3 Improvement of algorithm and data processing; By adopting the method of 9-point calibration, the camera error caused by the torque deformation of the camera axis during each operation can be detected and calculated, and combined with real-time monitoring of parameters such as temperature, strain, and jitter, the calibration process can be adaptively triggered, and the deformation of the camera installation axis can be timely detected and dealt with, improving the calibration accuracy; By constructing a time-varying model and an affine transformation matrix, polynomial or B-spline curve fitting is automatically selected for non-linear error compensation, and Fourier transform and adaptive filters are used to process periodic deformation and interference, effectively reducing the error caused by external factors and ensuring accurate and stable installation of the camera.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial camera calibration, and specifically provides a method for quickly identifying high-precision calibration of camera torsion. Background Art

[0002] Through calibration, the industrial camera of the chip mounter can accurately identify the Mark points (reference points) on the PCB board and the positions of components, thereby achieving high-precision component placement. The calibration process eliminates factors such as distortion and error in the camera imaging process, ensuring the accurate correspondence between the camera coordinate system and the physical world coordinate system. A camera that has not been accurately calibrated may cause deviation in the placement position, affecting product quality. Through calibration, such errors can be significantly reduced, and the placement accuracy and consistency can be improved.

[0003] Since the chip mounter positions the product through the industrial camera during the processing of the product, whether the industrial camera can accurately position the product is related to the accuracy of the operation. For example, a shielding cover chip mounter control system and method disclosed in Publication No. CN108243600A includes an industrial main board, and a chip head control module, an XY-axis motion control module, a feeding control module, and an identification camera connected to the industrial main board through a communication channel. The identification camera includes a Mark point camera, a fixed camera, and an external camera. The industrial main board is also connected to an alarm detection module. This shielding cover chip mounter control system utilizes the modular concept. Dedicated PCB boards are provided on the chip head control module, the XY-axis motion control module, and the feeding control module to improve the data processing ability and transportation accuracy of the entire shielding cover chip mounter placement system. By setting a Mark point camera and fixed cameras and external cameras with different precisions, the system image recognition technology is improved to achieve the purpose of stable, accurate, and fast placement of the shielding cover chip mounter.

[0004] However, as shown in the above technology, it improves the accuracy by loading multiple cameras in cooperation, which is a temporary solution. In actual use, due to factors such as the rise in equipment temperature and equipment vibration, the camera mounting axis will deform, resulting in errors in the camera itself. Therefore, at this time, only improving the accuracy of image recognition cannot solve the recognition errors caused by the errors of the camera itself. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a method for quickly identifying high-precision calibration of camera torsion, which solves the problem of recognition errors caused by the deformation of the camera's own mounting axis.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for quickly identifying high-precision calibration of camera torsion specifically includes the following steps:

[0007] S1 Fixed Point Layout and Dynamic Sensing: Layout fixed points, determine the initial angle between the camera coordinate system and the fixed point coordinate system by scanning the fixed points with a camera. After operation, scan again to determine the secondary angle. Set an adaptive calibration frequency to automatically trigger the calibration process by monitoring temperature, structural strain, and camera jitter parameters in real time;

[0008] S2 Mathematical Model Expansion and Error Compensation: Construct a time-varying model and an affine transformation matrix, decompose the torsional angle θ into static installation errors and dynamic thermal deformations, and introduce translation errors and scaling factors to describe the composite deformation of the camera installation axis; According to the spectral characteristics and change trends of the data, automatically select polynomial fitting or B-spline curve fitting for non-linear error compensation to reduce errors caused by external factors; Identify periodic deformation patterns through Fourier transform and use an adaptive filter to suppress specific frequency interference;

[0009] S3 Algorithm and Data Processing Improvement: Combine IMU or laser rangefinder data to fuse multi-source information through extended Kalman filtering, improving the real-time performance and noise resistance of angle estimation; Use deep learning-assisted calibration, directly output the torsional angle through a neural network for torsional angle compensation and threshold determination to trigger the calibration process; Use synthetic data augmentation and transfer learning to improve the generalization ability of the model; Adopt an incremental algorithm to reduce calculation latency; Regularly perform global recalibration to avoid cumulative errors.

[0010] Preferably, the S1 fixed point layout and dynamic sensing specifically include:

[0011] S1.1 Fixed Point Layout Design:

[0012] Global Fixed Points: Set multiple basic points covering the entire calibration plate range to calculate coarse-grained deformation parameters; When the fixed points are photographed by the camera for the first time, calculate the angle θ1 between the camera coordinate system and the fixed point coordinate system through the coordinates of the fixed points;

[0013] S1.2 Adaptive Calibration Frequency:

[0014] Real-time Monitoring: Take another photo after operation. At this time, the camera installation axis will be deformed and twisted due to temperature, structural strain, and jitter. Calculate the angle θ2 between the camera coordinate system and the fixed point coordinate system again through the coordinates of the fixed points; Integrate a temperature sensor or strain gauge on the camera installation axis to monitor the temperature field or structural strain; At the same time, integrate an acceleration sensor near the camera installation position to monitor the camera jitter parameters in real time;

[0015] Dynamic Trigger: When the temperature change exceeds the threshold, the strain exceeds the set value, or the camera jitter parameter exceeds the threshold, automatically trigger the calibration process.

[0016] Preferably, the basic points cover the entire calibration plate range, and the number of global fixed points is set to N global= 9, calculating the coarse-grained deformation parameters through the global calibration points, including the overall rotation angle θ global and the translation amounts (tx global , ty global ), where tx global and ty global represent the translation amounts on the X-axis and Y-axis respectively;

[0017] In the adaptive calibration frequency, assuming the temperature field is T, the structural strain is ε, the camera jitter amplitude is A, and the frequency is f, when , or |ε| > ε′, or A > A′, or f exceeds [f min , f max , the calibration process is automatically triggered, where T0 is the initial temperature, ΔT′ is the temperature change threshold, ε′ is the structural strain threshold, A′ is the jitter amplitude threshold, and f min and f max are the upper and lower limits of the frequency threshold respectively.

[0018] Preferably, the S2 mathematical model expansion and error compensation specifically include:

[0019] S2.1 Composite deformation modeling:

[0020] Time-varying model: Decompose the torsion angle θ into static installation error and dynamic thermally induced deformation , and establish the model;

[0021] Affine transformation matrix: Introduce translation errors (dx, dy), scaling factor k, and construct a complete affine transformation matrix;

[0022] S2.2 Nonlinear error compensation:

[0023] Automatic model selection fitting:

[0024] Polynomial fitting: Perform quadratic / cubic polynomial fitting on to describe the long-term thermal deformation trend;

[0025] B-spline curve: In high-frequency vibration scenarios, use B-spline curve fitting for nonlinear changes; Similarly, the automatic model selection algorithm automatically selects B-spline curve fitting according to data characteristics when detecting complex changes such as high-frequency vibration;

[0026] S2.3 Frequency domain analysis:

[0027] Fourier transform: Perform FFT on the θ sequence of multiple calibrations to identify periodic deformation patterns;

[0028] Adaptive Filter: An adaptive filter is adopted to dynamically adjust the parameters of the filter according to the real-time collected data, so as to better suppress interference while retaining useful signals.

[0029] Preferably, the affine transformation matrix is: ;

[0030] The polynomial fitting is to analyze the spectral characteristics and change trends of the data. If the data shows a relatively smooth long-period change, it is automatically determined to use polynomial fitting. Let the polynomial fitting function of Δθ(t) be , where the order n = 2 or 3, (b n , b n-1 ,..., b1, b0) are the coefficients of the polynomial. The coefficients are solved by the least squares method to minimize the error between the fitting function P(t) and ; By analyzing the spectral characteristics S(ω) and change trends C(t) of the data, if S(ω) shows a relatively smooth long-period characteristic and C(t) changes slowly, polynomial fitting is automatically selected; The polynomial coefficients b j are solved by the least squares method, and j ∈ (1, 2,..., n), so that is minimized;

[0031] In the B-spline curve step, let the control points of the B-spline curve be , and the knot vector be , where the number of control points ranges from 0 to m, a total of m + 1. n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve;

[0032] In the adaptive filter, let the input signal of the adaptive filter be x(n'), the desired signal be d(n'), the output signal be y(n'), and the error signal be ; Dynamically adjust the filter parameter w(n') according to the real-time collected data to make minimized, where represents the mathematical expectation.

[0033] Preferably, the improvement of the S3 algorithm and data processing specifically includes:

[0034] S3.1 Heterogeneous Data Fusion:

[0035] Multi-source Information Fusion: Combine the data of the inertial measurement unit IMU or the laser rangefinder, and fuse multi-source information through the extended Kalman filter;

[0036] S3.2 Deep Learning-assisted Calibration:

[0037] Neural Network Training: Use the CNN or Transformer network, input the calibration board image feature I, and directly output the torsion angle θ;

[0038] Synthetic data augmentation: Generate synthetic data under different working conditions through simulation, and combine the measured data to train the model;

[0039] Transfer learning: For different types of pick-and-place machines or working conditions, fine-tune the model parameters with new data;

[0040] S3.3 Real-time dynamic calibration:

[0041] Incremental algorithm: Use the previous calibration result as a prior, and update the θ value only through 3 - 4 new calibration points;

[0042] Regular global recalibration: To avoid cumulative errors in the incremental algorithm after long-term operation, perform regular global recalibration to eliminate cumulative errors;

[0043] S3.4 Angle calculation and result application:

[0044] After obtaining the coordinates of the calibration points in the first shooting and after work, calculate the included angles θ1 and θ2 between the camera coordinate system and the calibration point coordinate system using the coordinates, and obtain the torsion angle of the camera mounting axis by calculating the difference between θ2 and θ1, which is used for error compensation to improve the accuracy of camera calibration. And when the torsion angle exceeds the torsion angle threshold, trigger the recalibration process.

[0045] Preferably, in the S3.1 heterogeneous data fusion, the IMU data includes the angular velocity ω and the angular acceleration ω˙, and the laser rangefinder data is represented as d laser ; Let the state vector be ; The observation vector is

[0046] , T represents the transpose operation; The state transition matrix is , Δt is the time step; The observation matrix is , where h1 and h2 are determined according to the specific observation model;

[0047] In the S3.2 deep learning assisted calibration, let the loss function of the neural network be , where, θ pred is the predicted value, θ true is the true value, R is the total number of samples, r ∈ (1, 2,..., R);

[0048] In transfer learning, let the original model parameters be w original , the new data is , the fine-tuned model parameters are w new , Q new is the number of samples of the new data, and ;

[0049] In the S3.3 real-time dynamic calibration, let the previous calibration result be θ prev , the new calibration point is , c ∈ 1 to C, and C is 3 or 4, and the θ value is updated by the least squares method;

[0050] In the periodic global recalibration, let the running time threshold be t′ and the measurement task quantity threshold be N′. When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, a full calibration is performed.

[0051] Preferably, it is characterized in that: the calibration points are designed with non-uniform distribution to enhance the ability to capture non-linear deformation. Let the distance between calibration points in the edge area of the calibration plate be d edge , and the distance between calibration points in the central area be d center ; assume that the length of the edge area of the calibration plate is L edge , and the length of the central area is L center , then the number of calibration points in the edge area , the number of calibration points in the central area .

[0052] Preferably, on the basis of the original 9 calibration points, redundant calibration points are added in the key edge areas. Let the original number of calibration points be N0 = 9, and N1 corner redundant points are added. Abnormal points are removed through a one-time verification algorithm to obtain the optimal model parameters.

[0053] Preferably, the steps of removing abnormal points through the one-time verification algorithm specifically include: for a set of calibration point data , randomly select a part of the points as a subset from it, use the selected subset points to fit the model, and calculate the number of inliers that conform to the model among the other points according to the set error threshold δ; repeat the above steps, and select the model with the largest number of inliers as the optimal model. After the optimal model is determined, remove the abnormal points that do not conform to the model.

[0054] The present invention provides a method for high-precision calibration of rapid identification of camera torsion. Compared with the prior art, it has the following beneficial effects:

[0055] 1. In the first embodiment of the present invention, by adopting a 9-point calibration method, the camera error caused by the torque deformation of the camera axis during each operation can be detected and calculated. Combining with real-time monitoring of parameters such as temperature, strain, and jitter, the calibration process can be adaptively triggered, and the deformation of the camera installation axis can be detected and addressed in a timely manner, improving the calibration accuracy. By constructing a time-varying model and an affine transformation matrix, polynomial or B-spline curve fitting is automatically selected for non-linear error compensation. Periodic deformation and interference are processed using Fourier transform and adaptive filters, effectively reducing errors caused by external factors. By combining IMU or laser rangefinder data through extended Kalman filter to fuse multi-source information, deep learning is used to assist calibration, synthetic data augmentation and transfer learning are adopted to improve the generalization ability of the model, and incremental algorithms are used to reduce calculation latency and perform periodic global recalibration, further improving the real-time performance and noise resistance of calibration, avoiding cumulative errors, and ensuring accurate and stable camera installation.

[0056] 2. In the second embodiment of the present invention, the calibration points are non-uniformly distributed, with a small spacing and a large number of points in the edge region, and a large spacing and a small number of points in the central region. This further enhances the ability to capture non-linear deformation, enabling the calibration to more accurately reflect the deformation conditions of different regions of the calibration board, improving the calibration accuracy, and is particularly suitable for complex deformation scenarios.

[0057] 3. In the third embodiment of the present invention, redundant calibration points are added in the key edge regions based on the original 9 calibration points, and outlier points are removed through a one-time verification algorithm. This increases the number and distribution rationality of the calibration points, enabling more comprehensive capture of deformation information. At the same time, this algorithm can effectively identify and remove abnormal data, reducing its impact on model parameters, making the obtained model parameters more accurate and reliable, further improving the accuracy and stability of camera calibration, and enhancing the credibility of the calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a schematic flow chart of the steps of the present invention;

[0059] Figure 2 is a schematic diagram of the torsion of the camera installation axis of the present invention;

[0060] Figure 3 is a schematic diagram of the deflection of the calibration points in two scans in the first embodiment of the present invention;

[0061] Figure 4 is a schematic diagram of the calibration points in the second embodiment of the present invention;

[0062] Figure 5 is a schematic diagram of the calibration points in the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0063] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0064] Refer to Figures 1 - 5 , the present invention provides the following three technical solutions:

[0065] The first implementation manner: A method for high-precision calibration of rapid recognition of camera torsion, specifically including the following steps:

[0066] S1 Layout of calibration points and dynamic perception: Layout calibration points, determine the initial angle between the camera coordinate system and the calibration point coordinate system by scanning the calibration points with the camera, and scan again after work to determine the secondary angle, and set the adaptive calibration frequency to automatically trigger the calibration process by monitoring the temperature, structural strain, and camera jitter parameters in real time; specifically including:

[0067] S1.1 Layout design of calibration points:

[0068] Global calibration points: Set multiple basic points covering the entire calibration board range, and calculate coarse-grained deformation parameters (such as overall rotation, translation); the basic points cover the entire calibration board range, and the number of global calibration points is set to N global =9, calculate the coarse-grained deformation parameters through the global calibration points, including the overall rotation angle θ global and translation amount (tx global , ty global ), where tx global and ty global respectively represent the translation amounts on the X-axis and Y-axis; when the camera takes pictures of the calibration points for the first time, calculate the angle θ1 between the camera coordinate system and the calibration point coordinate system through the coordinates of the calibration points;

[0069] S1.2 Adaptive calibration frequency:

[0070] Real-time monitoring: Take pictures again after work. At this time, the camera mounting axis will be deformed and twisted due to temperature, structural strain, and jitter. Calculate the angle θ2 between the camera coordinate system and the calibration point coordinate system again through the coordinates of the calibration points; Integrate a temperature sensor or strain gauge on the camera mounting axis to monitor the temperature field or structural strain; at the same time, integrate an acceleration sensor near the camera mounting position to monitor the jitter parameters of the camera in real time;

[0071] Dynamic triggering: When the temperature change exceeds the threshold (such as ±2°C), or the strain exceeds the set value, or the camera jitter parameter exceeds the threshold, the calibration process is automatically triggered. Let the temperature field be T, the structural strain be ε, the camera jitter amplitude be A, and the frequency be f. When or |ε| > ε′, or A > A′, or f exceeds [f min , f max , the calibration process is automatically triggered. Among them, T0 is the initial temperature, ΔT′ is the temperature change threshold, ε′ is the structural strain threshold, A′ is the jitter amplitude threshold, f min and f max are the upper and lower limits of the frequency threshold respectively;

[0072] S2 Mathematical model expansion and error compensation: Construct a time-varying model and an affine transformation matrix, decompose the torsion angle θ into static installation error and dynamic thermal deformation, and introduce translation error and scaling factor to describe the composite deformation of the camera installation axis; According to the spectral characteristics and change trend of the data, automatically select polynomial fitting or B-spline curve fitting for non-linear error compensation to reduce the error caused by external factors; Identify periodic deformation patterns through Fourier transform and use an adaptive filter to suppress specific frequency interference; Specifically include:

[0073] S2.1 Composite deformation modeling:

[0074] Time-varying model: Decompose the torsion angle θ into static installation error and dynamic thermal deformation , and establish model;

[0075] Affine transformation matrix: Introduce translation error (dx, dy) and scaling factor k to construct a complete affine transformation matrix ;

[0076] S2.2 Non-linear error compensation:

[0077] Automatic model selection fitting:

[0078] Polynomial fitting: Perform quadratic / cubic polynomial fitting on to describe the long-period thermal deformation trend; Polynomial fitting is to analyze the spectral characteristics and change trend of the data. If the data shows a relatively smooth long-period change, it is automatically judged to use polynomial fitting. Let 's polynomial fitting function be , where the order n = 2 or 3, (b n , b n-1, ..., b1, b0) are the coefficients of the polynomial. The coefficients are solved by the least squares method to minimize the error between the fitting function P(t) and Δθ(t). By analyzing the spectral characteristics S(ω) and the change trend C(t) of the data, if S(ω) shows a relatively smooth long-period characteristic and C(t) changes slowly, the polynomial fitting is automatically selected. The polynomial coefficients b are solved by the least squares method. j ,and , making smallest;

[0079] B-spline curve: In high-frequency vibration scenarios, the B-spline curve is used to fit the nonlinear changes of θ(t). Similarly, the automatic model selection algorithm automatically selects the B-spline curve for fitting when complex changes such as high-frequency vibration are detected based on data characteristics. The control points of the B-spline curve are set to , the node vector is , where the number of control points is m+1 from 0 to m, and n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve.

[0080] S2.3 Frequency Domain Analysis:

[0081] Fourier transform: Perform FFT on the multiple calibrated θ sequences to identify periodic deformation patterns (such as mechanical vibration frequencies);

[0082] Adaptive filter: Adaptive filter is used to dynamically adjust the filter parameters according to the real-time collected data to better suppress interference while retaining the useful signal. For example, when the interference frequency changes, the adaptive filter can automatically adjust its parameters to ensure effective suppression of interference. Suppose the input signal of the adaptive filter is x(n'), the desired signal is d(n'), the output signal is y(n'), and the error signal is ; Dynamically adjust the filter parameters w(n') according to the real-time collected data, so that Minimum, among which represents the mathematical expectation;

[0083] S3 algorithm and data processing improvements: Combine IMU or laser rangefinder data with multi-source information through extended Kalman filtering to improve the real-time and noise resistance of angle estimation; use deep learning to assist calibration, and directly output the torsion angle through a neural network for torsion angle compensation and threshold determination to trigger the calibration process; use synthetic data enhancement and transfer learning to improve model generalization capabilities; use incremental algorithms to reduce computational latency; and perform regular global recalibration to avoid cumulative errors. Specific improvements include:

[0084] S3.1 Heterogeneous Data Fusion:

[0085] Multi-source information fusion: Combine the data of the inertial measurement unit (IMU) or the laser rangefinder, and fuse the multi-source information through the extended Kalman filter (EKF) to improve the real-time performance and noise resistance of angle estimation. The IMU data includes the angular velocity ω and the angular acceleration ω˙, and the laser rangefinder data is represented as d laser ; Let the state vector be ; The observation vector is , where T represents the transpose operation; The state transition matrix is , where Δt is the time step; The observation matrix is , where h1 and h2 are determined according to the specific observation model. h1 is related to how the first element in the state vector, the angular velocity ω, affects the second element in the observation vector, the laser rangefinder data d laser . If the laser rangefinder data does not directly depend on the angular velocity, then h1 is 0; h2 is related to how the second element in the state vector, the angular acceleration ω˙, affects the second element in the observation vector, the laser rangefinder data d laser . If the laser rangefinder data does not directly depend on the angular acceleration, then h2 is also 0; By collecting the laser rangefinder data under different angular velocities and angular accelerations, and then using these data to estimate h1 and h2;

[0086] S3.2 Deep learning-assisted calibration:

[0087] Neural network training: Use a CNN or Transformer network, input the calibration board image feature I, and directly output the torsion angle θ. Let the loss function of the neural network be , where, θ pred is the predicted value, θ true is the true value, R is the total number of samples, and r ∈ (1, 2,..., R);

[0088] Synthetic data augmentation: Generate synthetic data under different working conditions (such as temperature gradient, mechanical vibration) through simulation, combine with the measured data to train the model, and improve the generalization ability;

[0089] Transfer learning: For different types of pick-and-place machines or working conditions, fine-tune the model parameters through new data. Let the original model parameters be w original , the new data be , and the fine-tuned model parameters be w new , Q new is the number of samples of the new data, and ;

[0090] S3.3 Real-time dynamic calibration:

[0091] Incremental algorithm: Use the previous calibration result as a prior, and only update the θ value through 3 - 4 new calibration points to reduce the calculation delay. Let the previous calibration result be θ prev, the new calibration point is , c ∈ 1 to C, and C is 3 or 4, update the θ value by the least squares method;

[0092] Periodic global recalibration: To avoid cumulative errors in the incremental algorithm after long-term operation, perform periodic global recalibration to eliminate cumulative errors; for example, after running for a certain period of time (such as 8 hours) or completing a certain number of measurement tasks (such as 1000 measurements), perform a comprehensive calibration. Set the running time threshold as t′ (such as 8h) and the measurement task quantity threshold as N′ (such as 1000). When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, perform a comprehensive calibration;

[0093] To avoid cumulative errors in the incremental algorithm after long-term operation, perform periodic global recalibration to eliminate cumulative errors; for example, after running for a certain period of time (such as 8 hours) or completing a certain number of measurement tasks (such as 1000 measurements), perform a comprehensive calibration. Set the running time threshold as t′ (such as 8h) and the measurement task quantity threshold as N′ (such as 1000). When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, perform a comprehensive calibration;

[0094] S3.4 Angle calculation and result application:

[0095] After obtaining the coordinates of the calibration points in the initial shot and after work, calculate the included angles θ1 and θ2 between the camera coordinate system and the calibration point coordinate system, and obtain the torsion angle of the camera mounting axis by calculating the difference between θ2 and θ1, which is used for error compensation to improve the accuracy of camera calibration. And when the torsion angle exceeds the torsion angle threshold, trigger the recalibration process to ensure the accuracy and stability of camera installation.

[0096] By adopting the 9-point calibration method, the camera error caused by the torque deformation of the camera axis during each work can be detected and calculated, and combined with real-time monitoring of parameters such as temperature, strain, and jitter, the calibration process can be adaptively triggered, which can timely detect and respond to the deformation of the camera mounting axis and improve the calibration accuracy; and by constructing a time-varying model and an affine transformation matrix, automatically select polynomial or B-spline curve fitting for non-linear error compensation, use Fourier transform and adaptive filter to process periodic deformation and interference, effectively reduce the errors caused by external factors; by combining IMU or laser rangefinder data through extended Kalman filter to fuse multi-source information, use deep learning to assist calibration, adopt synthetic data augmentation and transfer learning to improve the generalization ability of the model, and use the incremental algorithm to reduce the calculation delay and perform periodic global recalibration, further improve the real-time performance and anti-noise performance of calibration, avoid cumulative errors, and ensure the accurate and stable installation of the camera.

[0097] The second implementation mode, the main difference from the first implementation mode is:

[0098] The calibration points are designed with non-uniform distribution to enhance the ability to capture non-linear deformation. Let the distance between calibration points in the edge area of the calibration plate be d edge , and the distance between calibration points in the central area be d center ; Assume the length of the edge area of the calibration plate is L edge , and the length of the central area is L center , then the number of calibration points in the edge area , and the number of calibration points in the central area .

[0099] In this embodiment, compared with the first embodiment, the calibration points are non-uniformly distributed. The distance between calibration points in the edge area is small and the number is large, while the distance in the central area is large and the number is small. This further enhances the ability to capture non-linear deformation, enables the calibration to more accurately reflect the deformation conditions of different areas of the calibration plate, improves the calibration accuracy, and is especially suitable for complex deformation scenarios.

[0100] The main difference between the third embodiment and the first embodiment is that: based on the original 9 calibration points, redundant calibration points are added in the key edge areas. Let the original number of calibration points be N0 = 9, and N1 corner redundant points are added. Abnormal points are removed through a one-time calibration algorithm to obtain the optimal model parameters, which specifically include: for a set of calibration point data , randomly select a part of the points as a subset, use the selected subset points to fit a model (for example, a linear model, a polynomial model, etc.), and calculate the number of inlier points that conform to the model among the other points according to the set error threshold δ; repeat the above steps, and select the model with the largest number of inlier points as the optimal model. After the optimal model is determined, remove the abnormal points that do not conform to the model.

[0101] In this embodiment, redundant calibration points are added in the key edge areas based on the original 9 calibration points, and abnormal points are removed through a one-time calibration algorithm. This increases the number of calibration points and the rationality of distribution, and can capture deformation information more comprehensively. At the same time, this algorithm can effectively identify and remove abnormal data, reduce its influence on the model parameters, make the obtained model parameters more accurate and reliable, further improve the accuracy and stability of camera calibration, and enhance the credibility of the calibration results.

[0102] After our factory improved the algorithm of the mounter according to this solution, the statistical data compared with the original algorithm are shown in Table 1 below:

[0103] Table 1: Table of data changes before and after calibration

[0104]

[0105] Note:

[0106] Yield rate: After calibration, due to the improvement of the mounting accuracy, the defective products caused by position deviation are reduced, and the yield rate is increased from 92% to 98%.

[0107] Qualified rate: Similarly, due to the improvement of the accuracy, the qualified rate is also increased from 95% to 99%.

[0108] Mounting accuracy: Calibration significantly reduces the mounting deviation, from ±0.05mm to ±0.02mm, improving the accuracy by 60%.

[0109] Calibration frequency: It changes from the irregular calibration relying on manual judgment to the calibration process that is adaptively triggered according to the changes of parameters such as temperature, strain, and jitter, realizing the automation of calibration.

[0110] Production efficiency: Due to the reduction of the adjustment and downtime caused by the mounting error, the production efficiency is increased by 12.5%.

[0111] Failure rate: The calibrated camera system is more stable, and the failure rate is reduced from 3 times per month to 1 time per month.

[0112] Maintenance cost: Due to the reduction of the calibration requirements and the number of repairs, the maintenance cost is also reduced accordingly.

[0113] Meanwhile, the content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

[0114] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0115] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for high-precision calibration of rapid recognition of camera torsion, characterized in that Specifically, it includes the following steps: S1 Marking Point Layout and Dynamic Sensing: Layout the marking points, determine the initial angle between the camera coordinate system and the marking point coordinate system by scanning the marking points with the camera. After work, scan again to determine the secondary angle. Set the adaptive calibration frequency to automatically trigger the calibration process by monitoring the temperature, structural strain, and camera jitter parameters in real time; S2 Mathematical Model Expansion and Error Compensation: Construct a time-varying model and an affine transformation matrix, decompose the torsion angle θ into static installation errors and dynamic thermally induced deformations, and introduce translation errors and scaling factors to describe the composite deformation of the camera installation axis; According to the spectral characteristics and change trends of the data, automatically select polynomial fitting or B-spline curve fitting for non-linear error compensation to reduce errors caused by external factors; Identify periodic deformation patterns through Fourier transform and use an adaptive filter to suppress specific frequency interference; S3 Algorithm and Data Processing Improvement: Combine IMU or laser rangefinder data to fuse multi-source information through extended Kalman filtering, improving the real-time performance and noise resistance of angle estimation; Use deep learning to assist in calibration, directly output the torsion angle through the neural network for torsion angle compensation and threshold determination to trigger the calibration process; Use synthetic data augmentation and transfer learning to improve the generalization ability of the model; Adopt an incremental algorithm to reduce calculation latency; Perform global recalibration regularly to avoid cumulative errors.

2. A method for quickly identifying high-precision calibration of camera torsion according to claim 1, characterized in that: The S1 marking point layout and dynamic sensing specifically include: S1.1 Marking Point Layout Design: Global Marking Points: Set multiple basic points covering the entire calibration board range to calculate coarse-grained deformation parameters; When the camera takes pictures of the marking points for the first time, calculate the angle θ1 between the camera coordinate system and the marking point coordinate system through the coordinates of the marking points; S1.2 Adaptive Calibration Frequency: Real-time Monitoring: Take pictures again after work. At this time, the camera installation axis will be deformed and twisted due to temperature, structural strain, and jitter. Calculate the angle θ2 between the camera coordinate system and the marking point coordinate system again through the coordinates of the marking points; Integrate a temperature sensor or strain gauge on the camera installation axis to monitor the temperature field or structural strain; At the same time, integrate an acceleration sensor near the camera installation position to monitor the camera jitter parameters in real time; Dynamic Trigger: When the temperature change exceeds the threshold or the strain exceeds the set value, or the camera jitter parameter exceeds the threshold, automatically trigger the calibration process.

3. A method for quickly identifying high-precision calibration of camera torsion according to claim 2, characterized in that: The basic points cover the entire calibration board range, and the number of global calibration points is set to N global = 9. Coarse-grained deformation parameters are calculated through the global calibration points, including the overall rotation angle θ global and the translation amounts (tx global , ty global ), where tx global and ty global represent the translation amounts on the X-axis and Y-axis respectively; In the adaptive calibration frequency, let the temperature field be T, the structural strain be ε, the camera jitter amplitude be A, and the frequency be f. When , or |ε| > ε', or A > A', or f exceeds [f min , f max , the calibration process is automatically triggered. Among them, T0 is the initial temperature, ΔT' is the temperature change threshold, ε' is the structural strain threshold, A' is the jitter amplitude threshold, and f min and f max are the upper and lower limits of the frequency threshold, respectively.

4. A method for quickly identifying high-precision calibration of camera torsion according to claim 1, characterized in that: The S2 mathematical model expansion and error compensation specifically include: S2.1 Composite Deformation Modeling: Time-varying model: Decompose the torsional angle θ into static installation error and dynamic thermally induced deformation , and establish the model; Affine Transformation Matrix: Introduce translation errors (dx, dy) and a scaling factor k to construct a complete affine transformation matrix; S2.2 Non-linear Error Compensation: Automatic Model Selection Fitting: Polynomial fitting: Perform quadratic / cubic polynomial fitting on to describe the long-term thermal deformation trend; B-spline curve: In high-frequency vibration scenarios, use B-spline curve fitting for non-linear changes; similarly, the automatic model selection algorithm automatically selects B-spline curve fitting according to data characteristics when detecting complex changes in high-frequency vibration; S2.3 Frequency Domain Analysis: Fourier Transform: Perform FFT on the θ sequence of multiple calibrations to identify periodic deformation patterns; Adaptive Filter: Adopt an adaptive filter to dynamically adjust the parameters of the filter according to the real-time collected data to better suppress interference while retaining useful signals.

5. A method for quickly identifying high-precision calibration of camera torsion according to claim 4, characterized in that: The affine transformation matrix is as follows: ; The polynomial fitting is performed by analyzing the spectral characteristics and variation trends of the data. If the data exhibits relatively smooth long-period variations, polynomial fitting is automatically determined. Let the polynomial fitting function of Δθ(t) be , where the order n = 2 or 3, (b n , b n-1 ,..., b1, b0) are the coefficients of the polynomial. The coefficients are solved by the least squares method to minimize the error between the fitting function P(t) and Δθ(t); by analyzing the spectral characteristic S(ω) and variation trend C(t) of the data, if S(ω) exhibits relatively smooth long-period characteristics and C(t) changes slowly, polynomial fitting is automatically selected; the polynomial coefficients b j are solved by the least squares method, and j ∈ (1, 2,..., n), such that is minimized. In the B-spline curve step, let the control points of the B-spline curve be , and the knot vector be , where the number of control points ranges from 0 to m, a total of m + 1, and n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve. In the said adaptive filter, let the input signal of the adaptive filter be x(n'), the desired signal be d(n'), the output signal be y(n'), and the error signal be ; the parameters w(n') of the filter are dynamically adjusted according to the real-time collected data, so that is minimized, where represents the mathematical expectation.

6. A method for quickly identifying high-precision calibration of camera torsion according to claim 1, characterized in that: The S3 algorithm and data processing improvement specifically include: S3.1 Heterogeneous Data Fusion: Multi-source Information Fusion: Combine inertial measurement unit IMU or laser rangefinder data to fuse multi-source information through extended Kalman filtering; S3.2 Deep Learning Assisted Calibration: Neural network training: Using a CNN or Transformer network, input the calibration board image feature I and directly output the torsion angle θ; Synthetic data augmentation: Generate synthetic data under different working conditions through simulation and train the model in combination with measured data; Transfer learning: For different pick-and-place machine models or working conditions, fine-tune the model parameters with new data; S3.3 Real-time dynamic calibration: Incremental algorithm: Use the previous calibration result as a prior and update the θ value only through 3 - 4 new calibration points; Periodic global recalibration: To avoid cumulative errors in the incremental algorithm after long-term operation, perform periodic global recalibration to eliminate cumulative errors; S3.4 Angle calculation and result application: After obtaining the coordinates of the calibration points in the two shots of the initial shooting and after work, calculate the included angles θ1 and θ2 between the camera coordinate system and the calibration point coordinate system using the coordinates, and obtain the torsion angle of the camera mounting axis by calculating the difference between θ2 and θ1, which is used for error compensation to improve the accuracy of camera calibration. And when the torsion angle exceeds the torsion angle threshold, trigger the recalibration process.

7. A method for quickly identifying high-precision calibration of camera torsion according to claim 6, characterized in that: In the S3.1 heterogeneous data fusion, the IMU data includes the angular velocity ω and the angular acceleration ω˙, and the lidar data is represented as d laser ; Let the state vector be ; The observation vector is , where T represents the transpose operation; The state transition matrix is , where Δt is the time step; The observation matrix is , where h1 and h2 are determined according to the specific observation model; In the above-mentioned S3.2 deep learning-assisted calibration, let the loss function of the neural network be , where, θ pred is the predicted value, θ true is the true value, R is the total number of samples, and r ∈ (1, 2,..., R); In transfer learning, let the original model parameters be w original , the new data be , and the fine-tuned model parameters be w new , Q new be the number of samples of the new data, and ; In the S3.3 real-time dynamic calibration, let the previous calibration result be θ prev , the new calibration point is , c ∈ 1 to C, and C is 3 or 4, and update the value of θ by the least squares method; In the periodic global recalibration, set the running time threshold as t′ and the measurement task quantity threshold as N′. When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, perform a comprehensive calibration.

8. A method for high-precision calibration of rapid identification of camera torsion according to any one of claims 1-2, 4-7, characterized in that: The non-uniform distribution design is carried out for the calibration points to enhance the ability to capture non-linear deformation. Let the distance between calibration points in the edge area of the calibration plate be d edge , and the distance between calibration points in the central area be d center ; Assume that the length of the edge area of the calibration plate is L edge , and the length of the central area is L center , then the number of calibration points in the edge area , and the number of calibration points in the central area .

9. A method for quickly identifying high-precision calibration of camera torsion according to claim 3, characterized in that: On the basis of the original 9 calibration points, add redundant calibration points in the key edge areas. Let the original number of calibration points be N0 = 9, add N1 corner redundant points, and eliminate abnormal points through a one-time verification algorithm to obtain the optimal model parameters.

10. A method for quickly identifying high-precision calibration of camera torsion according to claim 9, characterized in that: The steps of eliminating abnormal points through the one-time verification algorithm specifically include: for a set of calibration point data , randomly select a part of the points as a subset from them, use the selected subset points to fit a model, and calculate the number of inliers that conform to the model among other points according to the set error threshold δ; repeat the above steps, select the model with the largest number of inliers as the optimal model, and after the optimal model is determined, eliminate the abnormal points that do not conform to the model.

Images