High-precision calibration method for quickly identifying camera torsion

Abstract

The invention discloses a high-precision calibration method for quickly identifying camera torsion, and relates to the technical field of industrial camera calibration. The high-precision calibration method for quickly identifying camera torsion specifically comprises the following steps: S1, carrying out calibration point layout and dynamic perception; s2, mathematical model expansion and error compensation; s3, algorithm and data processing improvement; by adopting a nine-point calibration mode, camera errors caused by torque deformation of a camera shaft during each time of work can be detected and calculated, and in combination with real-time monitoring of parameters such as temperature, strain and jitter, a calibration process is adaptively triggered, so that deformation of the camera mounting shaft can be timely found and dealt with, and the calibration precision is improved; nonlinear error compensation is carried out by constructing a time-varying model and an affine transformation matrix and automatically selecting polynomial or B spline curve fitting, and periodic deformation and interference are processed by using Fourier transform and an adaptive filter, so that errors caused by external factors are effectively reduced, and accurate and stable installation of the camera is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of industrial camera calibration, and in particular to a method for rapid identification of camera torsion and high-precision calibration. Background Art

[0002] Through calibration, the industrial camera of the placement machine can accurately identify the mark points (reference points) on the PCB board and the position of components, thereby achieving high-precision placement of components. The calibration process eliminates factors such as distortion and error in the camera imaging process, ensuring the accurate correspondence between the camera coordinate system and the physical world coordinate system. An inaccurately calibrated camera may cause deviations in the placement position and affect product quality. Through calibration, this error can be significantly reduced and the placement accuracy and consistency can be improved.

[0003] Since the placement machine uses industrial cameras to locate the position of the product during the processing, whether the industrial camera can accurately locate the product is related to the accuracy of the operation. For example, the control system and method of a shielding cover placement machine disclosed in publication number CN108243600A include an industrial mainboard, a placement head control module, an XY axis motion control module, a feeding control module, and a recognition camera connected to the industrial mainboard through a communication channel. The recognition camera includes a Mark point camera, a fixed camera, and an external camera. The industrial mainboard is also connected to an alarm detection module. The control system of the shielding cover placement machine uses a modular concept. Special PCB boards are set on the placement head control module, the XY axis motion control module, and the feeding control module to improve the data processing capability and transportation accuracy of the entire shielding cover placement machine placement system. By setting a Mark point camera and fixed cameras and external cameras of different precisions, the system image recognition technology is improved to achieve the purpose of stable, accurate, and fast placement of the shielding cover placement machine.

[0004] However, as shown in the above technology, it improves accuracy by installing multiple cameras in coordination, which is a temporary solution and not a fundamental solution. In actual use, with the increase in device temperature, the vibration of the device and other factors, the camera mounting axis will be deformed, which will cause errors in the camera itself. Therefore, only improving the accuracy of image recognition at this time cannot solve the recognition error caused by the error of the camera itself. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method for quickly identifying camera torsion and high-precision calibration, which solves the problem of identification error caused by the deformation of the camera's own mounting axis.

[0006] To achieve the above objectives, the present invention is implemented by the following technical solutions: a method for quickly identifying camera torsion with high precision calibration, specifically comprising the following steps: S1 Calibration point layout and dynamic perception: layout calibration points, determine the initial angle between the camera coordinate system and the calibration point coordinate system by scanning the calibration points with the camera, scan again after work to determine the secondary angle, set the adaptive calibration frequency to monitor the temperature, structural strain, and camera shake parameters in real time to automatically trigger the calibration process; S2 mathematical model expansion and error compensation: construct a time-varying model and affine transformation matrix, decompose the torsion angle θ into static installation error and dynamic thermal deformation, and introduce translation error and scaling factor to describe the composite deformation of the camera installation axis; according to the spectral characteristics and change trend of the data, automatically select polynomial fitting or B-spline curve fitting for nonlinear error compensation to reduce the error caused by external factors; identify periodic deformation patterns through Fourier transform, and use adaptive filters to suppress specific frequency interference; Improvements to the S3 algorithm and data processing: Combine IMU or laser rangefinder data with extended Kalman filtering to fuse multi-source information, thereby improving the real-time and noise resistance of angle estimation; use deep learning to assist calibration, and directly output the torsion angle through a neural network for torsion angle compensation and threshold determination to trigger the calibration process; use synthetic data enhancement and transfer learning to improve model generalization capabilities; use an incremental algorithm to reduce computational delays; and perform global recalibration regularly to avoid cumulative errors.

[0007] Preferably, the S1 calibration point layout and dynamic perception specifically include: S1.1 Calibration point layout design: Global calibration points: Set multiple basic points covering the entire calibration plate range to calculate the coarse-grained deformation parameters; when the calibration point is photographed with the camera for the first time, the angle θ between the camera coordinate system and the calibration point coordinate system is calculated using the coordinates of the calibration point 1 ; S1.2 Adaptive calibration frequency: Real-time monitoring: After the work is completed, the camera mounting axis will be deformed and twisted due to temperature, structural strain, and vibration. The angle θ between the camera coordinate system and the calibration point coordinate system is calculated again through the coordinates of the calibration point. 2 ; Integrate a temperature sensor or strain gauge on the camera mounting axis to monitor the temperature field or structural strain; at the same time, integrate an acceleration sensor near the camera mounting position to monitor the camera's jitter parameters in real time; Dynamic trigger: When the temperature change exceeds the threshold or the strain exceeds the set value, or the camera shake parameter exceeds the threshold, the calibration process is automatically triggered.

[0008] Preferably, the basic points cover the entire calibration plate range, and the number of global calibration points is N. global = 9, calculate the coarse-grained deformation parameters through the global calibration points, including the overall rotation angle θ global and translation (tx global ,tyglobal ), where tx global Andty global Respectively represent the translation amount on the X-axis and Y-axis; In the adaptive calibration frequency, let the temperature field be T, the structural strain be ε, let the camera shake amplitude be A, and the frequency be f. , or |ε|>ε′, or A>A′, or f exceeds [f min , f max ], the calibration process is automatically triggered, where T 0 is the initial temperature, ΔT′ is the temperature change threshold, ε′ is the structural strain threshold, A′ is the jitter amplitude threshold, and f min and f max are the upper and lower limits of the frequency threshold respectively.

[0009] Preferably, the S2 mathematical model construction and error compensation specifically include: S2.1 Composite deformation modeling: Time-varying model: Decomposing the torsion angle θ into static installation errors and dynamic thermal deformation ,Establish Model; Affine transformation matrix: introduce translation error (dx, dy) and scaling factor k to construct a complete affine transformation matrix; S2.2 Non-linear error compensation: Automatic model selection fitting: Polynomial Fitting: Perform quadratic / cubic polynomial fitting to describe the long-period thermal deformation trend; B-spline curve: In high-frequency vibration scenarios, use B-spline curve fitting Similarly, the automatic model selection algorithm automatically selects B-spline curve fitting based on data characteristics when complex changes such as high-frequency vibration are detected; S2.3 Frequency domain analysis: Fourier transform: FFT is performed on the multiple calibrated θ sequences to identify periodic deformation patterns; Adaptive filter: Adaptive filter is used to dynamically adjust the filter parameters according to the real-time collected data to better suppress interference while retaining useful signals.

[0010] Preferably, the affine transformation matrix is: ; The polynomial fitting is adopted by analyzing the spectrum characteristics and change trends of the data. If the data presents a relatively smooth long-period change, the polynomial fitting is automatically judged to be adopted. The polynomial fitting function of Δθ(t) is set as , where the order n = 2 or 3, (bn , b n-1 , ..., b 1 , b 0 ) are the coefficients of the polynomial, and the coefficients are solved by the least squares method so that the fitting function P(t) is The error between them is the smallest; by analyzing the spectral characteristics S(ω) and the change trend C(t) of the data, if S(ω) shows a relatively smooth long-period characteristic and C(t) changes slowly, the polynomial fitting is automatically selected; the polynomial coefficient b is solved by the least squares method j , and j∈(1,2,...,n), such that Minimum; In the B-spline curve step, the control points of the B-spline curve are assumed to be , the node vector is , where the number of control points ranges from 0 to m, and there are a total of m+1. n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve. In the adaptive filter, the input signal of the adaptive filter is x(n'), the expected signal is d(n'), the output signal is y(n'), and the error signal is ; Dynamically adjust the filter parameters w(n') according to the real-time collected data, so that Minimum, among which represents mathematical expectation.

[0011] Preferably, the S3 algorithm and data processing improvements specifically include: S3.1 Heterogeneous Data Fusion: Multi-source information fusion: Combine inertial measurement unit IMU or laser rangefinder data and fuse multi-source information through extended Kalman filtering; S3.2 Deep learning assisted calibration: Neural network training: Use CNN or Transformer network, input the calibration plate image feature I, and directly output the torsion angle θ; Synthetic data enhancement: Generate synthetic data under different working conditions through simulation and train the model with measured data; Transfer learning: fine-tune model parameters through new data for different SMT machine models or working conditions; S3.3 real-time dynamic calibration: Incremental algorithm: Use the previous calibration results as a priori and update the θ value only through 3-4 new calibration points; Regular global recalibration: To avoid the cumulative error of the incremental algorithm after a long period of operation, regular global recalibration is performed to eliminate the cumulative error; S3.4 Angle calculation and result application: After obtaining the coordinates of the two calibration points for the first shooting and after work, the angle θ between the camera coordinate system and the calibration point coordinate system is calculated using the coordinates 1 and θ 2 , by calculating θ 2 With θ 1 The difference between the values ​​of the torsion angle and the torsion angle of the camera mounting axis is used for error compensation to improve the accuracy of camera calibration. When the torsion angle exceeds the torsion angle threshold, the recalibration process is triggered.

[0012] Preferably, in the heterogeneous data fusion of S3.1, the IMU data includes angular velocity ω and angular acceleration ω˙, and the laser rangefinder data is represented by d laser ; Let the state vector be The observation vector is , T represents the transposition operation; the state transfer matrix is , Δt is the time step; the observation matrix is , where h 1 and h 2 Determined according to the specific observation model; In the deep learning assisted calibration described in S3.2, the loss function of the neural network is assumed to be , where θ pred is the predicted value, θ true is the true value, R is the total number of samples, r∈(1, 2, ..., R); In transfer learning, the original model parameter is set to w original , the new data is , the fine-tuned model parameters are w new , Q new is the number of samples of new data, and ; In the real-time dynamic calibration of S3.3, the previous calibration result is assumed to be θ prev The new calibration point is , update the value of θ by the least squares method; In the periodic global recalibration, the running time threshold is set as t′ and the measurement task number threshold is set as N′. When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, a comprehensive calibration is performed.

[0013] Preferably, the feature is that: the calibration points are designed to be non-uniformly distributed to enhance the ability to capture nonlinear deformation, and the calibration point spacing in the edge area of ​​the calibration plate is set to d edge , the distance between the calibration points in the center area is d center ; Assume that the length of the edge area of ​​the calibration plate is L edge , the length of the central area is L center , then the number of calibration points in the edge area , the number of calibration points in the central area .

[0014] Preferably, on the basis of the original 9 calibration points, redundant calibration points are added in the edge key area, and the number of original calibration points is set to N 0 =9, increase N 1 The redundant corner points are removed through a one-time verification algorithm to obtain the optimal model parameters.

[0015] Preferably, the step of removing abnormal points by a one-time verification algorithm specifically includes: for a set of calibration point data , randomly select a part of the points as a subset, use the selected subset points to fit the model according to the set error threshold δ, and calculate the number of inliers that meet the model among other points; repeat the above steps, and select the model with the largest number of inliers as the optimal model. After the optimal model is determined, remove the abnormal points that do not meet the model.

[0016] The present invention provides a method for rapid identification of camera torsion and high-precision calibration. Compared with the prior art, it has the following beneficial effects: 1. In the first embodiment of the present invention, by adopting a 9-point calibration method, the camera error caused by the torque deformation of the camera axis during each operation can be detected and calculated, and by combining real-time monitoring of parameters such as temperature, strain, and jitter, the calibration process can be adaptively triggered, so that the deformation of the camera installation axis can be discovered and responded to in time, thereby improving the calibration accuracy; and by constructing a time-varying model and an affine transformation matrix, polynomial or B-spline curve fitting is automatically selected for nonlinear error compensation, and Fourier transform and adaptive filter are used to process periodic deformation and interference, thereby effectively reducing the error caused by external factors; by combining IMU or laser rangefinder data and fusing multi-source information through extended Kalman filtering, deep learning is used to assist calibration, synthetic data enhancement and transfer learning are used to improve the generalization ability of the model, and an incremental algorithm is used to reduce calculation delays and regular global recalibration is performed, thereby further improving the real-time and noise resistance of the calibration, avoiding cumulative errors, and ensuring accurate and stable camera installation.

[0017] 2. In the second embodiment of the present invention, the calibration points are distributed non-uniformly, with smaller spacing and more calibration points in the edge area, and larger spacing and fewer calibration points in the center area. This further enhances the ability to capture nonlinear deformation, enables the calibration to more accurately reflect the deformation of different areas of the calibration plate, improves the calibration accuracy, and is particularly suitable for complex deformation scenes.

[0018] 3. In the third embodiment of the present invention, redundant calibration points are added to the edge key area based on the original 9 calibration points, and abnormal points are removed through a one-time verification algorithm. This increases the number of calibration points and the rationality of their distribution, and can capture deformation information more comprehensively. At the same time, the algorithm can effectively identify and remove abnormal data, reduce its impact on model parameters, make the obtained model parameters more accurate and reliable, further improve the accuracy and stability of camera calibration, and enhance the credibility of calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the steps of the present invention; Figure 2 A schematic diagram of the camera installation axis torsion of the present invention; Figure 3 It is a schematic diagram of the deflection of the two-scan calibration points in the first embodiment of the present invention; Figure 4 This is a schematic diagram of calibration points in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of calibration points in embodiment 3 of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1-Figure 5 , the present invention provides the following three technical solutions: The first implementation mode: a method for quickly identifying camera torsion with high precision calibration, specifically comprising the following steps: S1 Calibration point layout and dynamic perception: layout calibration points, determine the initial angle between the camera coordinate system and the calibration point coordinate system by scanning the calibration points with the camera, scan again after work to determine the secondary angle, set the adaptive calibration frequency to monitor the temperature, structural strain, and camera shake parameters in real time to automatically trigger the calibration process; specifically include: S1.1 Calibration point layout design: Global calibration points: Cover the entire calibration plate range to set multiple basic points and calculate the coarse-grained deformation parameters (such as overall rotation and translation); the basic points cover the entire calibration plate range, and the number of global calibration points is set to N global = 9, calculate the coarse-grained deformation parameters through the global calibration points, including the overall rotation angle θ global and translation (tx global ,ty global), where tx global Andty global Represents the translation on the X-axis and Y-axis respectively; when the calibration point is photographed with the camera for the first time, the angle θ between the camera coordinate system and the calibration point coordinate system is calculated by the coordinates of the calibration point 1 ; S1.2 Adaptive calibration frequency: Real-time monitoring: After the work is completed, the camera mounting axis will be deformed and twisted due to temperature, structural strain, and vibration. The angle θ between the camera coordinate system and the calibration point coordinate system is calculated again through the coordinates of the calibration point. 2 ; Integrate a temperature sensor or strain gauge on the camera mounting axis to monitor the temperature field or structural strain; at the same time, integrate an acceleration sensor near the camera mounting position to monitor the camera's jitter parameters in real time; Dynamic trigger: When the temperature change exceeds the threshold (such as ±2℃) or the strain exceeds the set value, or the camera shake parameter exceeds the threshold, the calibration process is automatically triggered. Set the temperature field as T, the structural strain as ε, the camera shake amplitude as A, and the frequency as f. or |ε|>ε′, or A>A′, or f exceeds [f min , f max ], the calibration process is automatically triggered, where T 0 is the initial temperature, ΔT′ is the temperature change threshold, ε′ is the structural strain threshold, A′ is the jitter amplitude threshold, and f min and f max are the upper and lower limits of the frequency threshold, respectively; S2 mathematical model expansion and error compensation: construct a time-varying model and affine transformation matrix, decompose the torsion angle θ into static installation error and dynamic thermal deformation, and introduce translation error and scaling factor to describe the composite deformation of the camera installation axis; according to the spectral characteristics and change trend of the data, automatically select polynomial fitting or B-spline curve fitting for nonlinear error compensation to reduce the error caused by external factors; identify periodic deformation patterns through Fourier transform, and use adaptive filters to suppress specific frequency interference; specifically include: S2.1 Composite deformation modeling: Time-varying model: Decomposing the torsion angle θ into static installation errors and dynamic thermal deformation ,Establish Model; Affine transformation matrix: introduce translation error (dx, dy), scaling factor k, and construct a complete affine transformation matrix ; S2.2 Non-linear error compensation: Automatic model selection fitting: Polynomial Fitting: Perform quadratic / cubic polynomial fitting to describe the long-period thermal deformation trend; the polynomial fitting is adopted by analyzing the spectral characteristics and change trend of the data. If the data presents a relatively smooth long-period change, the polynomial fitting is automatically judged to be adopted. The polynomial fitting function is , where the order n = 2 or 3, (b n , b n-1 , ..., b 1 , b 0 ) are the coefficients of the polynomial, and the coefficients are solved by the least squares method to minimize the error between the fitting function P(t) and Δθ(t); by analyzing the spectral characteristics S(ω) and the change trend C(t) of the data, if S(ω) shows a relatively smooth long-period characteristic and C(t) changes slowly, the polynomial fitting is automatically selected; the polynomial coefficients b are solved by the least squares method j ,and , so that Minimum; B-spline curve: In the high-frequency vibration scenario, the B-spline curve is used to fit the nonlinear changes of θ(t); similarly, the automatic model selection algorithm automatically selects the B-spline curve for fitting when complex changes such as high-frequency vibration are detected based on the data characteristics. The control points of the B-spline curve are set to , the node vector is , where the number of control points ranges from 0 to m, and there are a total of m+1. n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve. S2.3 Frequency domain analysis: Fourier transform: Perform FFT on the multiple calibrated θ sequences to identify periodic deformation patterns (such as mechanical vibration frequency); Adaptive filter: Adaptive filter is used to dynamically adjust the parameters of the filter according to the real-time collected data to better suppress interference while retaining useful signals. For example, when the interference frequency changes, the adaptive filter can automatically adjust its parameters to ensure effective suppression of interference. Suppose the input signal of the adaptive filter is x(n'), the expected signal is d(n'), the output signal is y(n'), and the error signal is ; Dynamically adjust the filter parameters w(n') according to the real-time collected data, so that Minimum, among which represents mathematical expectation; S3 algorithm and data processing improvements: Combine IMU or laser rangefinder data to fuse multi-source information through extended Kalman filtering to improve the real-time and noise resistance of angle estimation; use deep learning to assist calibration, and directly output the torsion angle through a neural network for torsion angle compensation and threshold determination to trigger the calibration process; use synthetic data enhancement and transfer learning to improve model generalization capabilities; use incremental algorithms to reduce calculation delays; perform global recalibration regularly to avoid cumulative errors; specifically include: S3.1 Heterogeneous Data Fusion: Multi-source information fusion: Combined with the inertial measurement unit IMU or laser rangefinder data, multi-source information is fused through the extended Kalman filter (EKF) to improve the real-time and noise resistance of angle estimation. The IMU data includes angular velocity ω and angular acceleration ω˙, and the laser rangefinder data is expressed as d laser ; Let the state vector be The observation vector is , T represents the transposition operation; the state transfer matrix is , Δt is the time step; the observation matrix is , where h 1 and h 2 Determined according to the specific observation model, h 1 How does the angular velocity ω, the first element in the state vector, affect the laser rangefinder data d, the second element in the observation vector? laser If the laser rangefinder data does not depend directly on the angular velocity, then h 1 is 0; h 2 How does the second element of the state vector, angular acceleration ω˙, affect the second element of the observation vector, laser rangefinder data d laser If the laser rangefinder data does not depend directly on the angular acceleration, then h 2 is also 0; by collecting laser rangefinder data at different angular velocities and angular accelerations, and then using these data to estimate h 1 and h 2 ; S3.2 Deep learning assisted calibration: Neural network training: Use CNN or Transformer network, input the calibration plate image feature I, directly output the torsion angle θ, and set the loss function of the neural network to be , where θ pred is the predicted value, θ true is the true value, R is the total number of samples, r∈(1, 2, ..., R); Synthetic data enhancement: Generate synthetic data under different working conditions (such as temperature gradient and mechanical vibration) through simulation, and train the model with measured data to improve generalization ability; Transfer learning: For different SMT machine models or working conditions, fine-tune the model parameters through new data, assuming that the original model parameters are w original , the new data is , the fine-tuned model parameters are w new , Q new is the number of samples of new data, and ; S3.3 real-time dynamic calibration: Incremental algorithm: Use the previous calibration result as a priori, and only update the θ value through 3-4 new calibration points to reduce calculation delay. Set the previous calibration result as θ prev The new calibration point is , update the value of θ by the least squares method; Regular global recalibration: To avoid cumulative errors in the incremental algorithm after long-term operation, regular global recalibration is performed to eliminate the cumulative errors. For example, a comprehensive calibration is performed after a certain running time (such as 8 hours) or a certain number of measurement tasks (such as 1000 measurements). The running time threshold is t' (such as 8 hours) and the measurement task number threshold is N' (such as 1000). When the running time exceeds t' or the number of completed measurement tasks exceeds N', a comprehensive calibration is performed. In order to avoid the cumulative error of the incremental algorithm after long-term operation, global recalibration is performed regularly to eliminate the cumulative error; for example, after a certain running time (such as 8 hours) or a certain number of measurement tasks (such as 1000 measurements) are completed, a comprehensive calibration is performed. The running time threshold is t' (such as 8h), and the measurement task number threshold is N' (such as 1000). When the running time exceeds t' or the number of completed measurement tasks exceeds N', a comprehensive calibration is performed; S3.4 Angle calculation and application of results: After obtaining the coordinates of the two calibration points for the first shooting and after work, the angle θ between the camera coordinate system and the calibration point coordinate system is calculated using the coordinates 1 and θ 2 , by calculating θ 2 With θ 1 The difference is used to obtain the torsion angle of the camera installation axis, which is used for error compensation to improve the accuracy of camera calibration. When the torsion angle exceeds the torsion angle threshold, the recalibration process is triggered to ensure the accuracy and stability of the camera installation.

[0022] By adopting the 9-point calibration method, the camera error caused by the torque deformation of the camera axis during each operation can be detected and calculated, and combined with real-time monitoring of parameters such as temperature, strain, and jitter, the calibration process can be adaptively triggered, which can timely detect and respond to the deformation of the camera installation axis and improve the calibration accuracy; by constructing a time-varying model and affine transformation matrix, automatically selecting polynomial or B-spline curve fitting for nonlinear error compensation, and using Fourier transform and adaptive filters to process periodic deformation and interference, errors caused by external factors can be effectively reduced; by combining IMU or laser rangefinder data through extended Kalman filtering to fuse multi-source information, using deep learning to assist calibration, using synthetic data enhancement and transfer learning to improve model generalization ability, using incremental algorithms to reduce calculation delays and regularly recalibrate globally, further improving the real-time and noise resistance of calibration, avoiding cumulative errors, and ensuring accurate and stable camera installation.

[0023] The second implementation mode is mainly different from the first implementation mode in that: The calibration points are designed to be distributed non-uniformly to enhance the ability to capture nonlinear deformation. The calibration point spacing in the edge area of ​​the calibration plate is set to d edge , the distance between the calibration points in the center area is d center ; Assume that the length of the edge area of ​​the calibration plate is L edge , the length of the central area is L center , then the number of calibration points in the edge area , the number of calibration points in the central area .

[0024] Compared with the first implementation, this embodiment distributes the calibration points unevenly, with smaller spacing and more calibration points in the edge area, and larger spacing and fewer calibration points in the center area. This further enhances the ability to capture nonlinear deformation, enables calibration to more accurately reflect the deformation of different areas of the calibration plate, and improves calibration accuracy, which is especially suitable for complex deformation scenes.

[0025] The third implementation mode is mainly different from the first implementation mode in that: on the basis of the original 9 calibration points, redundant calibration points are added in the edge key area, and the number of the original calibration points is set to N. 0 =9, increase N 1 The redundant corner points are eliminated through a one-time verification algorithm to obtain the optimal model parameters, including: , randomly select a part of the points as a subset, use the selected subset points to fit the model (for example, linear model, polynomial model, etc.), and calculate the number of inliers that meet the model among other points according to the set error threshold δ; repeat the above steps and select the model with the largest number of inliers as the optimal model. After the optimal model is determined, remove the abnormal points that do not meet the model.

[0026] This embodiment adds redundant calibration points in the edge key area based on the original 9 calibration points, and removes abnormal points through a one-time verification algorithm. This increases the number of calibration points and the rationality of their distribution, and can capture deformation information more comprehensively. At the same time, the algorithm can effectively identify and remove abnormal data, reduce its impact on model parameters, make the obtained model parameters more accurate and reliable, further improve the accuracy and stability of camera calibration, and enhance the credibility of calibration results.

[0027] After our factory improved the placement machine algorithm through this solution, the statistical data compared with the original algorithm is shown in Table 1 below: Table 1: Data changes before and after calibration

[0028] illustrate: Yield rate: After calibration, the yield rate increased from 92% to 98% due to the improvement of placement accuracy and the reduction of defective products caused by position deviation; Qualification rate: Similarly, due to the improvement of accuracy, the qualification rate also increased from 95% to 99%; Mounting accuracy: Calibration significantly reduces the mounting deviation from ±0.05mm to ±0.02mm, improving the accuracy by 60%; Calibration frequency: From irregular calibration that relies on manual judgment to a calibration process that is adaptively triggered based on changes in parameters such as temperature, strain, and jitter, achieving calibration automation; Production efficiency: Production efficiency increased by 12.5% ​​due to reduced adjustment and downtime caused by placement errors; Failure rate: The camera system is more stable after calibration, and the failure rate is reduced from 3 times per month to 1 time per month; Maintenance costs: Maintenance costs are reduced due to reduced calibration needs and fewer repairs.

[0029] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0030] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0031] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for rapid identification of camera torsion and high-precision calibration, characterized in that: The specific steps include: S1 Calibration point layout and dynamic perception: layout calibration points, determine the initial angle between the camera coordinate system and the calibration point coordinate system by scanning the calibration points with the camera, scan again after work to determine the secondary angle, set the adaptive calibration frequency to monitor the temperature, structural strain, and camera shake parameters in real time to automatically trigger the calibration process; S2 mathematical model expansion and error compensation: construct a time-varying model and affine transformation matrix, decompose the torsion angle θ into static installation error and dynamic thermal deformation, and introduce translation error and scaling factor to describe the composite deformation of the camera installation axis; according to the spectral characteristics and change trend of the data, automatically select polynomial fitting or B-spline curve fitting for nonlinear error compensation to reduce the error caused by external factors; identify periodic deformation patterns through Fourier transform, and use adaptive filters to suppress specific frequency interference; Improvements to S3 algorithm and data processing: Combine IMU or laser rangefinder data to fuse multi-source information through extended Kalman filtering to improve the real-time and noise resistance of angle estimation; use deep learning to assist calibration, and directly output the torsion angle through a neural network for torsion angle compensation and threshold determination to trigger the calibration process; use synthetic data enhancement and transfer learning to improve model generalization capabilities; use incremental algorithms to reduce computational delays; Perform global recalibration periodically to avoid cumulative errors.

2. The method for rapid identification of camera torsion and high-precision calibration according to claim 1, characterized in that: The S1 calibration point layout and dynamic perception specifically include: S1.1 Calibration point layout design: Global calibration points: multiple basic points are set to cover the entire calibration plate range to calculate the coarse-grained deformation parameters; when the calibration point is photographed with a camera for the first time, the angle θ1 between the camera coordinate system and the calibration point coordinate system is calculated using the coordinates of the calibration point; S1.2 Adaptive calibration frequency: Real-time monitoring: After work, take pictures again. At this time, the camera installation axis will be deformed and twisted due to temperature, structural strain, and jitter. The angle θ2 between the camera coordinate system and the calibration point coordinate system is calculated again through the coordinates of the calibration point. A temperature sensor or strain gauge is integrated on the camera installation axis to monitor the temperature field or structural strain. At the same time, an acceleration sensor is integrated near the camera installation position to monitor the camera's jitter parameters in real time. Dynamic trigger: When the temperature change exceeds the threshold or the strain exceeds the set value, or the camera shake parameter exceeds the threshold, the calibration process is automatically triggered.

3. The method for rapid identification of camera torsion and high-precision calibration according to claim 2, characterized in that: The basic points cover the entire calibration plate range, and the number of global calibration points is set to N global = 9, calculate the coarse-grained deformation parameters through the global calibration points, including the overall rotation angle θ global and translation (tx global ,ty global ), where tx global Andty global Respectively represent the translation amount on the X-axis and Y-axis; In the adaptive calibration frequency, let the temperature field be T, the structural strain be ε, let the camera shake amplitude be A, and the frequency be f. , or |ε|>ε′, or A>A′, or f exceeds [f min , f max ], the calibration process is automatically triggered, where T0 is the initial temperature, ΔT′ is the temperature change threshold, ε′ is the structural strain threshold, A′ is the jitter amplitude threshold, and f min and f max are the upper and lower limits of the frequency threshold respectively.

4. The method for rapid identification of camera torsion and high-precision calibration according to claim 1, characterized in that: The S2 mathematical model construction and error compensation specifically include: S2.1 Composite deformation modeling: Time-varying model: Decomposing the torsion angle θ into static installation errors and dynamic thermal deformation ,Establish Model; Affine transformation matrix: introduce translation error (dx, dy) and scaling factor k to construct a complete affine transformation matrix; S2.2 Non-linear error compensation: Automatic model selection fitting: Polynomial Fitting: Perform quadratic / cubic polynomial fitting to describe the long-period thermal deformation trend; B-spline curve: In high-frequency vibration scenarios, use B-spline curve fitting Similarly, the automatic model selection algorithm automatically selects B-spline curve fitting based on data characteristics when complex changes in high-frequency vibration are detected; S2.3 Frequency domain analysis: Fourier transform: FFT is performed on the multiple calibrated θ sequences to identify periodic deformation patterns; Adaptive filter: Adaptive filter is used to dynamically adjust the filter parameters according to the real-time collected data to better suppress interference while retaining useful signals.

5. The method for rapid identification of camera torsion and high-precision calibration according to claim 4, characterized in that: The affine transformation matrix is: ; The polynomial fitting is adopted by analyzing the spectrum characteristics and change trends of the data. If the data presents a relatively smooth long-period change, the polynomial fitting is automatically judged to be adopted. The polynomial fitting function of Δθ(t) is set as , where the order n = 2 or 3, (b n , b n-1 , ..., b1, b0) are the coefficients of the polynomial. The coefficients are solved by the least squares method to minimize the error between the fitting function P(t) and Δθ(t). By analyzing the spectral characteristics S(ω) and the change trend C(t) of the data, if S(ω) shows a relatively smooth long-period characteristic and C(t) changes slowly, the polynomial fitting is automatically selected. The polynomial coefficients b are solved by the least squares method. j , and j∈(1,2,...,n), such that Minimum; In the B-spline curve step, the control points of the B-spline curve are assumed to be , the node vector is , where the number of control points ranges from 0 to m, and there are a total of m+1. n0 is the degree of the B-spline curve. The higher the degree, the smoother the curve. In the adaptive filter, the input signal of the adaptive filter is x(n'), the expected signal is d(n'), the output signal is y(n'), and the error signal is ; Dynamically adjust the filter parameters w(n') according to the real-time collected data, so that Minimum, among which represents mathematical expectation.

6. The method for rapid identification of camera torsion and high-precision calibration according to claim 1, characterized in that: The S3 algorithm and data processing improvements specifically include: S3.1 Heterogeneous Data Fusion: Multi-source information fusion: Combine inertial measurement unit IMU or laser rangefinder data and fuse multi-source information through extended Kalman filtering; S3.2 Deep learning assisted calibration: Neural network training: Use CNN or Transformer network, input the calibration plate image feature I, and directly output the torsion angle θ; Synthetic data enhancement: Generate synthetic data under different working conditions through simulation and train the model with measured data; Transfer learning: fine-tune model parameters through new data for different SMT machine models or working conditions; S3.3 real-time dynamic calibration: Incremental algorithm: Use the previous calibration results as a priori and update the θ value only through 3-4 new calibration points; Regular global recalibration: To avoid the cumulative error of the incremental algorithm after a long period of operation, regular global recalibration is performed to eliminate the cumulative error; S3.4 Angle calculation and application of results: After obtaining the coordinates of the two calibration points after the initial shooting and work, the coordinates are used to calculate the angles θ1 and θ2 between the camera coordinate system and the calibration point coordinate system. The torsion angle of the camera mounting axis is obtained by calculating the difference between θ2 and θ1, which is used for error compensation to improve the accuracy of camera calibration. When the torsion angle exceeds the torsion angle threshold, the recalibration process is triggered.

7. The method for rapid identification of camera torsion and high-precision calibration according to claim 1, characterized in that: In the heterogeneous data fusion of S3.1, the IMU data includes angular velocity ω and angular acceleration ω˙, and the laser rangefinder data is represented by d laser ; Let the state vector be The observation vector is , T represents the transposition operation; the state transfer matrix is , Δt is the time step; the observation matrix is , where h1 and h2 are determined according to the specific observation model; In the deep learning assisted calibration described in S3.2, the loss function of the neural network is assumed to be , where θ pred is the predicted value, θ true is the true value, R is the total number of samples, r∈(1, 2, ..., R); In transfer learning, the original model parameter is set to w original , the new data is , the fine-tuned model parameters are w new , Q new is the number of samples of new data, and ; In the real-time dynamic calibration of S3.3, the previous calibration result is assumed to be θ prev The new calibration point is , update the value of θ by the least squares method; In the periodic global recalibration, the running time threshold is set as t′ and the measurement task number threshold is set as N′. When the running time exceeds t′ or the number of completed measurement tasks exceeds N′, a comprehensive calibration is performed.

8. A method for rapid identification of camera torsion and high-precision calibration according to any one of claims 1-2 and 4-7, characterized in that: The calibration points are designed to be distributed non-uniformly to enhance the ability to capture nonlinear deformation. The calibration point spacing in the edge area of ​​the calibration plate is set to d edge , the distance between the calibration points in the center area is d center ; Assume that the length of the edge area of ​​the calibration plate is L edge , the length of the central area is L center , then the number of calibration points in the edge area , the number of calibration points in the central area .

9. The method for rapid identification of camera torsion and high-precision calibration according to claim 3, characterized in that: On the basis of the original 9 calibration points, redundant calibration points are added in the edge key area. The number of original calibration points is set to N0=9, and N1 redundant corner points are added. The abnormal points are eliminated through a one-time verification algorithm to obtain the optimal model parameters.

10. The method for rapid identification of camera torsion and high-precision calibration according to claim 9, characterized in that: The step of removing abnormal points by a one-time verification algorithm specifically includes: for a set of calibration point data , randomly select a part of the points as a subset, use the selected subset points to fit the model according to the set error threshold δ, and calculate the number of inliers that meet the model among other points; repeat the above steps, and select the model with the largest number of inliers as the optimal model. After the optimal model is determined, remove the abnormal points that do not meet the model.

Images