Progressive multi-scale self-calibration micron-sized visual calibration method and system

Abstract

The invention relates to the technical field of machine vision, and provides a progressive multi-scale self-calibration micron-sized vision calibration method and system.The progressive multi-scale self-calibration micron-sized vision calibration method comprises the steps that coarse calibration is conducted through a first calibration plate, and first initial estimation values of internal and external parameters of a camera are obtained; acquiring a local key area, performing fine calibration on the local key area through a second calibration plate, and acquiring second initial estimated values of internal and external parameters of the camera; and constructing a target optimization function, and obtaining optimal estimation values of internal and external parameters of the camera. According to the method, the global parameter rough estimation in a large view field and the local parameter high-precision requirement in a local key area are considered, and the camera internal and external parameters are calibrated through a plurality of different scales, so that the camera calibration precision can be improved; the problem that a traditional single-scale calibration method is difficult to consider global parameter estimation and high-precision requirements of a local area in a large field of view is solved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a progressive multi-scale self-calibration micron-level vision calibration method and system, which is particularly suitable for camera calibration in high-precision industrial vision inspection systems. Background Art

[0002] The precision requirements of visual inspection systems in the manufacture of precision electronic products such as PCBs are constantly increasing, reaching the micron level. In the existing technology, camera calibration is the basis of visual measurement, and its accuracy directly affects the overall performance of the system.

[0003] The existing camera calibration methods have the following main problems: the traditional single-scale calibration method is difficult to take into account both the global parameter estimation under a large field of view and the high-precision requirements of the local area. Therefore, it is urgent to develop a high-precision camera calibration method to meet the needs of modern industrial visual inspection. Summary of the invention

[0004] Based on this, in order to solve the problem that the traditional single-scale calibration method is difficult to take into account both the global parameter estimation under a large field of view and the high-precision requirements of the local area, the present invention provides a progressive multi-scale self-calibration micron-level visual calibration method and system, and its specific technical solution is as follows:

[0005] A micron-level visual calibration method for progressive multi-scale self-calibration comprises the following steps:

[0006] Performing a rough calibration using a first calibration plate to obtain a first initial estimate of the camera's intrinsic and extrinsic parameters;

[0007] Acquire a local key area, perform precise calibration on the local key area using a second calibration plate, and acquire a second initial estimation value of the camera's intrinsic and extrinsic parameters;

[0008] A target optimization function is constructed, and optimal estimation values ​​of the camera internal and external parameters are obtained according to the target optimization function, the first initial estimation value, and the second initial estimation value.

[0009] The progressive multi-scale self-calibration micron-level visual calibration method performs a coarse calibration through a first calibration plate to obtain a first initial estimate of the camera's intrinsic and extrinsic parameters, and performs a fine calibration on a local key area through a second calibration plate to obtain a second initial estimate of the camera's intrinsic and extrinsic parameters. According to the target optimization function, the first initial estimate, and the second initial estimate, the optimal estimate of the camera's intrinsic and extrinsic parameters is obtained, which takes into account both the rough estimation of global parameters under a large field of view and the high-precision requirement of local parameters in a local key area. By calibrating the camera's intrinsic and extrinsic parameters at multiple different scales, the accuracy of camera calibration can be improved, solving the problem that the traditional single-scale calibration method is difficult to take into account both the global parameter estimation under a large field of view and the high-precision requirement of the local area.

[0010] Preferably, the progressive multi-scale self-calibration micron-level visual calibration method further comprises the following steps:

[0011] Constructing a feature point accuracy evaluation model for evaluating the accuracy of feature points;

[0012] Evaluate the reliability index of each of the feature points according to the feature point accuracy evaluation model;

[0013] A dynamic optimization mechanism for feature points is constructed, and according to the dynamic optimization mechanism for feature points and the reliability index of each feature point, the weight of each feature point in the optimization process is dynamically adjusted.

[0014] Preferably, the progressive multi-scale self-calibration micron-level visual calibration method further comprises the following steps:

[0015] Construct a multi-scale feature extraction network and a corner location module with integrated attention mechanism;

[0016] A deep learning enhancement model for sub-pixel corner detection is constructed based on the multi-scale feature extraction network and the corner location module.

[0017] Preferably, the progressive multi-scale self-calibration micron-level visual calibration method further comprises the following steps:

[0018] Construct temperature-deformation mapping model;

[0019] Real-time monitoring of changes in temperature parameters related to camera calibration;

[0020] Establishing a temperature drift compensation mechanism according to the temperature parameter change and the temperature-deformation mapping model;

[0021] It is determined whether the temperature drift compensation mechanism is triggered, and if so, temperature drift compensation is performed on the internal and external parameters of the camera.

[0022] Preferably, the minimum value of the objective optimization function is solved by a least squares method or a gradient descent method.

[0023] A progressive multi-scale self-calibration micron-level visual calibration system, used to implement the progressive multi-scale self-calibration micron-level visual calibration method, comprising:

[0024] A coarse calibration module, used for performing coarse calibration through a first calibration plate to obtain a first initial estimation value of the camera's intrinsic and extrinsic parameters;

[0025] A precision calibration module, used for acquiring a local key area, and performing precision calibration on the local key area through a second calibration plate, to acquire a second initial estimation value of the camera's internal and external parameters;

[0026] An evaluation module is used to construct a target optimization function, and obtain optimal estimated values ​​of the camera's internal and external parameters according to the target optimization function, the first initial estimated value, and the second initial estimated value.

[0027] Preferably, the progressive multi-scale self-calibration micron-level visual calibration system further comprises:

[0028] A feature point accuracy evaluation model, used to evaluate the accuracy of feature points, and to evaluate the reliability index of each feature point according to the feature point accuracy evaluation model;

[0029] The weight adjustment module is used to dynamically adjust the weight of each feature point in the optimization process according to the feature point dynamic optimization mechanism and the reliability index of each feature point.

[0030] Preferably, the progressive multi-scale self-calibration micron-level visual calibration system further comprises:

[0031] Multi-scale feature extraction network and corner location module with integrated attention mechanism;

[0032] A deep learning enhancement model is used to detect sub-pixel corner points based on the multi-scale feature extraction network and the corner point location module.

[0033] Preferably, the progressive multi-scale self-calibration micron-level visual calibration system further comprises:

[0034] Temperature-deformation mapping model;

[0035] Temperature monitoring module, used to monitor the changes of temperature parameters related to camera calibration in real time;

[0036] The temperature drift compensation module is used to construct a temperature drift compensation mechanism according to the temperature parameter change and the temperature-deformation mapping model, determine whether the temperature drift compensation mechanism is triggered, and if so, perform temperature drift compensation on the internal and external parameters of the camera.

[0037] Preferably, the evaluation module obtains the minimum value of the target optimization function by a least squares method or a gradient descent method. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0039] Figure 1 It is a schematic diagram of the overall process of a progressive multi-scale self-calibration micron-level visual calibration method in one embodiment of the present invention;

[0040] Figure 2 is a flowchart of a specific method for dynamically adjusting the weights of the feature points during the optimization process in one embodiment of the present invention;

[0041] Figure 3 It is a flowchart of a specific method for constructing a deep learning enhancement model in one embodiment of the present invention;

[0042] Figure 4 It is a flowchart of a specific method for compensating for temperature drift of internal and external parameters of a camera in one embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0044] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only and do not represent the only implementation method.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0046] The “first” and “second” mentioned in the present invention do not represent specific quantities and orders, but are merely used to distinguish names.

[0047] like Figure 1 As shown, the present invention provides a progressive multi-scale self-calibration micron-level visual calibration method, comprising the following steps:

[0048] S1, perform a rough calibration through a first calibration plate to obtain a first initial estimate of the camera's internal and external parameters. The first calibration plate is a large field of view calibration plate, and the camera's vision is greater than 180 degrees. The camera's internal and external parameters include internal parameters and external parameters, the internal parameters include but are not limited to focal length and principal point coordinates, and the external parameters include but are not limited to camera position, posture, rotation matrix, and translation vector.

[0049] Since the rough calibration is performed through the large field of view calibration plate to obtain the first initial estimation value of the camera's intrinsic and extrinsic parameters, it is a conventional technical means in the art, and thus will not be described in detail here.

[0050] S2, for the target visual area, obtaining a local key area, and accurately calibrating the local key area through a second calibration plate to obtain a second initial estimation value of the camera's internal and external parameters.

[0051] The second calibration plate is a high-precision calibration plate. The local key area is precisely calibrated by the high-precision calibration plate, and the second initial estimated values ​​of the camera's internal and external parameters obtained are more accurate than the first initial estimated values.

[0052] S3, constructing a target optimization function, and obtaining optimal estimation values ​​of the camera's internal and external parameters according to the target optimization function, the first initial estimation value, and the second initial estimation value.

[0053] Specifically, the objective optimization function can be expressed as minimizing the difference between the projection position of the feature point observed by the camera on the image plane and the true value position self-check. The objective optimization function is preferably the sum of squares of the reprojection error. Based on the LM algorithm, the camera internal and external parameters (including the first initial estimated value and the second initial estimated value) are continuously adjusted and obtained through multiple iterations, so that the value of the objective optimization function is continuously reduced until it converges to a local optimal solution, achieving the optimal estimation of the global parameters, and obtaining the optimal estimated values ​​of the camera internal and external parameters.

[0054] In addition, based on the objective optimization function of the sum of squares of the reprojection errors, the minimum value of the objective optimization function can also be solved by the gradient descent method.

[0055] As a preferred technical solution, the three-dimensional coordinates of the marking point in the world coordinate system are obtained according to the first initial estimation value and the second initial estimation value, the reprojection coordinates are determined according to the three-dimensional coordinates, the reprojection error is determined according to the reprojection coordinates and the three-dimensional coordinates, and the reprojection error is minimized and optimized using the least squares method to obtain the optimal estimation value of the camera internal and external parameters.

[0056] The accuracy of the second calibration plate is higher than that of the first calibration plate, the accuracy of the second calibration plate is greater than 3 microns, and the accuracy of the second calibration plate is between 0.1 microns and 1.0 microns.

[0057] In steps S1 and S2, the micron-level visual calibration method is based on a progressive calibration method, which gradually optimizes and adjusts the internal and external parameters of the camera, and improves the accuracy and stability of the camera calibration through multiple iterations. In addition, in the process of rough calibration through the first calibration plate and fine calibration of the local key area through the second calibration plate, the spatial relationship between the camera and the first calibration plate and the second calibration plate is continuously adjusted, and the internal and external parameters of the camera are calibrated at different time points to obtain the first initial estimate and the second initial estimate at different spatial and temporal scales. In this way, through progressive multi-scale visual calibration, the calibration accuracy can be continuously optimized and improved, so that the system measurement accuracy reaches the micron level.

[0058] The progressive multi-scale self-calibration micron-level visual calibration method performs a coarse calibration through a first calibration plate to obtain a first initial estimate of the camera's intrinsic and extrinsic parameters, and performs a fine calibration on a local key area through a second calibration plate to obtain a second initial estimate of the camera's intrinsic and extrinsic parameters. According to the target optimization function, the first initial estimate, and the second initial estimate, the optimal estimate of the camera's intrinsic and extrinsic parameters is obtained, which takes into account both the rough estimation of global parameters under a large field of view and the high-precision requirement of local parameters in a local key area. By calibrating the camera's intrinsic and extrinsic parameters at multiple different scales, the accuracy of camera calibration can be improved, solving the problem that the traditional single-scale calibration method is difficult to take into account both the global parameter estimation under a large field of view and the high-precision requirement of the local area.

[0059] As a preferred technical solution, Figure 2 As shown, the progressive multi-scale self-calibration micron-level visual calibration method also includes the following steps:

[0060] S4, construct a feature point accuracy evaluation model for evaluating the accuracy of feature points.

[0061] In camera calibration, the accuracy of feature points has an important impact on the calibration results. The accuracy of feature point detection directly affects the size of reprojection error and backprojection residual, which in turn affects the accuracy of calibration results.

[0062] Specifically, the accuracy of feature point detection is an important factor affecting the reprojection error. If the feature point detection is inaccurate, it will lead to a large reprojection error, which will affect the accuracy of the calibration result. The reprojection error refers to the difference in distance between the points in the 3D world coordinate system projected onto the 2D image plane and the feature points actually detected in the image during the camera calibration process. This difference is usually measured in pixels. The smaller the reprojection error, the higher the accuracy of the camera calibration. The back-projection residual measures the distance between the actual 3D point after projecting it back onto the 2D image plane and the observed 2D point. A smaller back-projection residual indicates a higher accuracy of the camera calibration.

[0063] The estimation accuracy of the camera intrinsic parameters and distortion coefficients has a direct impact on the reprojection error. The number and quality of calibration images also affect the reprojection error. More calibration images and a wider field of view can usually lead to more accurate camera parameter estimates. In addition, the poses of the calibration plate in the image should be diverse to ensure a comprehensive estimation of the camera parameters.

[0064] For the feature point accuracy evaluation model, the feature point accuracy is evaluated by the following method: first, the feature points are detected in the calibration image, and the feature points are usually corner points or intersection points of the chessboard. Then, the camera intrinsic parameters (focal length, principal point coordinates) and distortion coefficients estimated during the calibration process are used to project the points in the three-dimensional world coordinate system onto the two-dimensional image plane. Then, for each projection point, the distance between it and the actual detected feature point is calculated. Finally, for all feature points in the calibration image, their reprojection errors are calculated, and the error ratio between the preset error threshold and the reprojection error is obtained to determine the accuracy of the feature points. Generally speaking, the larger the error ratio value, the higher the accuracy of the feature point.

[0065] In order to improve the accuracy of feature point detection, the following measures can be taken:

[0066] 1. Improve feature point detection accuracy: Use more advanced algorithms or improve image quality to improve the accuracy of feature point detection.

[0067] 2. Increase the number of calibration images: Collect more calibration images, especially those taken from different angles and positions.

[0068] 3. Optimize calibration algorithms: Use more advanced calibration algorithms, such as those based on nonlinear optimization, to improve the accuracy of parameter estimation.

[0069] However, increasing the number of calibration images and collecting more calibration images at different angles means that more time and computing resources are needed. In order to balance the accuracy of camera calibration with time and computing resources, preferably, the specific method for calibrating the number of images includes the following steps: based on the accuracy of the first calibration plate and the second calibration plate, respectively obtain the first reprojection estimated error α1 and the second reprojection estimated error β1 corresponding to the first calibration plate and the second calibration plate; perform a rough calibration based on the first calibration plate to obtain the first camera intrinsic parameter and the first distortion coefficient, and calculate the first reprojection actual error α2 based on the first camera intrinsic parameter and the first distortion coefficient; perform a fine calibration on the local key area based on the second calibration plate to obtain the second camera intrinsic parameter and the second distortion coefficient, and calculate the second reprojection actual error β2 based on the second camera intrinsic parameter and the second distortion coefficient; preset the first reprojection error threshold and the second reprojection error threshold; according to the formula Calculate the number of calibration images. G1 and G2 represent the preset calibration image bases of the first calibration plate and the preset calibration image bases of the second calibration plate, respectively, which can be set by technical personnel, γ1 and γ2 represent the preset calibration image adjustment coefficients of the first calibration plate and the preset calibration image adjustment coefficients of the second calibration plate, respectively, which are set by technical personnel, C1 and C2 represent the preset calibration image number correction factors of the first calibration plate and the preset calibration image number correction factors of the second calibration plate, respectively, and α3 and β3 represent the first reprojection error threshold and the second reprojection error threshold, respectively.

[0070] The purpose of setting C1 and C2 is to take into account the influence of external factors on the reprojection error, such as system temperature, vibration value, temperature and humidity value, and light brightness. C1 and C2 can be obtained by calculating the difference between the characteristic values ​​of several normalized external factors and the corresponding standard values, and then calculating the weighted average of the ratios based on the ratio between the difference and the standard value. The characteristic values ​​of the external factors refer to temperature value, vibration value, light brightness, etc.

[0071] The first reprojection estimated error and the second reprojection estimated error corresponding to the first calibration plate and the second calibration plate can be set by technicians based on experience.

[0072] From the formula It can be seen that by setting the first reprojection error threshold and the second reprojection error threshold, the influence of external factors on the reprojection error is taken into account, and based on the preset calibration image base of the first calibration plate, the preset calibration image base of the second calibration plate, the first reprojection estimated error α1 and the second reprojection estimated error β1, the first reprojection actual error α2, and the second reprojection actual error β2, a comprehensive measure of multiple different factors is taken, and the first calibration plate used for coarse calibration of a large field of view and the first calibration plate used for fine calibration of local key areas are combined together, which can balance the accuracy of camera calibration with time and computing resources to obtain an appropriate number of calibration images.

[0073] Preferably, it can also be based on the formula Calculate the number of iterations of the target optimization function, the number of iterations n is the preset value, [] indicates the rounding function, 2 n Indicates batch size.

[0074] The target optimization function is preferably the sum of squares of the reprojection error. Based on the LM algorithm, the camera internal and external parameters (including the first initial estimated value and the second initial estimated value) are continuously adjusted and obtained through multiple iterations, so that the value of the target optimization function is continuously reduced until it converges to a local optimal solution or the number of iterations is equal to N, and the optimal estimation of the global parameters is achieved, and the optimal estimated value of the camera internal and external parameters is obtained. In this way, the acquisition of the number of iterations not only takes into account the influence of external factors on the reprojection error, but also comprehensively weighs multiple different factors based on the preset calibration image cardinality of the first calibration plate, the preset calibration image cardinality of the second calibration plate, the first reprojection estimated error α1 and the second reprojection estimated error β1, the first reprojection actual error α2, and the second reprojection actual error β2, and combines the first calibration plate used for rough calibration of a large field of view and the first calibration plate used for fine calibration of a local key area, so as to obtain a more appropriate and satisfactory number of iterations, avoid the problem of overfitting caused by too many iterations, and ensure the accuracy of camera calibration.

[0075] S5, evaluating the reliability index of each feature point according to the feature point accuracy evaluation model. The reliability index of each feature point is equal to the error ratio between a preset error threshold and the reprojection error.

[0076] S6, constructing a dynamic optimization mechanism for feature points, and dynamically adjusting the weight of each feature point in the optimization process according to the dynamic optimization mechanism for feature points and the reliability index of each feature point.

[0077] Here, according to the size of the reliability index of each feature point, the weight of each feature point in the optimization process is dynamically adjusted through the feature point dynamic optimization mechanism. The feature point dynamic optimization mechanism can be expressed as follows: the larger the reliability index of the feature point, the smaller the corresponding weight in the optimization process, that is, the weight of the feature point in the optimization process is inversely proportional to its reliability index.

[0078] By constructing a dynamic optimization mechanism for feature points, and dynamically adjusting the weight of each feature point in the optimization process according to the dynamic optimization mechanism for feature points and the reliability index of each feature point, the progressive multi-scale self-calibration micron-level visual calibration method can perform adaptive weight adjustment and calibration according to the accuracy and reliability of different feature points, which is conducive to further improving the visual calibration accuracy and the degree of automation and intelligence.

[0079] As a preferred technical solution, Figure 3 As shown, the progressive multi-scale self-calibration micron-level visual calibration method also includes the following steps:

[0080] S7, builds a multi-scale feature extraction network and a corner localization module with integrated attention mechanism.

[0081] Multi-scale feature extraction network is an important network architecture in deep learning. It can effectively handle multi-scale problems, which is particularly important in computer vision tasks. Multi-scale problems refer to the situation in image processing where the same object or scene appears at different scales (i.e., size and resolution). For example, in tasks such as object detection, image segmentation, and video analysis, the target object may appear in different sizes and proportions in the image.

[0082] The purpose of designing the multi-scale feature extraction network is to improve the robustness of the model to scale changes, so that it can extract rich feature information at different scales. Preferably, the multi-scale feature extraction network is a feature pyramid structure. Since it belongs to conventional technical means in this field, it will not be described here.

[0083] The corner point localization module integrates the attention mechanism, which can effectively improve the accuracy of corner point localization.

[0084] S8, constructing a deep learning enhancement model for sub-pixel corner detection based on the multi-scale feature extraction network and the corner location module.

[0085] A deep learning enhancement model for sub-pixel corner detection is constructed based on the multi-scale feature extraction network and the corner positioning module, which can not only extract rich image feature information at different scales, but also improve the accuracy of corner positioning to the sub-pixel level, thereby reducing reprojection errors and improving the accuracy of camera vision calibration.

[0086] As a preferred technical solution, Figure 4 As shown, the progressive multi-scale self-calibration micron-level visual calibration method further includes the following steps:

[0087] S9, constructing a temperature-deformation mapping model. Specifically, the actual distortion coefficient of the camera at multiple different temperatures can be obtained through big data technology to construct a temperature-deformation mapping model. For each temperature, multiple actual distortion coefficients are obtained, and the average value of the multiple actual distortion coefficients is taken as the final actual distortion coefficient. The temperature and the final actual distortion coefficient are fitted to obtain a function curve between the temperature and the distortion coefficient and use it as the temperature-deformation mapping model.

[0088] S10, real-time monitoring of changes in temperature parameters related to camera calibration, wherein the related temperature parameters include ambient temperature, camera lens and other optical components real-time temperature.

[0089] S11, constructing a temperature drift compensation mechanism according to the temperature parameter change and the temperature-deformation mapping model. The temperature drift compensation mechanism can be expressed as obtaining the actual distortion coefficient in real time according to the temperature parameter change and the temperature-deformation mapping model, performing corresponding temperature drift compensation actions according to the actual distortion coefficient, and mapping corresponding camera internal and external parameter temperature drift compensation values ​​to different actual distortion coefficients.

[0090] S12, determining whether the temperature drift compensation mechanism is triggered, and if so, performing temperature drift compensation on the internal and external parameters of the camera. Determining whether the actual distortion coefficient is greater than a preset distortion coefficient threshold, and if so, triggering the temperature drift compensation mechanism, and performing temperature drift compensation on the internal and external parameters of the camera.

[0091] Through the temperature-deformation mapping model and temperature drift compensation mechanism, the robustness is significantly enhanced, the anti-interference ability of the visual calibration system is improved, and the impact of temperature drift on the system calibration accuracy is reduced.

[0092] Furthermore, in the formula In calculating the number of calibration images, C1 and C2 represent the calibration image number correction factor preset by the first calibration plate and the calibration image number correction factor preset by the second calibration plate, respectively, and both C1 and C2 include the first weighted average value of the temperature drift compensation value of the camera internal and external parameters corresponding to the different actual distortion coefficient mappings within the preset time period obtained according to the temperature-deformation mapping model and the temperature drift compensation mechanism. That is, C1 and C2 can be obtained by calculating the difference between the characteristic values ​​of several external factors other than the temperature factor after normalization and the corresponding standard value, and finally calculating the average of the second weighted average value of the ratio and the first weighted average value according to the ratio between the difference and the standard value.

[0093] In this way, by setting the first reprojection error threshold and the second reprojection error threshold, the influence of external factors, especially temperature drift, on the reprojection error is taken into account. Based on the preset calibration image base of the first calibration plate, the preset calibration image base of the second calibration plate, the first reprojection estimated error α1 and the second reprojection estimated error β1, the first reprojection actual error α2 and the second reprojection actual error β2, a comprehensive measure is taken of multiple different factors. The first calibration plate used for coarse calibration of a large field of view and the first calibration plate used for fine calibration of local key areas are combined together. This can balance the accuracy, time and computing resources of camera calibration to obtain a more appropriate number of calibration images.

[0094] As a preferred technical solution, the present invention also provides a progressive multi-scale self-calibration micron-level visual calibration system, which is used to implement the progressive multi-scale self-calibration micron-level visual calibration method, which includes a coarse calibration module, a fine calibration module and an evaluation module.

[0095] The coarse calibration module is used to perform coarse calibration through a first calibration plate to obtain first initial estimates of camera internal and external parameters; the fine calibration module is used to obtain a local key area for a target visual area, and perform fine calibration on the local key area through a second calibration plate to obtain second initial estimates of camera internal and external parameters; the evaluation module is used to construct a target optimization function, and obtain optimal estimates of camera internal and external parameters according to the target optimization function, the first initial estimates and the second initial estimates; wherein the accuracy of the second calibration plate is higher than that of the first calibration plate.

[0096] The evaluation module obtains the minimum value of the target optimization function by using the least squares method or the gradient descent method.

[0097] The target optimization function can be expressed as minimizing the difference between the projection position of the feature point observed by the camera on the image plane and the true value position self-check. The target optimization function is preferably the sum of squares of the reprojection error, and the target optimization function is iteratively optimized based on the gradient descent method or the LM algorithm, so that the value of the target optimization function is continuously reduced until it converges to a local optimal solution, achieving the optimal estimation of the global parameters, and obtaining the optimal estimated values ​​of the internal and external parameters of the camera.

[0098] The progressive multi-scale self-calibration micron-level visual calibration system performs a coarse calibration through a first calibration plate to obtain a first initial estimate of the camera's internal and external parameters, and performs a fine calibration on a local key area through a second calibration plate to obtain a second initial estimate of the camera's internal and external parameters. According to the target optimization function, the first initial estimate, and the second initial estimate, the optimal estimate of the camera's internal and external parameters is obtained, which takes into account both the rough estimation of global parameters under a large field of view and the high-precision requirement of local parameters in a local key area. By calibrating the camera's internal and external parameters at multiple different scales, the accuracy of camera calibration can be improved, solving the problem that the traditional single-scale calibration method is difficult to take into account both the global parameter estimation under a large field of view and the high-precision requirement of the local area.

[0099] As a preferred technical solution, the progressive multi-scale self-calibration micron-level visual calibration system also includes a feature point accuracy evaluation model and a weight adjustment module.

[0100] The feature point accuracy evaluation model is used to evaluate the accuracy of feature points, and to evaluate the reliability index of each feature point based on the feature point accuracy evaluation model; the weight adjustment module is used to dynamically adjust the weight of each feature point in the optimization process based on the feature point dynamic optimization mechanism and the reliability index of each feature point.

[0101] According to the size of the reliability index of each feature point, the weight of each feature point in the optimization process is dynamically adjusted through the feature point dynamic optimization mechanism. The feature point dynamic optimization mechanism can be expressed as follows: the larger the reliability index of the feature point, the smaller the corresponding weight in the optimization process, that is, the weight of the feature point in the optimization process is inversely proportional to its reliability index.

[0102] By constructing a dynamic optimization mechanism for feature points, and dynamically adjusting the weight of each feature point in the optimization process according to the dynamic optimization mechanism for feature points and the reliability index of each feature point, the progressive multi-scale self-calibration micron-level visual calibration method can perform adaptive weight adjustment and calibration according to the accuracy and reliability of different feature points, which is conducive to further improving the visual calibration accuracy and the degree of automation and intelligence.

[0103] Preferably, the progressive multi-scale self-calibrated micron-level visual calibration system further includes a corner location module of a multi-scale feature extraction network integrated with an attention mechanism and a deep learning enhancement model. The deep learning enhancement model is used to detect sub-pixel corners based on the multi-scale feature extraction network and the corner location module, so as to extract rich image feature information at different scales, improve the accuracy of corner location, reduce reprojection errors and improve the accuracy of camera visual calibration.

[0104] The progressive multi-scale self-calibration micron-level visual calibration system also includes a temperature-deformation mapping model, a temperature monitoring module and a temperature drift compensation module.

[0105] The temperature monitoring module is used to monitor the changes in temperature parameters related to camera calibration in real time; the temperature drift compensation module is used to construct a temperature drift compensation mechanism according to the changes in the temperature parameters and the temperature-deformation mapping model, and determine whether the temperature drift compensation mechanism is triggered. If so, temperature drift compensation is performed on the internal and external parameters of the camera.

[0106] Through the temperature-deformation mapping model and temperature drift compensation mechanism, the robustness is significantly enhanced, the anti-interference ability of the visual calibration system is improved, and the impact of temperature drift on the system calibration accuracy is reduced.

[0107] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A progressive multi-scale self-calibration micron-level visual calibration method, characterized in that: The micron-level visual calibration method of progressive multi-scale self-calibration comprises the following steps: Performing a rough calibration using a first calibration plate to obtain a first initial estimate of the camera's intrinsic and extrinsic parameters; Acquire a local key area, perform precise calibration on the local key area using a second calibration plate, and acquire a second initial estimation value of the camera's intrinsic and extrinsic parameters; A target optimization function is constructed, and optimal estimation values ​​of the camera internal and external parameters are obtained according to the target optimization function, the first initial estimation value, and the second initial estimation value.

2. A progressive multi-scale self-calibration micron-level visual calibration method as claimed in claim 1, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration method further comprises the following steps: Constructing a feature point accuracy evaluation model for evaluating the accuracy of feature points; Evaluate the reliability index of each of the feature points according to the feature point accuracy evaluation model; A dynamic optimization mechanism for feature points is constructed, and according to the dynamic optimization mechanism for feature points and the reliability index of each feature point, the weight of each feature point in the optimization process is dynamically adjusted.

3. A progressive multi-scale self-calibration micron-level visual calibration method as claimed in claim 2, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration method further comprises the following steps: Construct a multi-scale feature extraction network and a corner location module with integrated attention mechanism; A deep learning enhancement model for sub-pixel corner detection is constructed based on the multi-scale feature extraction network and the corner location module.

4. The progressive multi-scale self-calibration micron-level visual calibration method according to claim 3, characterized in that: The micron-level visual calibration method of progressive multi-scale self-calibration also includes the following steps: Construct temperature-deformation mapping model; Real-time monitoring of changes in temperature parameters related to camera calibration; Establishing a temperature drift compensation mechanism according to the temperature parameter change and the temperature-deformation mapping model; It is determined whether the temperature drift compensation mechanism is triggered, and if so, temperature drift compensation is performed on the internal and external parameters of the camera.

5. The progressive multi-scale self-calibration micron-level visual calibration method according to claim 4, characterized in that: The minimum value of the objective optimization function is solved by the least squares method or the gradient descent method.

6. A progressive multi-scale self-calibration micron-level visual calibration system, used to implement the progressive multi-scale self-calibration micron-level visual calibration method as described in any one of claims 1-5, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration system comprises: A coarse calibration module, used for performing coarse calibration through a first calibration plate to obtain a first initial estimation value of the camera's intrinsic and extrinsic parameters; A precision calibration module, used for acquiring a local key area, and performing precision calibration on the local key area through a second calibration plate, to acquire a second initial estimation value of the camera's internal and external parameters; An evaluation module is used to construct a target optimization function, and obtain optimal estimated values ​​of the camera's internal and external parameters according to the target optimization function, the first initial estimated value, and the second initial estimated value.

7. A progressive multi-scale self-calibration micron-level visual calibration system as claimed in claim 6, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration system further includes: A feature point accuracy evaluation model, used to evaluate the accuracy of feature points, and to evaluate the reliability index of each feature point according to the feature point accuracy evaluation model; The weight adjustment module is used to dynamically adjust the weight of each feature point in the optimization process according to the feature point dynamic optimization mechanism and the reliability index of each feature point.

8. The progressive multi-scale self-calibration micron-level visual calibration system according to claim 7, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration system further includes: Multi-scale feature extraction network and corner location module with integrated attention mechanism; A deep learning enhancement model is used to detect sub-pixel corner points based on the multi-scale feature extraction network and the corner point location module.

9. The progressive multi-scale self-calibration micron-level visual calibration system according to claim 8, characterized in that: The progressive multi-scale self-calibration micron-level visual calibration system further includes: Temperature-deformation mapping model; Temperature monitoring module, used to monitor the changes of temperature parameters related to camera calibration in real time; The temperature drift compensation module is used to construct a temperature drift compensation mechanism according to the temperature parameter change and the temperature-deformation mapping model, determine whether the temperature drift compensation mechanism is triggered, and if so, perform temperature drift compensation on the internal and external parameters of the camera.

10. The progressive multi-scale self-calibration micron-level visual calibration system according to claim 9, characterized in that: The evaluation module obtains the minimum value of the target optimization function by using the least squares method or the gradient descent method.

Images