Camera calibration error analysis method and system based on Kinect V2

Abstract

The invention provides a camera calibration error analysis method and system based on Kinect V2. The method comprises the following steps: calculating internal parameters and external parameters of a camera; applying a calibration result to a new image, calculating a distance between a re-projection point and an actual observation point, evaluating the calibration result, carrying out statistical analysis on re-projection errors of all images, and calculating statistical indexes such as an average error and a maximum error; the method comprises the following steps: adjusting input parameters such as a calibration plate position and a camera attitude, observing the change condition of a calibration result, determining the influence degree of each input parameter on the calibration result, carrying out eigenvalue decomposition on a covariance matrix of a calibration error, and analyzing an error source and the influence degree of the error source on the calibration result; if some error sources are found, calibration is carried out again by adding calibration points or changing calibration conditions. The system comprises a parameter calculation module, an error analysis module and a calibration optimization module. The accuracy and reliability of the calibration result are improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of three-dimensional reconstruction and human-computer interaction, and in particular to a camera calibration error analysis method and system based on Kinect V2. Background Art

[0002] Kinect V2 is a sensor based on a depth camera, which is widely used in human-computer interaction, robot navigation, virtual reality and other fields. In the field of automotive intelligent vision, Kinect V2 can also be used to realize functions such as vehicle surrounding environment perception and driver behavior recognition. However, due to the errors in the camera itself and the calibration process, the accuracy and stability of the camera are affected, which in turn affects the performance of the automotive intelligent vision system. Camera calibration error analysis based on Kinect V2 is an important research direction in the field of automotive intelligent vision. Its main purpose is to improve the accuracy and stability of camera calibration by analyzing and studying the errors in the camera calibration process, thereby improving the performance of the automotive intelligent vision system. In recent years, with the rapid development of computer vision and robotics technology, the research on camera calibration error analysis based on depth cameras has become a hot research field. In this context, Kinect V2 camera, as a commonly used depth camera, is widely used in computer vision and robot navigation and other fields. There are some research results on the calibration problem of Kinect V2 camera. For example, the RGB camera of Kinect V2 was calibrated using the MATLAB calibration toolbox based on Zhang Zhengyou's calibration method, and the intrinsic parameters and pixel errors of the sensor were solved. At the same time, the calibration library that comes with Kinect V2 can also be used for position calibration between the color camera and the infrared camera.

[0003] Prior art 1, application number: CN 202311220691.1 discloses a simulation analysis method, system, device and medium for extrinsic calibration of vehicle-mounted cameras, the method comprising: constructing a geometric model of three scene elements: a car, a camera and a target; according to the geometric model, after coordinate transformation, estimating the homography transformation from the imaging plane to the target plane, and obtaining the homography matrix; performing homography transformation on the pixel coordinates of the simulation image according to the homography matrix, and then determining the pixel values ​​corresponding to all pixel points in the imaging plane after texture mapping; setting simulation parameters, performing imaging simulation according to the pixel values, and obtaining a simulation image of the target; calibrating the extrinsic parameters of the vehicle-mounted camera according to the camera extrinsic calibration algorithm and the simulation image of the target, and obtaining the calibration result; taking the set geometric parameters as the true value, and using the true value and the simulation image of the target to evaluate the error of the calibration result. Although it can improve the accuracy of the calibration analysis of the extrinsic parameters of the vehicle-mounted camera and improve the efficiency of R&D and mass production testing; however, due to factors such as lighting conditions and changes in the internal and external parameters of the camera, there are errors in the calibration process of the camera and the target, which affects the accuracy of the simulation analysis results.

[0004] Prior Art Two, Application No.: CN202311426672.4 discloses an online calibration optimization method for camera parameters, which solves the problems of inaccurate camera parameters caused by sensor lens distortion in monocular vision measurement and the inability to perform online calibration in actual use. It provides an online calibration optimization method for camera parameters, which collects the calibration board image through a vision sensor, reads and analyzes the image; considering the non-linear distortion of the camera, inputs the calibration board image into the camera non-linear imaging model, and uses the Zhang's method for calibration to obtain the initial camera parameters; judges the accuracy of the camera parameter calibration according to the reprojection error of the camera, and uses the particle swarm algorithm to optimize the camera non-linear imaging model to obtain the optimized camera parameters, realizing the online calibration optimization of the parameters. Although it can solve the problem of difficult accurate calibration of cameras in coal mines, it can also improve the accuracy of camera calibration results, which is of great significance for image acquisition and vision measurement in coal mines. However, due to the poor light in coal mines, it is difficult to adjust the parameters of the sensor lens, resulting in the need to further improve the accuracy of the optimized camera parameters.

[0005] Prior Art Three, Application No.: CN 202311476039.6 discloses a camera calibration method and system based on stereo vision, which uses a method of calculating the calibration images of a three-eye camera and training the calibration images with a deep learning algorithm. Although it realizes the functions of accurately collecting calibration image data information and optimizing calculation, achieves the effects of accurate calibration image data of the three-eye camera and optimized calculation data processing, and uses a module for receiving calibration image corner points to calculate the calibration error and a ranging comparison and analysis intelligent optimization module, realizes the perfect preprocessing process of the computer receiving the calibration image and the machine learning training function of the algorithm data for calibrating the camera connected to the computer, achieving the effect of eliminating interference data information, and also achieving the technical effects of accurate and intelligent extraction of calibration image corner point data and optimizing and improving the algorithm process of the camera calibration image. However, the deep learning algorithm process is complex and overly dependent on a large amount of data, making the implementation process of the scheme complex and the calibration process time-consuming and laborious.

[0006] Currently, Prior Art One, Prior Art Two, and Prior Art Three have problems that the errors existing in the camera itself and the calibration process affect the accuracy and stability of the camera, and further affect the performance of the automotive intelligent vision system. Therefore, the present invention provides a method and system for verifying camera calibration error analysis based on Kinect V2. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a method, which includes the following steps:

[0008] Collect multiple sets of image data using a Kinect V2 camera, and perform preprocessing operations such as denoising and grayscale conversion on the collected images; select a calibration board, extract the corner coordinates of the calibration board in each image, use tools for camera calibration, and calculate the internal and external parameters of the camera;

[0009] Apply the calibration results to new images, calculate the distance between the reprojection points and the actual observation points, evaluate the calibration results, perform statistical analysis on the reprojection errors of all images, and calculate statistical indicators such as the average error and the maximum error; by adjusting the input parameters of the calibration board position and camera pose, observe the changes in the calibration results, determine the influence degree of each input parameter on the calibration results, perform eigenvalue decomposition on the covariance matrix of the calibration error, and analyze the error sources and their influence degrees on the calibration results;

[0010] If it is found that some error sources, re - calibrate by increasing the calibration points or changing the calibration conditions; use a non - linear optimization algorithm to optimize the calibration results; collect new images for validating the calibration results, calculate the reprojection error and statistical analysis of the average error for the new images.

[0011] Optionally, the process of calculating the internal and external parameters of the camera includes the following steps:

[0012] Obtain a checkerboard calibration board for camera positioning and recognition, and use a Kinect V2 camera to take multiple sets of images from different angles and distances; perform image calibration through a camera calibration toolbox and extract the corner points on the checkerboard calibration board;

[0013] Perform preprocessing operations such as undistortion, segmentation, and edge detection on the images, extract the positions of the calibration points on the checkerboard calibration board, and calculate the internal and external parameters of the camera according to the positions; among them, the internal parameters include the focal length, principal point coordinates, and camera distortion coefficients, which are used to describe the properties of the camera itself; the external parameters include the position and pose of the camera, which are used to describe the position and orientation of the camera in the world coordinate system;

[0014] After completing the camera calibration, it is necessary to verify the calibration results, use other measurement tools or cameras for comparison, and check the accuracy and stability of the calibration results; if it is found that the calibration results have errors or deviations exceeding the preset threshold, readjust the positions of the calibration board and the camera and re - perform the calibration process; after calibration, apply the calibration results to specific measurement or calculation tasks.

[0015] Optionally, the process of analyzing the error sources and their influence degrees on the calibration results includes the following steps:

[0016] Using the internal and external camera parameters obtained through calibration, project the points in the three-dimensional world coordinate system onto the two-dimensional image plane to obtain reprojection points; calculate the Euclidean distance between the reprojection points and the actual observed points, i.e., the reprojection error; calculate the average value of the reprojection errors of all images, and calculate the maximum value of the reprojection errors in all images; analyze the distribution of the reprojection errors to identify the main sources of errors;

[0017] Adjust the input parameters such as the position of the calibration board and the camera pose, observe the changes in the calibration results, record the reprojection errors after each adjustment, and analyze the influence of different input parameters on the errors; calculate the covariance matrix of the calibration errors, and perform eigenvalue decomposition to analyze the error sources and their influence degrees on the calibration results. The eigenvalues and eigenvectors of the covariance matrix identify the main sources of errors;

[0018] Draw a histogram of the reprojection errors to observe the distribution of the errors. The histogram shows the frequency distribution of the errors; identify the main sources of errors through the error distribution diagram; according to the results of the error distribution analysis, correct the calibration parameters and perform calibration again.

[0019] Optionally, the process of analyzing the distribution of the reprojection errors includes the following steps:

[0020] Perform statistical analysis on the reprojection error data, and calculate the basic statistics such as the mean, median, and standard deviation of the errors. The basic statistics can reflect the overall distribution characteristics of the errors. For example, the mean reflects the central position of the errors, and the standard deviation reflects the degree of dispersion of the errors;

[0021] By observing the skewness and kurtosis of the error distribution, judge whether the error distribution is symmetric and whether there are characteristics of sharp peaks or flatness; a skewness of zero indicates a symmetric distribution, a positive skewness indicates a right-skewed distribution, a negative skewness indicates a left-skewed distribution, a kurtosis greater than 3 indicates a sharp peak characteristic of the distribution, and a kurtosis less than 3 indicates a flat distribution;

[0022] Extract the local characteristics of the error distribution. By calculating the local extreme points of the error distribution, identify the peak regions and valley regions in the error distribution; classify and quantify the error sources according to the statistical characteristics and morphological characteristics of the error distribution.

[0023] Optionally, the process of using the eigenvalues and eigenvectors of the covariance matrix to identify the main sources of errors includes the following steps:

[0024] During the calibration process, record the reprojection error data after each adjustment of the calibration board position and the camera pose; based on the collected reprojection error data, calculate the covariance matrix of the error vector, and the covariance matrix reflects the correlation and variance between the dimensions of the error vector; perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and the corresponding eigenvectors, sort the eigenvalues by size, and the larger the eigenvalue, the greater the contribution of the corresponding eigenvector to the error distribution.

[0025] By analyzing the magnitudes of the eigenvalues, identify the main error sources. The error dimension corresponding to the eigenvector with a larger eigenvalue is the main error source; calculate the proportion of each eigenvalue to the sum of all eigenvalues to quantify the influence degree of each error source on the calibration result. The larger the eigenvalue proportion, the greater the influence of the corresponding error source on the calibration result.

[0026] Through eigenvector and eigenvalue analysis, obtain the classification of error sources.

[0027] Optionally, the process of identifying the main sources of error through the error distribution diagram includes the following steps:

[0028] Obtain the reprojection error data of all images, group the reprojection error data according to intervals, and draw a histogram. The horizontal axis of the histogram represents the magnitude of the error, and the vertical axis represents the frequency of the error.

[0029] Observe the overall shape of the histogram, identify the characteristics of the error distribution, whether it shows a normal distribution or a skewed distribution; determine the peak position in the histogram, analyze the error interval corresponding to the peak, and identify the concentrated area of the error; observe the tail of the histogram, analyze the distribution of extreme error values, and identify possible abnormal error sources.

[0030] Based on the distribution characteristics of the histogram, identify the main sources of error; if the peak is concentrated in a specific interval, it may indicate that the parameters or conditions corresponding to this interval are the main error sources; analyze the secondary peaks or tail distributions of the histogram to identify secondary error sources; evaluate their contribution to the overall error by calculating the frequency proportion of different error intervals; combine the analysis results of the covariance matrix to confirm the correlation between error sources and identify possible error propagation paths.

[0031] Optionally, the process of identifying the main and secondary sources of error includes the following steps:

[0032] Observe the peak position in the histogram to determine the interval where the error is most concentrated. If the peak is concentrated in the interval where the error value is between 0.5 and 1.0, then infer that the parameters or conditions in this interval have the greatest impact on the error; calculate the proportion of the error frequency in the peak interval to the total error frequency to evaluate its contribution to the overall error.

[0033] In a histogram, observe the tail of the histogram, which contains extreme error values, analyze the shape and frequency of the tail distribution, and identify possible sources of abnormal errors;

[0034] Based on the analysis results of the main error sources, secondary error sources, and error propagation paths, conduct a comprehensive evaluation to confirm which parameters or conditions have the greatest impact on the overall error, which are the secondary error sources, and how the error propagates in the system; finally confirm the main and secondary error sources of the error.

[0035] Optionally, the process of finally confirming the main and secondary error sources of the error includes the following steps:

[0036] Collect all error data, including error data in the peak interval of the histogram, error data in the tail distribution, and data on the error propagation path; organize the collected data and classify it into main error source data, secondary error source data, and error propagation path data;

[0037] Calculate the proportion of the error frequency in the peak interval to the total error frequency; calculate the proportion of the tail error frequency to the total error frequency;

[0038] Analyze how the error propagates in the system, identify the main and secondary paths of error propagation; evaluate the degree of influence of the error propagation path on the final result.

[0039] Optionally, the process of re - calibration includes the following steps:

[0040] Through the statistical analysis of the calibration results, identify the main error sources affecting the calibration accuracy; conduct a detailed analysis of the identified error sources to determine their specific influence methods and degrees on the calibration results; camera distortion mainly affects the shape of straight lines in the image, and the calibration board position error mainly affects the accuracy of corner point extraction;

[0041] For the identified main error sources, increase the number of rows and columns of the calibration board; place the calibration board at different angles and distances;

[0042] At different angles and distances, re - collect multiple groups of image data; use tools to re - extract the corner point coordinates of the calibration board in each image, re - perform camera calibration, and calculate the internal and external parameters of the camera.

[0043] A camera calibration error analysis system based on Kinect V2 provided by the present invention includes:

[0044] A parameter calculation module, which is responsible for using the Kinect V2 camera to collect multiple groups of image data, performing denoising and grayscale pre - processing operations on the collected images; selecting a calibration board, extracting the corner point coordinates of the calibration board in each image, using tools for camera calibration, and calculating the internal and external parameters of the camera;

[0045] An error analysis module, which is responsible for applying the calibration result to a new image, calculating the distance between the reprojection point and the actual observation point, evaluating the calibration result, statistically analyzing the reprojection errors of all images, and calculating statistical indicators such as the average error and the maximum error; by adjusting the input parameters of the calibration board position and the camera pose, observing the changes in the calibration result, determining the influence degree of each input parameter on the calibration result, performing eigenvalue decomposition on the covariance matrix of the calibration error, and analyzing the error sources and their influence degrees on the calibration result;

[0046] A calibration optimization module, which is responsible for re - calibrating if it is found that some error sources by adding calibration points or changing the calibration conditions; optimizing the calibration result using a non - linear optimization algorithm; collecting new images for verifying the calibration result, calculating the reprojection error of the new images and statistically analyzing the average error.

[0047] The data acquisition and calibration of the present invention obtain high - quality image data to ensure the accuracy of the calibration process; denoising and grayscale processing improve the clarity and contrast of the images, facilitating corner extraction; accurately extracting the corner coordinates of the calibration board provides accurate data for calibration; calculating the internal and external parameters of the camera through tools provides basic data for error analysis and compensation. Significance: Ensure the accuracy and reliability of the calibration data, providing a solid foundation for error analysis; obtain the accurate internal and external parameters of the camera, providing an accurate camera model for image processing and 3D reconstruction. Error evaluation and decomposition, evaluating the accuracy of the calibration result through the reprojection error, quantifying the error magnitude; calculating statistical indicators such as the average error and the maximum error to comprehensively understand the error distribution; by adjusting the input parameters, observing the changes in the calibration result, determining the influence degree of each parameter on the calibration result; through eigenvalue decomposition, analyzing the main error sources and their influence degrees on the calibration result. Significance: Quantify the calibration error through the reprojection error and statistical analysis, providing a basis for error compensation; identify the main error sources through sensitivity analysis and principal component analysis, providing a direction for targeted compensation; providing a theoretical basis and data support for error compensation and optimization. Error compensation and verification, re - calibrating by adding calibration points or changing the calibration conditions to reduce the influence of fixed error sources; optimizing the calibration result using a non - linear optimization algorithm to further reduce the error; collecting new image data for verifying the accuracy of the calibration result; calculating the reprojection error of the new images and statistically analyzing the average error to ensure the reliability of the calibration result. Significance: Reduce the calibration error and improve the accuracy of the calibration result through re - calibration and non - linear optimization; verify the calibration result through new image data to ensure the reliability and stability of the calibration result; provide empirical data and improvement directions for the calibration work, continuously improving the calibration accuracy.

[0048] Other features and advantages of the present invention will be set forth in the following description, and in part will be obvious from the description, or may be learned by practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the structure particularly pointed out in the written description and the drawings.

[0049] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0050] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the drawings:

[0051] Figure 1 It is a flowchart of the camera calibration error analysis method based on Kinect V2 in Embodiment 1 of the present invention;

[0052] Figure 2 It is a process diagram of calculating the internal parameters and external parameters of the camera in Embodiment 2 of the present invention;

[0053] Figure 3 It is a process diagram of analyzing the error sources and their influence degrees on the calibration results in Embodiment 3 of the present invention;

[0054] Figure 4 It is a process diagram of analyzing the distribution of reprojection errors in Embodiment 4 of the present invention;

[0055] Figure 5 It is a process diagram of identifying the main sources of errors of the eigenvalues and eigenvectors of the covariance matrix in Embodiment 5 of the present invention;

[0056] Figure 6 It is a process diagram of identifying the main sources of errors through the error distribution diagram in Embodiment 6 of the present invention;

[0057] Figure 7 It is a process diagram of identifying the main and secondary sources of errors in Embodiment 7 of the present invention;

[0058] Figure 8 It is a process diagram of finally confirming the main sources of errors and the secondary error sources in Embodiment 8 of the present invention;

[0059] Figure 9 It is a process diagram of re - calibrating in Embodiment 9 of the present invention;

[0060] Figure 10 It is a block diagram of the camera calibration error analysis system based on Kinect V2 in Embodiment 10 of the present invention. Detailed Embodiments

[0061] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. In the embodiments of the present application, the singular forms "a", "said" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0063] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0064] Embodiment 1: As Figure 1 shown, the embodiment of the present invention provides a method for analyzing the camera calibration error based on Kinect V2, which includes the following steps:

[0065] S100: Use the Kinect V2 camera to collect multiple groups of image data, and perform preprocessing operations such as denoising and grayscale conversion on the collected images; select a calibration board, extract the corner coordinates of the calibration board in each image, use tools for camera calibration, and calculate the internal parameters and external parameters of the camera;

[0066] S200: Apply the calibration result to a new image, calculate the distance between the reprojection point and the actual observation point, evaluate the calibration result, perform statistical analysis on the reprojection errors of all images, and calculate statistical indicators such as the average error and the maximum error; by adjusting input parameters such as the position of the calibration board and the camera pose, observe the change of the calibration result, determine the influence degree of each input parameter on the calibration result, perform eigenvalue decomposition on the covariance matrix of the calibration error, and analyze the error source and its influence degree on the calibration result;

[0067] S300: If it is found that some error sources can be recalibrated by adding calibration points or changing calibration conditions; use a non-linear optimization algorithm to optimize the calibration results; collect new images for validating the calibration results, calculate the reprojection error and statistical average error for the new images.

[0068] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, a Kinect V2 camera is used to collect multiple sets of image data, and preprocessing operations such as denoising and grayscale conversion are performed on the collected images; a calibration board is selected, the corner coordinates of the calibration board are extracted from each image, and a tool is used for camera calibration to calculate the internal parameters and external parameters of the camera; secondly, the calibration results are applied to new images, the distance between the reprojection points and the actual observation points is calculated, the calibration results are evaluated, and statistical analysis is performed on the reprojection errors of all images to calculate statistical indicators such as the average error and the maximum error; by adjusting input parameters such as the position of the calibration board and the camera attitude, observing the changes in the calibration results, determining the influence degree of each input parameter on the calibration results, performing eigenvalue decomposition on the covariance matrix of the calibration errors, analyzing the error sources and their influence degrees on the calibration results; finally, if it is found that some error sources, by increasing the calibration points or changing the calibration conditions, re-perform calibration; use a nonlinear optimization algorithm to optimize the calibration results; collect new images for verifying the calibration results, and calculate the reprojection error and average error statistics for the new images. The steps of the above solution, step S100 data collection and calibration, obtain high-quality image data to ensure the accuracy of the calibration process; denoising and grayscale conversion processing improve the clarity and contrast of the images, facilitating corner extraction; accurately extract the corner coordinates of the calibration board to provide accurate data for calibration; calculate the internal and external parameters of the camera through the tool to provide basic data for error analysis and compensation. Significance: Ensure the accuracy and reliability of the calibration data, providing a solid foundation for error analysis; obtain the accurate internal and external parameters of the camera, providing an accurate camera model for image processing and 3D reconstruction. Step S200 error evaluation and decomposition, evaluate the accuracy of the calibration results through the reprojection error, quantifying the error magnitude; calculate statistical indicators such as the average error and the maximum error to comprehensively understand the distribution of the errors; by adjusting the input parameters, observing the changes in the calibration results, determining the influence degree of each parameter on the calibration results; through eigenvalue decomposition, analyze the main error sources and their influence degrees on the calibration results. Significance: Quantify the calibration errors through the reprojection error and statistical analysis, providing a basis for error compensation; identify the main error sources through sensitivity analysis and principal component analysis, providing a direction for targeted compensation; providing a theoretical basis and data support for error compensation and optimization. Step S300 error compensation and verification, by increasing the calibration points or changing the calibration conditions, re-perform calibration to reduce the influence of fixed error sources; use a nonlinear optimization algorithm to optimize the calibration results to further reduce the errors; collect new image data for verifying the accuracy of the calibration results; calculate the reprojection error and average error statistics for the new images to ensure the reliability of the calibration results. Significance: Reduce the calibration errors through re-calibration and nonlinear optimization, improving the accuracy of the calibration results; verify the calibration results through new image data to ensure the reliability and stability of the calibration results; provide empirical data and improvement directions for the calibration work, continuously improving the calibration accuracy.

[0069] In summary, this embodiment can systematically analyze and compensate for the calibration errors of the Kinect V2 camera, improving the accuracy and reliability of the calibration results. Each step cooperates with each other to ensure that the entire process from data acquisition to error compensation and then to result verification is effectively controlled, ultimately achieving the goal of improving the camera calibration accuracy.

[0070] Embodiment 2: As Figure 2 shown, on the basis of Embodiment 1, the process of calculating the internal and external parameters of the camera provided by the embodiment of the present invention includes the following steps:

[0071] S101: Obtain the checkerboard calibration board for camera positioning and recognition, and use the Kinect V2 camera to capture multiple groups of images from different angles and distances; perform image calibration through the camera calibration toolbox to extract the corner points on the checkerboard calibration board;

[0072] S102: Perform preprocessing such as undistortion, segmentation, and edge detection on the images, extract the positions of the calibration points on the checkerboard calibration board, and calculate the internal and external parameters of the camera according to the positions; among them, the internal parameters include focal length, principal point coordinates, and camera distortion coefficients, etc., which are used to describe the properties of the camera itself; the external parameters include the position and attitude of the camera, which are used to describe the position and orientation of the camera in the world coordinate system;

[0073] S103: After completing the camera calibration, it is necessary to verify the calibration results, use other measurement tools or cameras for comparison, and check the accuracy and stability of the calibration results; if it is found that the calibration results have errors or deviations exceed the preset threshold, readjust the positions of the calibration board and the camera, and re-perform the calibration process; after the calibration is completed, apply the calibration results to specific measurement or calculation tasks.

[0074] Among them, the focal length calculation uses a deep neural network to optimize the focal length, and a neural network model is defined where θ is the parameter of the deep neural network;

[0075]

[0076] In the formula, W 1 , W 2 is the weight matrix, b 1 , b 2 is the bias vector;

[0077] Define the loss function L(f x , f y ):

[0078]

[0079] By minimizing L(fx , f y ), the focal length f can be obtained x and f y ;

[0080] The principal point coordinates are calculated using a neural network to optimize the principal point coordinates c x and c y , and a neural network model is defined

[0081]

[0082] Define the loss function L(c x , c y ):

[0083]

[0084] By minimizing L(c x , c y ), the principal point coordinates c x and c y ;

[0085] The distortion coefficients are calculated using a neural network to optimize the distortion coefficients (k 1 , k 2 , k 3 , p 1 , p 2 ), and a neural network model is defined

[0086]

[0087]

[0088] Define the loss function L(k 1 , k 2 , k 3 , p 1 , p 2 ):

[0089]

[0090] By minimizing L(k 1 , k 2 , k 3 , p 1 , p 2 ), the distortion coefficients (k 1 , k 2 , k 3 , p 1 , p 2 ) can be obtained;

[0091] Optimize the rotation matrix R using a neural network and define the neural network model

[0092]

[0093] Where: vec(R) is the vectorized representation of the rotation matrix R;

[0094] Define the loss function L(R):

[0095]

[0096] By minimizing L(R), the rotation matrix R can be obtained;

[0097] Optimize the translation vector using a neural network and define the neural network model

[0098]

[0099] Define the loss function L(t):

[0100]

[0101] By minimizing L(t), the translation vector t can be obtained;

[0102] Combining the internal and external parameters, the projection matrix P of the camera is expressed as:

[0103] P = K[R|t]

[0104] Where K is the internal parameter matrix and R|t is the external parameter matrix;

[0105] f x and f y represent the focal lengths of the camera in the x and y directions, c x and c y represent the coordinates of the image center (principal point), d xi ,d yi represents the pixel distance of the i-th checkerboard in the image, p xi ,p yi represents the physical distance of the i-th checkerboard in the real world, k 1 ,k 2 ,k 3 represent the radial distortion coefficients, p 1 ,p 2 represent the tangential distortion coefficients, r i represents the radial distance of the i-th checkerboard, R represents the rotation matrix, t represents the translation vector, p i represents the position of the i-th checkerboard in the world coordinate system, q iRepresents the position of the i-th checkerboard grid in the camera coordinate system, and p represents the average position of the checkerboard grid in the world coordinate system. Represents the average position of the checkerboard grid in the camera coordinate system. Represents the neural network model, θ represents the parameters of the neural network, W 1 , W 2 Represents the weight matrix, b 1 , b 2 Represents the bias vector. Through the above steps and formulas, the internal parameters and external parameters of the camera can be accurately calculated, thereby realizing the calibration of the camera. Combining the advantages of deep learning and neural networks, it can more precisely optimize the camera parameters.

[0106] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, a checkerboard calibration board for camera positioning and recognition is first obtained, and a Kinect V2 camera is used to capture multiple groups of images from different angles and distances; the images are calibrated through a camera calibration toolbox, and the corner points on the checkerboard calibration board are extracted; secondly, preprocessing such as undistortion, segmentation, and edge detection is performed on the images to extract the positions of the calibration points on the checkerboard calibration board, and the internal and external parameters of the camera are calculated based on the positions; among them, the internal parameters include the focal length, the coordinates of the principal point, and the camera distortion coefficient, etc., which are used to describe the properties of the camera itself; the external parameters include the position and orientation of the camera, which are used to describe the position and orientation of the camera in the world coordinate system; finally, after the camera calibration is completed, the calibration results need to be verified, and other measurement tools or cameras are used for comparison to check the accuracy and stability of the calibration results; if it is found that the calibration results have errors or deviations exceeding the preset threshold, the positions of the calibration board and the camera are readjusted, and the calibration process is performed again; after the calibration is completed, the calibration results are applied to specific measurement or calculation tasks. In step S101 of the above solution, data acquisition and corner point extraction, multiple groups of images are captured from different angles and distances to ensure the diversity and comprehensiveness of the calibration data; the corner points on the checkerboard calibration board are extracted through the camera calibration toolbox to provide accurate data for calibration. Significance: By capturing images from multiple angles and distances, wide coverage of the calibration data is ensured, and the accuracy and robustness of the calibration are improved; the corner points of the checkerboard are accurately extracted to provide a high-quality data basis for subsequent calibration parameter calculation. In step S102, image preprocessing and parameter calculation, preprocessing operations such as undistortion, segmentation, and edge detection improve the quality of the images and the accuracy of corner point extraction; the internal parameters such as the focal length, the coordinates of the principal point, and the camera distortion coefficient are calculated to describe the properties of the camera itself; the external parameters such as the position and orientation of the camera are calculated to describe the position and orientation of the camera in the world coordinate system. Significance: Through preprocessing operations, the clarity and contrast of the images are improved, and the accuracy of corner point extraction is ensured; by calculating the internal and external parameters, the characteristics, position, and orientation of the camera are comprehensively described, providing an accurate camera model for image processing and 3D reconstruction. In step S103, calibration result verification and application, other measurement tools or cameras are used for comparison to check the accuracy and stability of the calibration results; if it is found that the calibration results have errors or deviations exceeding the preset threshold, the positions of the calibration board and the camera are readjusted, and the calibration process is performed again; the calibration results are applied to specific measurement or calculation tasks to ensure the accuracy and reliability of the tasks. Significance: By verifying the calibration results, the accuracy and stability of the calibration parameters are ensured, and error accumulation is avoided; by readjusting and recalibrating, the calibration error is reduced, and the accuracy of the calibration results is improved; applying the calibration results to specific measurement or calculation tasks ensures the accuracy and reliability of the tasks and improves the performance of the overall system.

[0107] In summary, this embodiment can systematically calculate the internal and external parameters of the camera and ensure the accuracy and stability of the calibration results. Each step cooperates with each other to ensure that the entire process from data acquisition to parameter calculation and then to result verification is effectively controlled, ultimately achieving the purpose of improving the camera calibration accuracy and application effect.

[0108] Embodiment 3: As Figure 3 shown, based on Embodiment 1, the process of analyzing the error sources and their influence degrees on the calibration results provided by the embodiment of the present invention includes the following steps:

[0109] S201: Use the calibrated internal and external parameters of the camera to project the points in the three-dimensional world coordinate system onto the two-dimensional image plane to obtain reprojection points; calculate the Euclidean distance between the reprojection points and the actual observation points, that is, the reprojection error; calculate the average value of the reprojection errors of all images, and calculate the maximum value of the reprojection errors in all images; analyze the distribution of the reprojection errors to identify the main sources of errors;

[0110] S202: Adjust the input parameters such as the position of the calibration board and the camera pose, observe the changes in the calibration results, record the reprojection errors after each adjustment, and analyze the influence of different input parameters on the errors; calculate the covariance matrix of the calibration errors and perform eigenvalue decomposition to analyze the error sources and their influence degrees on the calibration results. The eigenvalues and eigenvectors of the covariance matrix identify the main sources of errors;

[0111] S203: Draw a histogram of the reprojection errors to observe the distribution of the errors. The histogram shows the frequency distribution of the errors; identify the main sources of errors through the error distribution diagram; according to the results of the error distribution analysis, correct the calibration parameters and perform calibration again.

[0112] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, the internal and external parameters of the camera obtained by calibration are first used to project the points in the three-dimensional world coordinate system onto the two-dimensional image plane to obtain reprojection points; the Euclidean distance between the reprojection points and the actual observation points is calculated, that is, the reprojection error; the average value of the reprojection errors of all images is calculated, and the maximum value of the reprojection errors in all images is calculated; the distribution of the reprojection errors is analyzed to identify the main sources of errors; secondly, the input parameters such as the position of the calibration board and the camera pose are adjusted, the changes in the calibration results are observed, the reprojection errors after each adjustment are recorded, and the influence of different input parameters on the errors is analyzed; the covariance matrix of the calibration errors is calculated and eigenvalue decomposition is performed to analyze the error sources and their influence degrees on the calibration results, and the eigenvalues and eigenvectors of the covariance matrix identify the main sources of errors; finally, a histogram of the reprojection errors is drawn to observe the distribution of the errors, and the histogram shows the frequency distribution of the errors; through the error distribution diagram, the main sources of errors are identified; according to the results of the error distribution analysis, the calibration parameters are corrected and calibration is performed again. In the reprojection error analysis of step S201 of the above solution, by using the internal and external parameters of the camera to project the points in the three-dimensional world coordinate system onto the two-dimensional image plane to obtain reprojection points, the correctness of the calibration parameters is verified; the Euclidean distance between the reprojection points and the actual observation points is calculated, that is, the reprojection error, which directly reflects the accuracy of the calibration result; the average value and the maximum value of the reprojection errors of all images are calculated to help understand the overall distribution of the errors and the maximum error value; by analyzing the distribution of the reprojection errors, the main sources of errors are identified, such as lens distortion, calibration board position error, etc. The achieved significance: Through reprojection error analysis, the accuracy of the calibration result can be evaluated, and it can be judged whether the calibration meets the expected requirements; the main sources of errors are identified, providing a basis for subsequent error correction. In step S202, input parameter adjustment and error analysis, the input parameters such as the position of the calibration board and the camera pose are adjusted, the changes in the calibration results are observed, and the reprojection errors after each adjustment are recorded, which helps to understand the influence of different parameters on the calibration results; the covariance matrix of the calibration errors is calculated and eigenvalue decomposition is performed to analyze the error sources and their influence degrees on the calibration results, and the eigenvalues and eigenvectors of the covariance matrix can help us identify the main sources of errors. The achieved significance: By adjusting the input parameters, it can be understood which parameters have the greatest influence on the calibration result, so that more attention can be paid to the precise control of these parameters in practical applications; through covariance matrix analysis, the influence degrees of different error sources on the calibration result can be quantified, providing a scientific basis for error correction.Step S203: Error distribution graph and parameter correction. Plot a histogram of the reprojection error to observe the distribution of the errors. The histogram shows the frequency distribution of the errors, which helps to intuitively understand the distribution characteristics of the errors. Through the error distribution graph, further identify the main sources of the errors. According to the results of the error distribution analysis, correct the calibration parameters and perform calibration again, which helps to improve the accuracy of the calibration results. Significance achieved: Through the histogram, the distribution of the errors can be intuitively analyzed, and the main sources of the errors can be identified. According to the results of the error distribution analysis, correcting the calibration parameters can significantly improve the accuracy of the calibration results and ensure the reliability of the calibration results in practical applications.

[0113] In summary, this embodiment can not only evaluate the accuracy of the calibration results, but also identify the main sources of the errors, and improve the accuracy of the calibration results through parameter adjustment and correction. It has important technical effects and practical significance in the camera calibration process, ensuring the reliability and accuracy of the calibration results, and providing a solid foundation for image processing and computer vision applications.

[0114] Embodiment 4: As Figure 4 shown, on the basis of Embodiment 3, the process of analyzing the distribution of the reprojection error provided by the embodiment of the present invention includes the following steps:

[0115] S2011: Perform statistical analysis on the reprojection error data, and calculate basic statistics such as the mean, median, and standard deviation of the errors. The basic statistics can reflect the overall distribution characteristics of the errors. For example, the mean reflects the central position of the errors, and the standard deviation reflects the degree of dispersion of the errors.

[0116] S2012: By observing the skewness and kurtosis of the error distribution, judge whether the error distribution is symmetric and whether there are characteristics of sharp peaks or flatness. A skewness of zero indicates a symmetric distribution, a positive skewness indicates a right-skewed distribution, a negative skewness indicates a left-skewed distribution, a kurtosis greater than 3 indicates a distribution with a sharp peak characteristic, and a kurtosis less than 3 indicates a flat distribution.

[0117] S2013: Extract the local characteristics of the error distribution. By calculating the local extreme points (such as local maximum and minimum values) of the error distribution, identify the peak regions and valley regions in the error distribution. Classify and quantify the error sources according to the statistical characteristics and morphological characteristics of the error distribution.

[0118] Among them, radial distortion: It is manifested as the peak at the edge of the error distribution. By identifying the high-frequency errors in the edge region of the error distribution, the influence of radial distortion can be quantified.

[0119] Tangential distortion: It is manifested as the peak in a specific direction of the error distribution. By identifying the high-frequency errors in a specific direction of the error distribution, the influence of tangential distortion can be quantified.

[0120] Flatness error of the calibration plate: It is manifested as the high discreteness of the error distribution. By calculating the standard deviation of the error distribution, the influence of the flatness error of the calibration plate can be quantified;

[0121] Grid accuracy error of the calibration plate: It is manifested as the high concentration trend of the error distribution. By calculating the mean and median of the error distribution, the influence of the grid accuracy error of the calibration plate can be quantified;

[0122] Camera rotation error: It is manifested as the peak value in a specific direction of the error distribution. By identifying the high-frequency errors in a specific direction of the error distribution, the influence of the camera rotation error can be quantified;

[0123] Camera translation error: It is manifested as the high discreteness of the error distribution. By calculating the standard deviation of the error distribution, the influence of the camera translation error can be quantified.

[0124] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, statistical analysis is performed on the reprojection error data, and basic statistics such as the mean, median, and standard deviation of the error are calculated. The basic statistics can reflect the overall distribution characteristics of the error. For example, the mean reflects the central position of the error, and the standard deviation reflects the degree of dispersion of the error. Secondly, by observing the skewness and kurtosis of the error distribution, it is judged whether the error distribution is symmetric and whether there are characteristics of sharp peaks or flatness. A skewness of zero indicates a symmetric distribution, a positive skewness indicates a right-skewed distribution, and a negative skewness indicates a left-skewed distribution. A kurtosis greater than 3 indicates a sharp peak characteristic, and less than 3 indicates a flat distribution. Finally, local characteristics of the error distribution are extracted. By calculating local extreme points of the error distribution (such as local maximum and minimum values), peak regions and valley regions in the error distribution are identified. According to the statistical characteristics and morphological characteristics of the error distribution, error sources are classified and quantified. Among them, radial distortion: manifested as edge peaks in the error distribution. By identifying high-frequency errors in the edge region of the error distribution, the influence of radial distortion can be quantified. Tangential distortion: manifested as peaks in a specific direction of the error distribution. By identifying high-frequency errors in a specific direction of the error distribution, the influence of tangential distortion can be quantified. Calibration plate flatness error: manifested as high dispersion in the error distribution. By calculating the standard deviation of the error distribution, the influence of the calibration plate flatness error can be quantified. Calibration plate grid accuracy error: manifested as a high concentration trend in the error distribution. By calculating the mean and median of the error distribution, the influence of the calibration plate grid accuracy error can be quantified. Camera rotation error: manifested as peaks in a specific direction of the error distribution. By identifying high-frequency errors in a specific direction of the error distribution, the influence of the camera rotation error can be quantified. Camera translation error: manifested as high dispersion in the error distribution. By calculating the standard deviation of the error distribution, the influence of the camera translation error can be quantified. Step S2011 of the above solution performs statistical analysis on the reprojection error data, reflects the central position of the error, and helps to understand the overall level of the error; reflects the median value of the error, is not affected by extreme values, and can more robustly describe the central position of the error; reflects the degree of dispersion of the error and helps to understand the fluctuation range of the error. Significance: Through these basic statistics, the overall distribution characteristics of the reprojection error can be initially understood, providing basic data for in-depth analysis. Step S2012 judges whether the error distribution is symmetric by observing the skewness and kurtosis of the error distribution. A positive skewness indicates a right skew, and a negative skewness indicates a left skew; judges the sharp peak or flat characteristic of the error distribution. A kurtosis greater than 3 indicates a sharp peak, and less than 3 indicates a flatness. Significance: The analysis of skewness and kurtosis helps to further understand the shape characteristics of the error distribution, identify whether there are outliers or errors in a specific pattern, and provide a basis for the classification of error sources.Step S2013 extracts the local features of the error distribution, identifies the peak regions and valley regions in the error distribution to help locate the high-incidence regions of errors; quantifies the impact by identifying high-frequency errors in the edge regions; quantifies the impact by identifying high-frequency errors in a specific direction; quantifies the impact of its high discreteness by calculating the standard deviation; quantifies the impact of its high central tendency by calculating the mean and median; quantifies the impact by identifying high-frequency errors in a specific direction; quantifies the impact of its high discreteness by calculating the standard deviation. Significance: By extracting local features, the impacts of different error sources can be more accurately identified and quantified, providing specific guiding directions for calibration and optimization.

[0125] In summary, through the detailed analysis of the reprojection error distribution in this embodiment, various error sources can be more accurately identified and quantified, thereby improving the overall accuracy of the system; understanding the distribution characteristics and sources of errors helps to design more effective calibration algorithms and reduce the impact of errors; through the in-depth analysis of the error distribution, the reliability and stability of the system can be enhanced to ensure its performance in various application scenarios. It can not only comprehensively understand the reprojection error distribution but also provide strong data support for optimization and calibration, thus enhancing the overall performance of the system.

[0126] Embodiment 5: As Figure 5 shown, on the basis of Embodiment 3, the process of identifying the main sources of errors by the eigenvalues and eigenvectors of the covariance matrix provided by the embodiment of the present invention includes the following steps:

[0127] S2021: During the calibration process, record the reprojection error data after each adjustment of the calibration board position and the camera pose; based on the collected reprojection error data, calculate the covariance matrix of the error vector, and the covariance matrix reflects the correlation and variance between the dimensions of the error vector; perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and the corresponding eigenvectors, and sort the eigenvalues by size. The larger the eigenvalue, the greater the contribution of the corresponding eigenvector to the error distribution.

[0128] S2022: Identify the main error sources by analyzing the magnitudes of the eigenvalues. The error dimension corresponding to the eigenvector with a larger eigenvalue is the main error source; calculate the proportion of each eigenvalue in the sum of the total eigenvalues to quantify the influence degree of each error source on the calibration result. The larger the eigenvalue proportion, the greater the influence of the corresponding error source on the calibration result.

[0129] S2023: Obtain the error source classification through the analysis of eigenvectors and eigenvalues.

[0130] Among them, the radial distortion error is analyzed through eigenvector analysis to identify the main direction of the radial distortion error. The radial distortion error is manifested as the edge peak of the error distribution; the tangential distortion error is analyzed through eigenvector analysis to identify the main direction of the tangential distortion error, and the tangential distortion error is manifested as the peak in a specific direction of the error distribution; the flatness error of the calibration plate is analyzed through eigenvalue analysis to identify the influence of the flatness error of the calibration plate, and the flatness error of the calibration plate is manifested as the high discreteness of the error distribution; the grid accuracy error of the calibration plate is analyzed through eigenvalue analysis to identify the influence of the grid accuracy error of the calibration plate, and the grid accuracy error of the calibration plate is manifested as the high concentration trend of the error distribution; the camera rotation error is analyzed through eigenvector analysis to identify the main direction of the camera rotation error, and the camera rotation error is manifested as the peak in a specific direction of the error distribution; the camera translation error is analyzed through eigenvalue analysis to identify the influence of the camera translation error, and the camera translation error is manifested as the high discreteness of the error distribution.

[0131] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, during the calibration process, the reprojection error data after each adjustment of the calibration board position and the camera pose is recorded; based on the collected reprojection error data, the covariance matrix of the error vector is calculated, and the covariance matrix reflects the correlation and variance between the dimensions of the error vector; the covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and corresponding eigenvectors, and the eigenvalues are sorted by size. The larger the eigenvalue, the greater the contribution of the corresponding eigenvector to the error distribution; secondly, by analyzing the size of the eigenvalues, the main error sources are identified. The error dimension corresponding to the eigenvector with a larger eigenvalue is the main error source; calculate the proportion of each eigenvalue in the sum of the total eigenvalues to quantify the influence degree of each error source on the calibration result. The larger the eigenvalue proportion, the greater the influence of the corresponding error source on the calibration result; finally, through eigenvector and eigenvalue analysis, the error source classification is obtained; among them, the radial distortion error is analyzed through eigenvectors to identify the main direction of the radial distortion error. The radial distortion error is manifested as the edge peak of the error distribution; the tangential distortion error is analyzed through eigenvectors to identify the main direction of the tangential distortion error. The tangential distortion error is manifested as the peak in a specific direction of the error distribution; the flatness error of the calibration board is analyzed through eigenvalues to identify the influence of the flatness error of the calibration board. The flatness error of the calibration board is manifested as the high discreteness of the error distribution; the grid accuracy error of the calibration board is analyzed through eigenvalues to identify the influence of the grid accuracy error of the calibration board. The grid accuracy error of the calibration board is manifested as the high concentration trend of the error distribution; the camera rotation error is analyzed through eigenvectors to identify the main direction of the camera rotation error. The camera rotation error is manifested as the peak in a specific direction of the error distribution; the camera translation error is analyzed through eigenvalues to identify the influence of the camera translation error. The camera translation error is manifested as the high discreteness of the error distribution. Step S2021 of the above solution records and calculates the covariance matrix of the error vector; by recording the reprojection error data after each adjustment of the calibration board position and the camera pose and calculating the covariance matrix of the error vector, the correlation and variance between the dimensions of the error vector can be quantified; the eigenvalue decomposition of the covariance matrix can reveal the main direction and intensity of the error distribution; it provides basic data for error source identification, helps to understand the error distribution in different dimensions, and lays a foundation for accurately identifying the main error sources. Step S2022 analyzes the eigenvalues and eigenvectors to identify the main error sources. By analyzing the size of the eigenvalues, the dimension with the greatest contribution to the error distribution can be identified, that is, the main error source; the error dimension corresponding to the eigenvector with a larger eigenvalue is the main error source. By calculating the proportion of each eigenvalue in the sum of the total eigenvalues, the influence degree of each error source on the calibration result can be quantified; it helps to accurately identify and quantify the main error sources, provides a clear direction for subsequent error correction and optimization, and improves the accuracy and reliability of the calibration result.In step S2023, through eigenvector and eigenvalue analysis, the error source classification is obtained. By deeply analyzing the eigenvectors and eigenvalues, different types of error sources can be identified, such as radial distortion error, tangential distortion error, calibration plate flatness error, calibration plate grid accuracy error, camera rotation error, and camera translation error, etc. Each error source exhibits different characteristics in the error distribution, such as edge peaks, specific direction peaks, high discreteness, or high central tendency. The significance achieved: helps decompose the complex error distribution into different types of error sources, providing a basis for targeted error correction; through classification and identification, error correction and optimization can be carried out more effectively, further improving the accuracy and stability of the calibration results.

[0132] In summary, through systematic data collection, covariance matrix calculation, eigenvalue decomposition, and eigenvector analysis in this embodiment, the accurate identification and classification of the main error sources in the calibration process are realized. It not only improves the accuracy and reliability of the calibration results but also provides a scientific basis for error correction and optimization, ensuring the efficiency and accuracy of the calibration process.

[0133] Embodiment 6: As Figure 6 shown, on the basis of Embodiment 3, the process of identifying the main sources of errors through the error distribution map provided by the embodiment of the present invention includes the following steps:

[0134] S2031: Obtain the reprojection error data of all images from steps S201 and S202, group the reprojection error data according to intervals, and draw a histogram. The horizontal axis of the histogram represents the magnitude of the error, and the vertical axis represents the frequency of the error;

[0135] S2032: Observe the overall shape of the histogram, identify the characteristics of the error distribution, such as whether it presents a normal distribution, skewed distribution, or other complex distributions; determine the peak positions in the histogram, analyze the error intervals corresponding to the peaks, and identify the concentrated areas of the errors; observe the tails of the histogram, analyze the distribution of extreme error values, and identify possible abnormal error sources;

[0136] S2033: Identify the main sources of errors according to the distribution characteristics of the histogram; if the peaks are concentrated in a certain specific interval, it may indicate that the parameters or conditions corresponding to this interval are the main error sources; analyze the secondary peaks or tail distributions of the histogram to identify secondary error sources; evaluate their contributions to the overall error by calculating the frequency ratios of different error intervals; combine the results of covariance matrix analysis to confirm the correlation between error sources and identify possible error propagation paths.

[0137] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, the reprojection error data of all images are obtained from step S201 and step S202, grouped according to intervals, and a histogram is plotted. The horizontal axis of the histogram represents the magnitude of the error, and the vertical axis represents the frequency of the error. Secondly, observe the overall shape of the histogram, identify the characteristics of the error distribution, such as whether it shows a normal distribution, skewed distribution or other complex distributions. Determine the peak position in the histogram, analyze the error interval corresponding to the peak, and identify the concentrated area of the error. Observe the tail of the histogram, analyze the distribution of extreme values of the error, and identify possible abnormal error sources. Finally, according to the distribution characteristics of the histogram, identify the main sources of error. For example, if the peak is concentrated in a certain specific interval, it may indicate that the parameters or conditions corresponding to this interval are the main error sources. Analyze the secondary peaks or tail distributions of the histogram to identify secondary error sources. Evaluate their contribution to the overall error by calculating the frequency proportion of different error intervals. Combine the analysis results of the covariance matrix to confirm the correlation between error sources and identify possible error propagation paths. In step S2031 of the above solution, the reprojection error data are obtained and a histogram is plotted, and the reprojection error data are grouped according to intervals, making the data distribution more intuitive. The horizontal axis of the histogram represents the magnitude of the error, and the vertical axis represents the frequency of the error, visually showing the distribution of the error. Significance: Through the histogram, the overall distribution of the error can be quickly understood, the concentrated area and distribution characteristics of the error can be identified, providing a basis for in-depth analysis. In step S2032, observe the overall shape of the histogram to determine whether the error distribution shows a normal distribution, skewed distribution or other complex distributions, helping to understand the overall shape of the error. Analyze the error interval corresponding to the peak to identify the concentrated area of the error and understand the location of the main error sources. Observe the tail of the histogram, analyze the distribution of extreme values of the error, and identify possible abnormal error sources. Significance: By observing the overall shape of the histogram, the main sources and distribution characteristics of the error can be initially judged, providing a basis for subsequent error source identification. In step S2033, through the distribution characteristics of the histogram, the main sources of error can be accurately identified and quantified, providing specific guidance for subsequent error correction and optimization. Through the analysis of the covariance matrix, the correlation between error sources can be understood, the error propagation path can be identified, and the comprehensiveness and accuracy of error analysis can be further improved.

[0138] In summary, in this embodiment, by identifying and quantifying the main sources of errors, targeted correction and optimization can be carried out, thereby improving the overall accuracy of the system; understanding the main sources and distribution characteristics of errors helps to design more effective correction algorithms and reduce the impact of errors; through in-depth analysis of the error distribution, the reliability and stability of the system can be enhanced to ensure its performance in various application scenarios. It can not only comprehensively understand the distribution of reprojection errors, but also provide strong data support for optimization and correction, thereby improving the overall performance of the system. Through the above hierarchical analysis process, step S203 can not only intuitively identify the main sources of errors, but also quantify the influence degree of different error sources on the calibration results, providing a scientific basis for accurately correcting calibration parameters. The difference from the prior art is mainly reflected in the intuitive analysis of the histogram, combined with statistical methods, to deeply explore the distribution characteristics of errors, so as to achieve fine-tuning and optimization of the calibration results.

[0139] Embodiment 7: As Figure 7 shown, on the basis of Embodiment 6, the process of identifying the main and secondary sources of errors provided by the embodiment of the present invention includes the following steps:

[0140] S20331: Observe the peak position in the histogram to determine the interval where the errors are most concentrated. If the peak is concentrated in the interval where the error value is from 0.5 to 1.0, then it is inferred that the parameters or conditions of the interval have the greatest influence on the errors; calculate the proportion of the error frequency in the peak interval to the total error frequency to evaluate its contribution to the overall error;

[0141] S20332: In the histogram, observe the tail of the histogram, which contains extreme error values, analyze the shape and frequency of the tail distribution, and identify possible abnormal error sources;

[0142] S20333: Synthesize the analysis results of the main error sources, secondary error sources, and error propagation paths for comprehensive evaluation to confirm which parameters or conditions have the greatest influence on the overall error, which are the secondary error sources, and how the errors propagate in the system; finally confirm the main sources and secondary error sources of the errors.

[0143] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, observe the peak position in the histogram to determine the interval where the errors are most concentrated. If the peak is concentrated in the interval where the error value is from 0.5 to 1.0, it is inferred that the parameters or conditions in this interval have the greatest impact on the error. Calculate the proportion of the error frequency in the peak interval to the total error frequency to evaluate its contribution to the overall error. Secondly, in the histogram, observe the tail of the histogram, which contains extreme error values, analyze the shape and frequency of the tail distribution, and identify possible abnormal error sources. Finally, comprehensively evaluate the analysis results of the main error sources, secondary error sources, and error propagation paths to confirm which parameters or conditions have the greatest impact on the overall error, which are the secondary error sources, and how the error propagates in the system; ultimately confirm the main and secondary error sources of the error. In step S20331 of the above solution, observe the peak position in the histogram. By observing the peak position in the histogram, the interval where the errors are most concentrated can be quickly located, thereby identifying the parameters or conditions that have the greatest impact on the error; calculating the proportion of the error frequency in the peak interval to the total error frequency can quantify the contribution degree of this interval to the overall error. The achieved significance: By analyzing the peak position and the proportion of the error frequency, it can be confirmed which parameters or conditions have the greatest impact on the overall error, thereby providing a clear direction for error correction and optimization; after understanding the main error sources, resources and efforts can be more targeted, and the problems that have the greatest impact on the overall error can be solved first. In step S20332, observe the tail of the histogram. By observing the tail of the histogram, the interval containing extreme error values can be identified, and these extreme values are often the sources of abnormal errors; analyzing the shape and frequency of the tail distribution can further confirm the sources and influence ranges of the abnormal errors. The achieved significance: By analyzing the tail distribution, potential problems or abnormal situations that may exist in the system can be discovered, so as to prevent and handle them in advance; identifying and handling abnormal error sources helps to improve the stability and reliability of the system and reduce system failures caused by abnormal errors. In step S20333, comprehensive evaluation. By comprehensively evaluating the analysis results of the main error sources, secondary error sources, and error propagation paths, the sources and propagation methods of the error can be comprehensively understood; through comprehensive evaluation, it can be confirmed which parameters or conditions have the greatest impact on the overall error, which are the secondary error sources, and how the error propagates in the system. The achieved significance: Through comprehensive evaluation, the system can be comprehensively optimized, not only solving the main error sources but also taking into account the secondary error sources, thereby improving the overall performance of the system; the comprehensive evaluation results provide comprehensive data support for decision-making, which helps to formulate more scientific and effective error control strategies.

[0144] In summary, this embodiment can systematically identify the main and secondary error sources of the error and conduct a comprehensive evaluation. It not only helps to optimize the system performance, improve the decision-making quality, but also enhances the stability and reliability of the system, thereby providing better services for users.

[0145] Example 8: As Figure 8 shown, based on Example 7, the process of finally confirming the main error sources and secondary error sources provided by the embodiments of the present invention includes the following steps:

[0146] S203331: Collect all error data, including error data in the peak interval of the histogram, error data in the tail distribution, and data on the error propagation path; organize the collected data and classify it into main error source data, secondary error source data, and error propagation path data;

[0147] S203332: Calculate the proportion of the error frequency in the peak interval to the total error frequency; calculate the proportion of the tail error frequency to the total error frequency;

[0148] S203333: Analyze how errors propagate in the system, identify the main and secondary paths of error propagation; evaluate the degree of influence of the error propagation path on the final result.

[0149] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, all error data is collected, including error data in the peak interval of the histogram, error data in the tail distribution, and data on the error propagation path; the collected data is organized and classified into main error source data, secondary error source data, and error propagation path data; second, calculate the proportion of the error frequency in the peak interval to the total error frequency; calculate the proportion of the tail error frequency to the total error frequency; finally, analyze how errors propagate in the system, identify the main and secondary paths of error propagation; evaluate the degree of influence of the error propagation path on the final result. In step S203331 of the above solution, data collection and organization ensure the comprehensiveness and accuracy of the analysis by collecting all relevant error data; classifying the data into main error source data, secondary error source data, and error propagation path data facilitates subsequent analysis and processing. The significance achieved is to provide solid basic data support for error frequency calculation and propagation path analysis; through classification and organization, it is clear which data are the main error sources, which are the secondary error sources, and how errors propagate in the system, providing a clear basis for weight assignment and error source confirmation. Step S203332 provides a scientific basis for assigning weights to each error source, ensuring the rationality and accuracy of weight assignment; through the calculation of error frequencies, it is clear which error sources need to be processed first and which can be processed secondarily, improving the efficiency and pertinence of error control. In step S203333, error propagation path analysis can optimize the system design targetedly by identifying the main propagation paths, reducing the impact of error propagation on the final result; by evaluating the degree of influence of the propagation path, more effective error control strategies can be formulated to ensure the stability and reliability of the system; it provides an important basis for comprehensively evaluating the influence of error sources, ensuring the comprehensiveness and accuracy of the evaluation results.

[0150] In summary, this embodiment can comprehensively and systematically analyze the main error sources and minor error sources, identify the error propagation paths, and quantify the influence degree of the errors, providing a scientific basis for error control and optimization. Specifically, ensure that the basic data for analysis is comprehensive and accurate, clarify the data sources and classifications; quantify the error influence, identify the main and minor error sources, and provide a basis for weight assignment; identify the main and minor propagation paths, evaluate the influence of the propagation paths, and optimize the system design and error control strategies. By implementing these steps, the efficiency and pertinence of error control can be improved, the stability and reliability of the system can be ensured, and a scientific basis for system optimization and improvement can be provided.

[0151] Embodiment 9: As Figure 9 shown, based on Embodiment 1, the process of re - calibration provided by the embodiment of the present invention includes the following steps:

[0152] S301: Through statistical analysis of the calibration results, identify the main error sources affecting the calibration accuracy; conduct a detailed analysis of the identified error sources to determine their specific influence modes and degrees on the calibration results; for example, camera distortion mainly affects the shape of straight lines in the image, and the calibration board position error mainly affects the accuracy of corner point extraction;

[0153] S302: For the identified main error sources, increase the number of rows and columns of the calibration board; place the calibration board at different angles and distances;

[0154] S303: At different angles and distances, re - collect multiple groups of image data; use tools to re - extract the corner point coordinates of the calibration board in each image, re - perform camera calibration, and calculate the internal and external parameters of the camera.

[0155] The working principle and beneficial effects of the above technical solution are as follows: In this embodiment, first, through the statistical analysis of the calibration results, the main error sources affecting the calibration accuracy are identified; the identified error sources are analyzed in detail to determine their specific influence modes and degrees on the calibration results; for example, camera distortion mainly affects the shape of straight lines in the image, and the calibration board position error mainly affects the accuracy of corner point extraction. Secondly, for the identified main error sources, the number of rows and columns of the calibration board is increased; the calibration board is placed at different angles and distances. Finally, at different angles and distances, multiple groups of image data are re-acquired; tools are used to re-extract the corner point coordinates of the calibration board in each image, and camera calibration is performed again to calculate the internal and external parameters of the camera. In step S301 of the above solution, error source identification and analysis, through the statistical analysis of the calibration results, the main error sources affecting the calibration accuracy are identified, such as camera distortion, calibration board position error, etc.; the identified error sources are analyzed in detail to determine their specific influence modes and degrees on the calibration results. For example, camera distortion mainly affects the shape of straight lines in the image, and the calibration board position error mainly affects the accuracy of corner point extraction. The significance achieved is: By identifying and analyzing the main error sources, the specific reasons for the low calibration accuracy are clarified, providing a direction for improvement measures; understanding the specific influence modes and degrees of the error sources can help design targeted improvement schemes to improve the calibration accuracy and reliability. In step S302, increasing the complexity and diversity of the calibration board, by increasing the number of rows and columns of the calibration board, the complexity of the calibration board is increased, and the number of data points in the calibration process is increased, thereby improving the calibration accuracy and stability; the calibration board is placed at different angles and distances to simulate the diversity in the actual use environment and increase the richness and representativeness of the calibration data. The significance achieved is: By increasing the complexity and diversity of the calibration board, the influence of different error sources can be more comprehensively covered, improving the calibration accuracy and reliability; the diverse calibration environment can enhance the robustness of the calibration results, enabling them to maintain a high accuracy under different use environments. In step S303, re-acquisition and calibration, at different angles and distances, multiple groups of image data are re-acquired to ensure the comprehensiveness and diversity of the data; tools are used to re-extract the corner point coordinates of the calibration board in each image to ensure the accuracy and consistency of corner point extraction; camera calibration is performed again to calculate the internal and external parameters of the camera to ensure the accuracy and reliability of the calibration results. The significance achieved is: By re-acquiring multiple groups of image data, the comprehensiveness and diversity of the calibration data are ensured, improving the calibration accuracy and reliability; by re-extracting the corner point coordinates and re-calibrating, the errors in the previous calibration can be eliminated, improving the calibration accuracy and stability; re-calculating the internal and external parameters of the camera ensures the accuracy and applicability of these parameters, improving the overall performance of the camera.

[0156] In summary, the present embodiment can comprehensively and systematically re - calibrate the camera, improving the accuracy and reliability of calibration. Specifically, it clarifies the specific reasons for the low calibration accuracy, providing a direction for improvement measures; improves the calibration accuracy and stability, enhancing the robustness of the calibration results; ensures the comprehensiveness and diversity of calibration data, improves the calibration accuracy and stability, and optimizes the internal and external parameters of the camera. By implementing these steps, the accuracy and reliability of camera calibration can be significantly improved, ensuring the performance and stability of the camera in actual use and providing a solid foundation for applications and development.

[0157] Embodiment 10: As Figure 10 shown, based on Embodiments 1 - 9, the camera calibration error analysis system provided by the embodiment of the present invention based on Kinect V2 includes:

[0158] A parameter calculation module, responsible for using the Kinect V2 camera to collect multiple groups of image data, performing pre - processing operations such as denoising and grayscale conversion on the collected images; selecting a calibration board, extracting the corner coordinates of the calibration board in each image, using tools for camera calibration, and calculating the internal and external parameters of the camera;

[0159] An error analysis module, responsible for applying the calibration results to new images, calculating the distance between the reprojection points and the actual observation points, evaluating the calibration results, statistically analyzing the reprojection errors of all images, and calculating statistical indicators such as the average error and the maximum error; by adjusting input parameters such as the calibration board position and the camera pose, observing the changes in the calibration results, determining the influence degree of each input parameter on the calibration results, performing eigenvalue decomposition on the covariance matrix of the calibration error, and analyzing the error sources and their influence degrees on the calibration results;

[0160] A calibration optimization module, responsible for re - calibrating if it is found that some error sources can be eliminated by increasing calibration points or changing calibration conditions; optimizing the calibration results using a non - linear optimization algorithm; collecting new images for verifying the calibration results, calculating the reprojection error and the average error statistics for the new images.

[0161] The working principle and beneficial effects of the above technical solution are as follows: The parameter calculation module in this embodiment uses a KinectV2 camera to collect multiple sets of image data, and performs preprocessing operations such as denoising and grayscale conversion on the collected images; selects a calibration board, extracts the corner coordinates of the calibration board in each image, uses tools for camera calibration, and calculates the internal parameters and external parameters of the camera; the error analysis module applies the calibration result to a new image, calculates the distance between the reprojection point and the actual observation point, evaluates the calibration result, statistically analyzes the reprojection errors of all images, and calculates statistical indicators such as the average error and the maximum error; by adjusting input parameters such as the position of the calibration board and the camera attitude, observes the change of the calibration result, determines the influence degree of each input parameter on the calibration result, performs eigenvalue decomposition on the covariance matrix of the calibration error, and analyzes the error sources and their influence degrees on the calibration result; if the calibration optimization module finds that some error sources, through adding calibration points or changing calibration conditions, re-performs calibration; uses a nonlinear optimization algorithm to optimize the calibration result; collects new images to verify the calibration result, calculates the reprojection error and statistically analyzes the average error of the new images. The parameter calculation module of the above solution uses a Kinect V2 camera to collect multiple sets of image data, and performs preprocessing operations such as denoising and grayscale conversion on the images to ensure the image quality and provide high-quality input data for calibration; extracts the corner coordinates of the calibration board in each image, accurately captures the characteristics of the calibration board, and provides accurate geometric information for camera calibration; uses tools for camera calibration, calculates the internal parameters of the camera (such as focal length, principal point position) and external parameters (such as rotation matrix, translation vector), and provides basic data for subsequent error analysis and optimization. The achieved significance: Through preprocessing and corner extraction, ensure the high quality of calibration data and provide a reliable basis for error analysis and optimization; through accurate calculation of camera parameters, provide accurate visual measurement capabilities for the system and improve the overall accuracy of the system. The error analysis module applies the calibration result to a new image, calculates the distance between the reprojection point and the actual observation point, and evaluates the accuracy of the calibration result; statistically analyzes the reprojection errors of all images, calculates statistical indicators such as the average error and the maximum error, and quantifies the error level of the calibration result; by adjusting input parameters such as the position of the calibration board and the camera attitude, observes the change of the calibration result, determines the influence degree of each input parameter on the calibration result; performs eigenvalue decomposition on the covariance matrix of the calibration error, analyzes the error sources and their influence degrees on the calibration result, and provides a quantitative analysis of the error sources. The achieved significance: Through reprojection error calculation and statistical analysis, quantify the error level of the calibration result and provide data support for calibration optimization; through parameter adjustment and covariance matrix analysis, identify the key parameters affecting the calibration result and provide a direction for optimization; through eigenvalue decomposition, deeply analyze the error sources and their influence degrees and provide a theoretical basis for error control.If the calibration optimization module detects certain error sources, it can re - calibrate by increasing the calibration points or changing the calibration conditions to optimize the calibration results. It uses a non - linear optimization algorithm to further improve the calibration accuracy. New images are collected to verify the calibration results, and reprojection error calculation and average error statistics are performed on the new images to ensure the reliability of the calibration results. Significance achieved: Through error source optimization and non - linear optimization, the calibration accuracy is further improved, enhancing the overall performance of the system. Through the verification and statistical analysis of new images, the reliability of the calibration results is ensured, providing a guarantee for practical applications. Through the optimization module, the system has the ability to continuously optimize and can improve the calibration results according to actual needs.

[0162] In summary, through the organic combination of three modules: parameter calculation, error analysis, and calibration optimization, this embodiment forms a systematic calibration process to ensure the comprehensiveness and accuracy of the calibration process. Through multi - level analysis and optimization, the high - precision calibration results are ensured, enhancing the visual measurement ability of the system. Through error analysis and optimization, error sources are effectively controlled, improving the stability and reliability of the system. Significance achieved: Through high - precision calibration and error control, the overall performance of the system is significantly improved, meeting the requirements of high - precision visual measurement. Through the systematic calibration process and optimization methods, theory and practice are closely combined, providing reliable technical support for practical applications. The system has the ability to continuously improve and can optimize the calibration results according to actual needs to adapt to the requirements of different application scenarios. The camera calibration error analysis system based on Kinect V2 in this embodiment can not only provide high - precision calibration results, but also effectively control error sources, enhancing the overall performance and reliability of the system (which is a general reference, including but not limited to fields such as human - computer interaction, robot navigation, virtual reality, and automotive intelligent vision), providing strong technical support for practical applications.

[0163] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of equivalent technologies of the present invention, the present invention also intends to include these modifications and variations.

Claims

1. A camera calibration error analysis method based on Kinect V2, characterized in that: The following steps are involved: Use Kinect V2 camera to collect multiple sets of image data, perform denoising and grayscale preprocessing on the collected images; select a calibration plate, extract the coordinates of the corner points of the calibration plate in each image, use tools to calibrate the camera, and calculate the intrinsic and extrinsic parameters of the camera; Apply the calibration results to the new image, calculate the distance between the reprojection point and the actual observation point, evaluate the calibration results, perform statistical analysis on the reprojection errors of all images, and calculate the average error and maximum error statistical indicators; adjust the calibration plate position and camera attitude input parameters, observe the changes in the calibration results, determine the influence of each input parameter on the calibration results, perform eigenvalue decomposition on the covariance matrix of the calibration error, and analyze the error sources and their influence on the calibration results; If some error sources are found, recalibrate by adding calibration points or changing calibration conditions; use nonlinear optimization algorithm to optimize the calibration results; collect new images to verify the calibration results, and perform reprojection error calculation and average error statistics on the new images.

2. The camera calibration error analysis method based on Kinect V2 as claimed in claim 1, characterized in that: The process of calculating the intrinsic and extrinsic parameters of the camera includes the following steps: Obtain a checkerboard calibration plate for camera positioning and identification, and use a Kinect V2 camera to take multiple sets of images from different angles and distances; calibrate the images using the camera calibration toolbox to extract the corner points on the checkerboard calibration plate; The image is preprocessed by dedistortion, segmentation and edge detection, the position of the calibration points on the chessboard calibration board is extracted, and the internal and external parameters of the camera are calculated according to the position; the internal parameters include focal length, principal point coordinates and camera distortion coefficient, which are used to describe the properties of the camera itself; the external parameters include the position and posture of the camera, which are used to describe the position and orientation of the camera in the world coordinate system; After completing the camera calibration, the calibration results need to be verified and compared with the camera to check the accuracy and stability of the calibration results. If the calibration results are found to have errors or the deviation exceeds the preset threshold, readjust the position of the calibration plate and the camera and repeat the calibration process. After the calibration is completed, apply the calibration results to specific measurement or calculation tasks.

3. The camera calibration error analysis method based on Kinect V2 as claimed in claim 1, characterized in that: The process of analyzing the sources of error and their impact on the calibration results includes the following steps: Using the camera intrinsic and extrinsic parameters obtained through calibration, the points in the 3D world coordinate system are projected onto the 2D image plane to obtain the reprojected points; the Euclidean distance between the reprojected points and the actual observed points, i.e., the reprojection error, is calculated; the average reprojection error of all images is calculated, and the maximum reprojection error of all images is calculated; the distribution of the reprojection error is analyzed, and the main sources of the error are identified; Adjust the position of the calibration plate and the input parameters of the camera attitude, observe the changes in the calibration results, record the reprojection error after each adjustment, and analyze the impact of different input parameters on the error; calculate the covariance matrix of the calibration error and perform eigenvalue decomposition to analyze the error sources and their impact on the calibration results. The eigenvalues ​​and eigenvectors of the covariance matrix identify the main sources of the error; Draw a histogram of the reprojection error and observe the distribution of the error. The histogram shows the frequency distribution of the error. Through the error distribution diagram, identify the main source of the error. According to the results of the error distribution analysis, correct the calibration parameters and recalibrate.

4. The camera calibration error analysis method based on Kinect V2 as claimed in claim 3, characterized in that: The process of analyzing the distribution of reprojection errors consists of the following steps: Perform statistical analysis on the reprojection error data and calculate the basic statistics of the error, such as the mean, median, and standard deviation. The basic statistics can reflect the overall distribution characteristics of the error. For example, the mean reflects the central position of the error, and the standard deviation reflects the degree of dispersion of the error. By observing the skewness and kurtosis of the error distribution, we can determine whether the error distribution is symmetrical and whether it has peak or flat features. A skewness of zero indicates a symmetrical distribution, a positive skewness indicates a right-skewed distribution, and a negative skewness indicates a left-skewed distribution. A kurtosis greater than 3 indicates a peaked distribution, and a kurtosis less than 3 indicates a flat distribution. The local features of the error distribution are extracted, and the peak and valley areas in the error distribution are identified by calculating the local extreme points of the error distribution; the error sources are classified and quantified according to the statistical and morphological characteristics of the error distribution.

5. The camera calibration error analysis method based on Kinect V2 as claimed in claim 3, characterized in that: The process of identifying the main sources of error from the eigenvalues ​​and eigenvectors of the covariance matrix consists of the following steps: During the calibration process, the reprojection error data after each adjustment of the calibration plate position and camera posture is recorded; based on the collected reprojection error data, the covariance matrix of the error vector is calculated. The covariance matrix reflects the correlation and variance between the dimensions of the error vector; the covariance matrix is ​​decomposed into eigenvalues ​​to obtain eigenvalues ​​and corresponding eigenvectors, and the eigenvalues ​​are sorted by size. The larger the eigenvalue, the greater the contribution of the corresponding eigenvector in the error distribution. By analyzing the size of the eigenvalue, the main error source is identified. The error dimension corresponding to the eigenvector with a larger eigenvalue is the main error source. The proportion of each eigenvalue to the total eigenvalue is calculated to quantify the influence of each error source on the calibration result. The larger the eigenvalue ratio, the greater the influence of the corresponding error source on the calibration result. The error source classification is obtained through eigenvector and eigenvalue analysis.

6. The camera calibration error analysis method based on Kinect V2 as claimed in claim 3, characterized in that: The process of identifying the main sources of error through the error distribution plot includes the following steps: Obtain the reprojection error data of all images, group the reprojection error data by interval, and draw a histogram, where the horizontal axis of the histogram represents the size of the error and the vertical axis represents the frequency of the error; Observe the overall shape of the histogram, identify the characteristics of the error distribution, and whether it presents a normal distribution or a skewed distribution; determine the peak position in the histogram, analyze the error interval corresponding to the peak, and identify the concentrated area of ​​the error; observe the tail of the histogram, analyze the distribution of the extreme values ​​of the error, and identify possible abnormal error sources; According to the distribution characteristics of the histogram, the main source of error can be identified; if the peaks are concentrated in a specific interval, it may indicate that the parameters or conditions corresponding to this interval are the main source of error; analyze the secondary peaks or tail distribution of the histogram to identify the secondary error sources; by calculating the frequency share of different error intervals, their contribution to the overall error can be evaluated; combined with the covariance matrix analysis results, the correlation between the error sources can be confirmed and the possible error propagation paths can be identified.

7. The camera calibration error analysis method based on Kinect V2 as claimed in claim 6, characterized in that: The process of identifying the primary and secondary sources of error includes the following steps: Observe the peak position in the histogram and determine the interval where the error is most concentrated. The peak is concentrated in the interval where the error value is 0.5 to 1.0, so the parameters or conditions in the inferred interval have the greatest impact on the error; calculate the ratio of the error frequency in the peak interval to the total error frequency and evaluate its contribution to the overall error; In the histogram, observe the tail of the histogram, which contains extreme error values, analyze the shape and frequency of the tail distribution, and identify possible abnormal error sources; Based on the analysis results of the main error sources, secondary error sources, and error propagation paths, a comprehensive evaluation is conducted to determine which parameters or conditions have the greatest impact on the overall error, which are the secondary error sources, and how the error propagates in the system; ultimately, the main and secondary error sources are determined.

8. The camera calibration error analysis method based on Kinect V2 as claimed in claim 7, characterized in that: The process of finally identifying the main and secondary sources of error includes the following steps: Collect all error data, including the error data of the histogram peak interval, the error data of the tail distribution, and the data of the error propagation path; organize the collected data and classify them into main error source data, secondary error source data, and error propagation path data; Calculate the ratio of the error frequency in the peak interval to the total error frequency; Calculate the ratio of tail error frequency to total error frequency; Analyze how errors propagate in the system, identify the primary and secondary paths of error propagation, and assess the impact of error propagation paths on the final results.

9. The camera calibration error analysis method based on Kinect V2 as claimed in claim 1, characterized in that: The recalibration process includes the following steps: Through statistical analysis of the calibration results, the main error sources that affect the calibration accuracy are identified; the identified error sources are analyzed in detail to determine the specific way and degree of their impact on the calibration results; camera distortion mainly affects the shape of straight lines in the image, and the calibration plate position error mainly affects the accuracy of corner point extraction; According to the main error sources identified, increase the number of rows and columns of the calibration plate; place the calibration plate at different angles and distances; Re-collect multiple sets of image data at different angles and distances; use tools to re-extract the coordinates of the corner points of the calibration plate in each image, re-calibrate the camera, and calculate the intrinsic and extrinsic parameters of the camera.

10. A camera calibration error analysis system based on Kinect V2, characterized in that: Include: The parameter calculation module is responsible for collecting multiple sets of image data using the Kinect V2 camera, and performing denoising and grayscale preprocessing operations on the collected images; selecting a calibration plate, extracting the coordinates of the corner points of the calibration plate in each image, using tools to calibrate the camera, and calculating the intrinsic and extrinsic parameters of the camera; The error analysis module is responsible for applying the calibration results to the new image, calculating the distance between the reprojection point and the actual observation point, evaluating the calibration results, performing statistical analysis on the reprojection errors of all images, and calculating the average error and maximum error statistical indicators; by adjusting the calibration plate position and camera attitude input parameters, observing the changes in the calibration results, determining the degree of influence of each input parameter on the calibration results, performing eigenvalue decomposition on the covariance matrix of the calibration error, and analyzing the error sources and their influence on the calibration results; The calibration optimization module is responsible for recalibrating by adding calibration points or changing calibration conditions if certain error sources are found; optimizing the calibration results using a nonlinear optimization algorithm; acquiring new images to verify the calibration results, and performing reprojection error calculation and average error statistics on the new images.

Images