Method and device for calibrating depth camera to accurately measure size of object

Abstract

The invention relates to the technical field of object size measurement, in particular to a method and device for calibrating a depth camera to accurately measure the size of an object. The method for calibrating the object size accurately measured by the depth camera comprises the following steps: presetting a curved surface with a known function, and manufacturing a curved surface object with a preset size according to a preset proportional scale; according to the method, the measurement value of the depth camera can be calibrated, the depth camera is enabled to resist interference of environmental factors, so that the size of an object is accurately measured, multiple frames of images do not need to be shot in the calibration process, only one frame of image data is needed, and the calibration method does not depend on the use environment and can be used in a scene lacking obvious feature points; in addition, the distortion degree of the measurement result of the depth camera can be calculated, different calibration methods are used according to the distortion degree, the measurement result of the depth camera is calibrated seamlessly, and both efficiency and accuracy are considered.

Description

Technical Field

[0001] The present invention relates to the technical field of object size measurement, and specifically, to a method and device for calibrating a depth camera to accurately measure the size of an object. Background Art

[0002] A depth camera can capture the depth information of a scene and has been widely used in various fields such as robot navigation, virtual reality, object recognition and measurement. However, the imaging system of the depth camera will experience performance fluctuations over time, such as lens distortion, changes in the performance of the filter, changes in the camera structure, etc. In addition, the thermal expansion and contraction effect caused by temperature changes will also cause changes in the performance of the depth camera, which is reflected in the fluctuation of the depth accuracy of the scene measured by the depth camera over time and cannot achieve accurate measurement.

[0003] In order to make the measurement value of the depth camera more accurate, the Chinese patent with the application number 201110364401.1 calibrates the camera measurement value using the true depth value of known spatial points. This method has requirements for the use environment, requires finding feature points with significant features and easy to measure the distance from the camera, and at the same time needs to ensure that the spatial relationship between the camera and the feature points remains unchanged, which limits the application scope of this method.

[0004] The Chinese patent with the application number 201910379483.3 proposes a self-calibration method. This method requires taking multiple depth maps with a shared field of view, and also has requirements for the camera field of view environment, requiring multiple feature points with significant features within the field of view. At the same time, this self-calibration method only calibrates the camera when it is found that the degree of camera deformation exceeds a certain threshold, and does not calibrate the camera within the threshold range, and cannot achieve accurate measurement.

[0005] In view of the problems in the above background art, the present invention provides a method and device for calibrating a depth camera to accurately measure the size of an object, aiming to improve the measurement accuracy and stability of the depth camera. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and device for calibrating a depth camera to accurately measure the size of an object, so as to solve the problems of limited application scope and inability to achieve accurate measurement proposed in the above background art.

[0007] In a first aspect, to solve the above technical problems, an embodiment of the present invention provides a method for calibrating a depth camera to accurately measure the size of an object, including the following steps:

[0008] Preset a known function surface, and make a surface object with a preset size according to a preset scale;

[0009] The formula of the preset known function surface is:

[0010]

[0011] Among them, the relative position relationship of each point (x, y, z) on the surface shape of the curved surface object can be described by a preset function F as z = F(x, y). F can be an ellipsoidal surface, a plane, or other functional surfaces, where a, b, c, d, and e are all parameters;

[0012] The surface of the curved surface object is preset with a characteristic region having a fixed pattern. At the same time, a preset curved surface object is photographed by a depth camera to obtain 3D camera data of the curved surface object. Keeping the form of the preset function F unchanged, making each parameter of F a variable to be fitted, fitting the preset function F, and according to the fitting result, judging whether to use an online calibration method or iteratively update the camera parameters, and finally accurately measure the object using the depth camera.

[0013] In an alternative embodiment, the method for fitting the preset function F is: using the 3D camera data of the obtained curved surface object to construct an objective function, and using a conventional fitting method to obtain the parameters of the function F corresponding to the minimum of the objective function. The objective function is constructed as follows:

[0014]

[0015] where N represents the number of 3D point data of the curved surface object camera obtained. The coordinates of the i-th 3D point of the camera are (x i , y i , z i ). Using the least squares method to minimize L, the parameter values of the fitting function F are obtained.

[0016] In an alternative embodiment, the specific operation method for judging whether to use an online calibration method or iteratively update the camera parameters according to the fitting result is: when obtaining the parameter values of the fitting function F, judge whether the minimum L min is less than a preset threshold. If so, use the online calibration method; otherwise, iteratively update the camera parameters.

[0017] In an alternative embodiment, the specific operation method for using the online calibration method is:

[0018] Using a feature point detection algorithm to detect the position coordinates of the characteristic region with a fixed pattern preset on the surface of the curved surface object in the depth camera image, combining with the function F of the fitting parameters, the current camera parameters, and the conventional imaging principle, calculating the depth value of the characteristic region, then calculating the 3D coordinates of the characteristic region, calculating the proportionality coefficient r between the Euclidean distance of the calculated 3D coordinates of two pairwise characteristic regions and the preset Euclidean distance, and calibrating the depth value measured by the depth camera according to the proportionality coefficient r to obtain a more accurate depth value;

[0019] An embodiment of calculating the 3D coordinates of a feature region, where the preset function F adopts a plane function:

[0020] z = a·x + b·y + c

[0021] Assume that the position coordinates of the preset feature region in the image are (u, v), and its depth value can be calculated according to the following formula:

[0022]

[0023] where f x , f y is the focal length of the depth camera, and c x , c y is the optical center coordinate of the depth camera. After obtaining the coordinates (u, v, z) of the feature region, convert it to the point cloud space according to the conventional method to obtain the 3D coordinates.

[0024] In an alternative embodiment, the formula for calculating the proportionality coefficient r of its Euclidean distance to the preset Euclidean distance is:

[0025]

[0026] where M represents the number of pairs of preset feature regions on the surface of the preset curved surface object, and dc i and dp i respectively represent the Euclidean distance between the calculated 3D coordinates of the i-th pair of feature regions and the preset Euclidean distance between the i-th pair of feature regions.

[0027] In an alternative embodiment, the specific operation steps for calibrating the depth value measured by the depth camera according to the proportionality coefficient r to obtain a more accurate depth value are:

[0028] S11. Use the depth camera to capture the object to be measured to obtain a depth map I;

[0029] S12. Divide the depth value in the depth map I by the proportionality coefficient r, and output the depth map constructed by the new depth value as a more accurate measurement result.

[0030] In an alternative embodiment, the specific operation steps for iteratively updating the camera parameters are:

[0031] S21. Use the depth camera to capture the preset curved surface object to obtain a depth image, and record the coordinates of each pixel as P i = (u i , v i , z i );

[0032] S22. Recalculate the depth values of each pixel in the image captured by the depth camera using the preset function F with preset parameters, and obtain the pixel coordinates denoted as PF i =(u i , v i , zf i ), where zf i is the depth value calculated by combining the preset function F with preset parameters, the current camera parameters K, and the conventional imaging principle;

[0033] S23. Let the optimization variable be the camera parameters, and construct the objective function:

[0034]

[0035] Use a conventional optimization algorithm such as the Gauss - Newton method or the Levenberg - Marquadt method to minimize the objective function value and optimize the camera parameters:

[0036] S24. Let K = K', and determine whether J is less than the preset threshold. If so, output the updated K; otherwise, go to step S22.

[0037] In a second aspect, an embodiment of the present invention further provides a device for calibrating a depth camera to accurately measure the size of an object, including:

[0038] A memory for storing program instructions;

[0039] A processor for calling the program instructions stored in the memory and executing the steps included in any one of the implementation manners in the first aspect according to the obtained program instructions.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] The present invention can calibrate the measurement values of the depth camera, make the depth camera resistant to environmental factor interference, thereby accurately measuring the size of an object. At the same time, during the calibration process, it is not necessary to capture multiple frames of images, and only one frame of image data is required. Moreover, the calibration method does not depend on the use environment and can be used in scenarios lacking obvious feature points. In addition, it can calculate the distortion degree of the depth camera measurement result, and use different calibration methods according to the distortion degree to seamlessly calibrate the depth camera measurement result, taking into account both efficiency and accuracy. Specific Embodiments

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] An embodiment of the present invention provides a method for calibrating a depth camera to accurately measure the size of an object, including the following steps:

[0044] Preset a known function surface, and fabricate a surface object with a preset size according to a preset scale;

[0045] The formula for the preset known function surface is:

[0046]

[0047] Among them, the relative position relationship of each point (x, y, z) on the surface shape of the surface object can be described by a preset function F as z = F(x, y). F can be an ellipsoidal surface, a plane, or other function surfaces. Here, a, b, c, d, and e are all parameters. On the surface of the surface object, a characteristic area with a fixed pattern is preset. For example, the characteristic area can be a rectangular small hole, a circular small hole, a triangular small hole, etc.;

[0048] On the surface of the surface object, a characteristic area with a fixed pattern is preset. At the same time, use a depth camera to photograph the preset surface object to obtain 3D camera data of the surface object. Keep the form of the preset function F unchanged, and let each parameter of F be a variable to be fitted. Fit the preset function F. According to the fitting result, judge whether to use an online calibration method or iteratively update the camera parameters, and finally use the depth camera to accurately measure the object.

[0049] The method for fitting the preset function F is: use the 3D camera data of the obtained surface object to construct an objective function, and use a conventional fitting method to obtain the parameters of the function F corresponding to the minimum of the objective function. The objective function is constructed as:

[0050]

[0051] Among them, N represents the number of 3D point data of the surface object obtained by the camera. The i-th 3D point coordinate of the camera is (x i , y i , z i ). Use the least squares method to minimize L to obtain the parameter values of each parameter of the fitting function F.

[0052] According to the fitting result, the specific operation method for judging whether to use an online calibration method or iteratively update the camera parameters is: when obtaining the parameter values of each parameter of the fitting function F, judge whether the minimum L min is less than a preset threshold. If so, use the online calibration method; otherwise, iteratively update the camera parameters.

[0053] The specific operation method for using the online calibration method is:

[0054] Use the feature point detection algorithm to detect the position coordinates of the feature regions with a preset fixed pattern on the surface of the curved object in the depth camera image. Combine with the function F using fitting parameters, the current camera parameters, and the conventional imaging principle to calculate the depth values of the feature regions. Then calculate the 3D coordinates of the feature regions. According to the calculated 3D coordinates of the pairwise feature regions, calculate the proportionality coefficient r between their Euclidean distance and the preset Euclidean distance. Calibrate the depth values measured by the depth camera according to the proportionality coefficient r to obtain more accurate depth values;

[0055] An embodiment of calculating the 3D coordinates of the feature region, where the preset function F adopts a plane function:

[0056] z = a·x + b·y + c

[0057] Suppose the position coordinates of the preset feature region in the image are (u, v), and its depth value can be calculated according to the following formula:

[0058]

[0059] where f x , f y is the focal length of the depth camera, and c x , c y is the optical center coordinate of the depth camera. After obtaining the coordinates (u, v, z) of the feature region, convert it to the point cloud space according to the conventional method to obtain the 3D coordinates.

[0060] The formula for calculating the proportionality coefficient r between its Euclidean distance and the preset Euclidean distance is:

[0061]

[0062] where M represents the number of pairs of preset feature regions on the surface of the preset curved object, and dc i and dp i respectively represent the Euclidean distance between the calculated 3D coordinates of the i-th pairwise feature region pair and the preset Euclidean distance between the i-th pairwise feature region pair.

[0063] The specific operation steps for calibrating the depth values measured by the depth camera according to the proportionality coefficient r to obtain more accurate depth values are:

[0064] Step 1: Use the depth camera to capture the object to be measured to obtain the depth map I;

[0065] Step 2: Divide the depth values in the depth map I by the proportionality coefficient r, and output the depth map constructed by the new depth values as a more accurate measurement result.

[0066] The specific operation steps for iteratively updating the camera parameters are:

[0067] Step 1: Use a depth camera to capture a preset curved object to obtain a depth image, and record the coordinates of each pixel as P i =(u i , v i , z i );

[0068] Step 2: Use a preset function F with preset parameters to recalculate the depth values of each pixel in the image captured by the depth camera, and obtain the pixel coordinates recorded as PF i =(u i , v i , zf i ), where zf i is the depth value calculated by combining the preset function F with preset parameters, the current camera parameters K, and the conventional imaging principle;

[0069] Step 3: Let the optimization variable be the camera parameters, and construct an objective function:

[0070]

[0071] Use a conventional optimization algorithm such as the Gauss-Newton method or the Levenberg-Marquadt method to minimize the objective function value and optimize the camera parameters:

[0072] Step 4: Let K = K', and determine whether J is less than a preset threshold. If so, output the updated K; otherwise, go to Step 2.

[0073] Using the updated camera parameters, the depth camera can measure the object to be measured more accurately.

[0074] Based on the same inventive concept, an embodiment of the present invention further provides a device for calibrating a depth camera to accurately measure the size of an object. The device for calibrating a depth camera to accurately measure the size of an object can implement the functions corresponding to the method for calibrating a depth camera to accurately measure the size of an object described above. The device for calibrating a depth camera to accurately measure the size of an object can be a hardware structure, a software module, or a combination of a hardware structure and a software module. The device for calibrating a depth camera to accurately measure the size of an object can be implemented by a chip system. The chip system can be composed of chips, or can include chips and other discrete devices. The device for calibrating a depth camera to accurately measure the size of an object includes:

[0075] A memory for storing program instructions;

[0076] A processor for calling the program instructions stored in the memory and executing the steps included in any one of the above embodiments according to the obtained program instructions.

[0077] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for calibrating a depth camera to accurately measure the size of an object, characterized in that, It includes the following steps: Preset a known function surface, and make a surface object of a preset size according to a preset scale; The surface of the surface object is preset with a characteristic region having a fixed pattern. At the same time, use a depth camera to photograph the preset surface object to obtain 3D camera data of the surface object. Keep the form of the preset function F unchanged, and let each parameter of F be a variable to be fitted. Fit the preset function F. According to the fitting result, judge whether to use an online calibration method or iteratively update the camera parameters, and finally use the depth camera to accurately measure the object.

2. The method for accurately measuring the size of an object by calibrating a depth camera according to claim 1, characterized in that, The method for fitting the preset function F is: use the 3D camera data of the obtained surface object to construct an objective function, and use a conventional fitting method to obtain the parameters of the function F corresponding to the minimum of the objective function. Construct the objective function: where N represents the number of 3D point data of the surface object obtained by the camera, and the coordinates of the i-th 3D point of the camera are (x i , y i , z i ). Using the least squares method to minimize L, the parameter values of the fitting function F are obtained.

3. The method for accurately measuring the size of an object by calibrating a depth camera according to claim 2, characterized in that, According to the fitting result, the specific operation method for determining whether to use the online calibration method or to iteratively update the camera parameters is as follows: when obtaining the parameter values of the fitting function F, determine whether the minimum L min is less than a preset threshold. If so, use the online calibration method; otherwise, iteratively update the camera parameters.

4. The method for calibrating a depth camera to accurately measure the size of an object according to claim 3, wherein The specific operation method of using the online calibration method is: Use a feature point detection algorithm to detect the position coordinates of the characteristic region with a fixed pattern on the surface of the surface object in the depth camera image. Combine the function F using the fitting parameters, the current camera parameters, and the conventional imaging principle to calculate the depth value of the characteristic region, and then calculate the 3D coordinates of the characteristic region. According to the calculated 3D coordinates of two characteristic regions, calculate the proportionality coefficient r between its Euclidean distance and the preset Euclidean distance. Calibrate the depth value measured by the depth camera according to the proportionality coefficient r to obtain a more accurate depth value.

5. The method for accurately measuring the size of an object by calibrating a depth camera according to claim 4, wherein The formula for calculating the proportionality coefficient r between its Euclidean distance and the preset Euclidean distance is: Where M represents the number of pairs of preset feature regions on the surface of the preset curved surface object, dc i and dp i respectively represent the Euclidean distance between the calculated 3D coordinates between the i-th pair of feature regions, and the preset Euclidean distance between the i-th pair of feature regions.

6. The method for calibrating a depth camera to accurately measure the size of an object according to claim 4, characterized in that, The specific operation steps for calibrating the depth value measured by the depth camera according to the proportionality coefficient r to obtain a more accurate depth value are: S11. Use the depth camera to photograph the object to be measured to obtain a depth map I; S12. Divide the depth value in the depth map I by the proportionality coefficient r, and output the depth map constructed by the new depth value as a more accurate measurement result.

7. The method for accurately measuring the size of an object by calibrating a depth camera according to claim 3, wherein The specific operation steps for iteratively updating the camera parameters are: S21. Use a depth camera to capture a preset curved object to obtain a depth image, and record the coordinates of each pixel as P i =(u i , v i , z i ); S22. Recalculate the depth value of each pixel in the image captured by the depth camera using the preset function F with preset parameters, and obtain the pixel coordinates denoted as PF i =(u i , v i , zf i ), where zf i is the depth value calculated by combining the preset function F with preset parameters, the current camera parameters K, and the conventional imaging principle; S23. Let the optimization variable be the camera parameters, and construct an objective function: Use conventional optimization algorithms such as the Gauss-Newton method or the Levenberg-Marquadt method to minimize the objective function value and optimize the camera parameters: S24. Let K = K', judge whether J is less than the preset threshold. If so, output the updated K, otherwise enter step S22.

8. A device for calibrating a depth camera to accurately measure the size of an object, characterized in that, It includes: A memory for storing program instructions; A processor for calling the program instructions stored in the memory and executing the steps included in the method for calibrating a depth camera to accurately measure the size of an object according to any one of claims 1-7 according to the obtained program instructions.

Images