Real distance acquisition method and device of tof camera calibration board, and electronic equipment

Abstract

The application discloses a kind of real distance acquisition method and device of ToF camera calibration board, electronic equipment.The method comprises: obtaining the vertex in each corner point of the gray scale diagram of calibration board in the field of view of TOF camera, wherein the checkerboard used in lens calibration is used as the calibration board;With the intrinsic matrix of lens calibration as initial value, the corresponding diagonal euclidean distance is obtained according to the vertex;According to the actual length and width of the checkerboard, the corresponding diagonal real distance is obtained;According to the difference between the diagonal euclidean distance and the diagonal real distance, the intrinsic matrix is optimized;And the real distance of each pixel point in the gray scale diagram to the image sensor of TOF camera is calculated according to the optimized intrinsic matrix.The application uses the checkerboard used in lens calibration as FPPN calibration board, saves calibration time and cost;Based on the diagonal real distance of checkerboard, the diagonal euclidean distance is corrected, the intrinsic matrix is optimized, and the accuracy of calculating real distance is improved.

Description

Technical Field

[0001] This invention relates to the field of ToF ranging technology, and in particular to a method, apparatus, and electronic device for obtaining the true distance of a ToF camera calibration plate. Background Technology

[0002] Binocular ranging, structured light, and time-of-flight (ToF) are the three mainstream 3D imaging technologies today. Among them, ToF, due to its advantages such as simple principle, simple and stable structure, and long measurement distance, has been gradually applied to fields such as gesture recognition, 3D modeling, autonomous driving, and machine vision. The working principle of ToF technology is as follows: an external light source (such as VCSEL or LED) emits continuously modulated light. After the emitted light shines on the surface of the object being measured, it is reflected back. The reflected light is captured by the image sensor of the ToF camera. The depth / distance of the object from the camera is obtained by calculating the time difference or phase difference between the emitted and reflected light. The method of calculating distance using time difference is called pulsed ToF, and the method of calculating distance using phase difference is called continuous-wave ToF.

[0003] Due to the manufacturing process and optical characteristics of ToF camera image sensors, a series of calibrations are required to ensure measurement accuracy, such as lens calibration, wiggling calibration, and FPPN calibration. FPPN calibration compensates for differences in each pixel, necessitating a white board covering the entire field of view (FOV) of the image sensor. The mounting tilt angle of this calibration board must be accurately determined, and the true distance from each pixel in the image to the image sensor must be calculated; otherwise, these errors will lead to inaccurate FPPN calibration. However, the placement of the white boards on the production line is uncertain, making the calculation of the true distances difficult. Furthermore, lens calibration typically uses a black and white checkerboard pattern, requiring switching between different calibration boards on the production line, increasing calibration costs.

[0004] In addition, there are generally two methods to calculate the true distance from each pixel to the image sensor: The first method uses a mechanism to ensure the installation of the whiteboard and, based on the known angle between the whiteboard and the ground, and the distance from the center point of the image sensor to the whiteboard, the true distance from each pixel to the image sensor can be calculated using trigonometric functions. However, since ensuring the installation of the whiteboard requires additional hardware and increases calibration costs, it cannot be well utilized on existing production lines. The second method uses monocular ranging, which uses the world coordinate to pixel coordinate transformation relationship and the intrinsic and extrinsic parameter matrices to calculate the world coordinates of each pixel, thereby calculating the true distance from each pixel to the image sensor. However, the low resolution of the image sensor and the non-uniformity of the emitted modulated light make accurate intrinsic parameter calibration difficult.

[0005] Therefore, how to accurately calculate the true distance while saving the calibration cost of ToF cameras is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide a method, device, and electronic device for obtaining the true distance of a ToF camera calibration board, which solves the technical problem of increased calibration costs or inaccurate calculation of the true distance when calculating the true distance of the calibration board in existing ToF cameras. By using the checkerboard pattern used for lens calibration as the calibration board, the true distance of the calibration board can be accurately calculated while saving the calibration cost of the ToF camera.

[0007] To achieve the above objectives, the present invention provides a method for obtaining the true distance of a ToF camera calibration board, comprising the following steps: obtaining vertices of each corner point of the grayscale image of the calibration board in the field of view of the ToF camera, wherein the checkerboard used for lens calibration is used as the calibration board; using the intrinsic parameter matrix determined by the lens calibration as the initial value, obtaining the corresponding diagonal Euclidean distance according to the vertex; obtaining the corresponding diagonal true distance according to the actual length and width of the checkerboard; optimizing the intrinsic parameter matrix according to the difference between the diagonal Euclidean distance and the diagonal true distance; and calculating the true distance from each pixel in the grayscale image to the image sensor of the ToF camera according to the optimized intrinsic parameter matrix.

[0008] Optionally, the step of obtaining the vertices of each corner point of the grayscale image of the calibration board in the field of view of the TOF camera further includes: calculating the brightness value of each pixel in the grayscale image based on the original data of the grayscale image of the calibration board in the field of view of the TOF camera output by the image sensor of the TOF camera; obtaining each corner point of the grayscale image based on the brightness value; and selecting the vertices of each corner point based on the coordinates of each corner point.

[0009] Optionally, the step of iteratively optimizing the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the actual diagonal distance further includes: using the intrinsic parameter matrix calibrated by the lens as the initial value, and iteratively searching and calculating a target value that makes the difference between the diagonal Euclidean distance and the actual diagonal distance less than or equal to a preset difference range, thereby optimizing the intrinsic parameter matrix.

[0010] To achieve the above objectives, the present invention also provides a device for obtaining the true distance of a ToF camera calibration board, comprising: a vertex acquisition module, used to acquire vertices of each corner point of the grayscale image of the calibration board in the field of view of the ToF camera, wherein the checkerboard used for lens calibration is used as the calibration board; a diagonal Euclidean distance acquisition module, used to acquire the corresponding diagonal Euclidean distance based on the vertices using the intrinsic parameter matrix determined by the lens calibration as an initial value; a diagonal true distance acquisition module, used to acquire the corresponding diagonal true distance based on the actual length and width of the checkerboard; an optimization module, used to iteratively optimize the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the diagonal true distance; and a true distance acquisition module, used to calculate the true distance from each pixel in the grayscale image to the image sensor of the ToF camera based on the optimized intrinsic parameter matrix.

[0011] To achieve the above objectives, the present invention also provides an electronic device, including a memory, a processor, and a computer-executable program stored in the memory and executable on the processor, wherein the processor executes the computer-executable program to implement the steps of the ToF camera calibration board real distance acquisition method as described in the present invention.

[0012] The present invention provides a method and apparatus for obtaining the true distance of a ToF camera calibration board. By using the checkerboard pattern used in lens calibration as the calibration board for FPPN calibration, there is no need to provide an additional whiteboard covering the entire field of view for FPPN calibration, thus eliminating the need to switch between different calibration board patterns during production line calibration. By obtaining the vertices of each corner point of the checkerboard grayscale image in the field of view and obtaining the corresponding diagonal Euclidean distance, the checkerboard can be placed at virtually any position in the ToF camera's field of view. This eliminates the need to limit the parallelism between the ToF camera and the calibration board or to accurately provide the installation tilt angle of the calibration board, and it does not require knowing the distance from the center point of the image sensor to the calibration board, saving calibration time and cost. Based on the true diagonal distance of the checkerboard, the diagonal Euclidean distance is corrected, and the intrinsic parameter matrix obtained from lens calibration is optimized, thereby improving the accuracy of calculating the true distance. Furthermore, by using the checkerboard pattern used in lens calibration as the FPPN calibration board, the true distance can be calculated using the monocular ranging principle, accurately calculating the true distance of the calibration board while saving the calibration cost of the ToF camera. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a method for obtaining the true distance of a ToF camera calibration board according to an embodiment of the present invention;

[0015] Figure 2 This is a grayscale image of a checkerboard pattern provided in an embodiment of the present invention;

[0016] Figure 3 This is a schematic diagram of a chessboard pattern provided in an embodiment of the present invention;

[0017] Figure 4 This is a structural block diagram of a ToF camera calibration board real distance acquisition device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] One embodiment of the present invention provides a method for obtaining the true distance of a ToF camera calibration plate.

[0020] Please see Figure 1 This is a flowchart of a method for obtaining the true distance of a ToF camera calibration board according to an embodiment of the present invention. Figure 1 As shown, the method described in this embodiment includes the following steps: S11, obtaining the vertices of each corner point of the grayscale image of the calibration board in the field of view of the TOF camera, wherein the checkerboard used for lens calibration is used as the calibration board; S12, using the intrinsic parameter matrix determined by the lens calibration as the initial value, obtaining the corresponding diagonal Euclidean distance according to the vertex; S13, obtaining the corresponding diagonal true distance according to the actual length and width of the checkerboard; S14, optimizing the intrinsic parameter matrix according to the difference between the diagonal Euclidean distance and the diagonal true distance; and S15, calculating the true distance from each pixel point in the grayscale image to the image sensor of the TOF camera according to the optimized intrinsic parameter matrix.

[0021] Regarding step S11, obtaining the vertices of each corner point of the grayscale image of the calibration board in the field of view of the TOF camera, wherein the checkerboard used for lens calibration is used as the calibration board. Specifically, the TOF camera captures the checkerboard used for lens calibration, which serves as the calibration board, to obtain the grayscale image of the checkerboard in the field of view (fov) of the TOF camera, and further obtains each corner point of the grayscale image, thereby obtaining the corresponding vertices. For example, for an n*m checkerboard, each vertex corner_11, corner_1n, corner_m1, and corner_mn are detected and obtained, such as... Figure 2 As shown.

[0022] The calibration plane of the calibration board is the surface of the checkerboard facing the lens of the ToF camera, that is, the plane in the checkerboard used to reflect the light emitted by the ToF camera. In this embodiment, the checkerboard as the calibration board can be placed at virtually any position in the field of view of the ToF camera, and it is not necessary to know the distance from the center point of the image sensor to the calibration board. By using the black and white checkerboard used in lens calibration as the calibration board for FPPN calibration, there is no need to provide an additional white board covering the entire field of view for FPPN calibration, so that different calibration board patterns do not need to be switched during calibration on the production line. By obtaining the vertices of each corner point of the grayscale image of the checkerboard in the field of view, the checkerboard can be placed at virtually any position in the field of view of the ToF camera, without limiting the parallelism between the ToF camera and the calibration board or accurately giving the installation tilt angle of the calibration board; and it is not necessary to know the distance from the center point of the image sensor to the calibration board, saving calibration time and cost.

[0023] In some embodiments, the step of obtaining vertices among the corner points of the grayscale image of the calibration board in the field of view of the TOF camera further includes: calculating the amplitude value of each pixel in the grayscale image based on the raw data of the grayscale image of the calibration board in the field of view of the TOF camera output by the image sensor of the TOF camera; obtaining each corner point of the grayscale image based on the amplitude value; and selecting the vertices among the corner points based on the coordinates of each corner point.

[0024] In some embodiments, the brightness value (amplitude) of each pixel in the grayscale image is calculated using the following formula: amplitude = abs(I) + abs(Q); where I = A0 - B0 - (A180 - B180), Q = A90 - B90 - (A270 - B270), and A and B are the original data corresponding to the 0, 90, 180, and 270 phases output by the image sensor. The modulation wave emitted by the TOF camera is a square wave.

[0025] In some embodiments, the step of obtaining each corner point of the grayscale image based on the brightness value further includes: obtaining each corner point of the grayscale image using an OpenCV corner detection method based on the brightness value. For example, the Harris algorithm, Shi-Tomasi algorithm, etc., can be used to find, filter, and mark each corner point in the grayscale image based on the brightness value of the grayscale image.

[0026] Regarding step S12, using the intrinsic parameter matrix calibrated by the lens as the initial value, the corresponding diagonal Euclidean distance is obtained based on the vertices. Specifically, after obtaining each vertex in the grayscale image, the corresponding diagonal Euclidean distance of each vertex can be calculated based on the intrinsic parameter matrix calibrated by the lens. The diagonal Euclidean distance obtained at this time is the initial diagonal Euclidean distance based on the intrinsic parameter matrix calibrated by the lens. Subsequently, the diagonal Euclidean distance is corrected based on the true diagonal distance, thereby optimizing the intrinsic parameter matrix calibrated by the lens and improving the accuracy of calculating the true distance.

[0027] In some embodiments, the step of obtaining the corresponding diagonal Euclidean distance based on the vertex further includes: (1) calculating the world coordinates of each vertex in the grayscale image using the following pixel coordinate to world coordinate conversion relationship:

[0028]

[0029] Among them, z c M1 represents the scaling factor, u and v are the coordinates of a pixel in the pixel coordinate system, the matrix corresponding to M1 is the intrinsic parameter matrix, u0 and v0 are the center points of the intrinsic parameter matrix, and f x and f y Let M1 be the focal length of the intrinsic parameter matrix, M2 be the corresponding extrinsic parameter matrix, r be the direction vector of the pixel coordinate system axes in the world coordinate system axes, t be the translation vector from the origin of the world coordinate system to the origin of the pixel coordinate system, and x be the direction vector of the pixel coordinate system axes in the world ... w y w z w (1) The coordinates of the corresponding points in the world coordinate system; and (2) the corresponding diagonal Euclidean distances d1 and d2 are calculated using the following formulas based on the world coordinates of each vertex:

[0030] d1 = sqrt((x w11 -x wmn ) 2 +(y w11 -y wmn ) 2 +(z w11 -z wmn ) 2 ),

[0031] d2=sqrt((x w1n -x wm1 ) 2 +(y w1n -y wm1 ) 2 +(z w1n -z wm1 ) 2 ).

[0032] After obtaining the vertices at each corner of the grayscale image, the pixel coordinates of each vertex are converted to world coordinates based on the intrinsic parameter matrix calibrated by the lens. Then, the corresponding diagonal Euclidean distances d1 and d2 are calculated based on the world coordinates of each vertex. Figure 2 As shown. The diagonal Euclidean distance obtained at this time is the initial diagonal Euclidean distance based on the intrinsic parameter matrix calibrated by the lens. Subsequently, the diagonal Euclidean distance is corrected based on the true diagonal distance to optimize the intrinsic parameter matrix calibrated by the lens, thereby improving the accuracy of calculating the true distance.

[0033] Regarding step S13, obtain the corresponding true diagonal distance based on the actual length and width of the chessboard grid. Specifically, for the selected chessboard grid, its actual length and width are known, and the true diagonal distance can be calculated using the Pythagorean theorem.

[0034] Specifically, the step of obtaining the corresponding true diagonal distance based on the actual length and width of the chessboard grid further includes: calculating the corresponding true diagonal distances drel1 and drel2 using the Pythagorean theorem: drel1 = drel2 = sqrt(x 2 +y 2 ); where x is the actual length of the chessboard square (the horizontal length of the chessboard square in the illustration), and y is the actual width of the chessboard square (the vertical length of the chessboard square in the illustration), such as Figure 3 As shown.

[0035] Regarding step S14, the intrinsic parameter matrix is ​​optimized based on the difference between the diagonal Euclidean distance and the true diagonal distance. Specifically, due to the low resolution of the ToF camera's image sensor and the non-uniformity of the modulated light, there may be a difference between the diagonal Euclidean distance obtained based on the intrinsic parameter matrix calibrated by the lens and the true diagonal distance; if the true distance is calculated based on the intrinsic parameter matrix calibrated by the lens, there may be a certain deviation. Therefore, using the intrinsic parameter matrix calibrated by the lens as the initial value, a suitable target intrinsic parameter matrix is ​​calculated through iterative search to improve the accuracy of the true distance calculation.

[0036] In some embodiments, the internal parameter matrix calibrated by the lens is used as an initial value (i.e., the initial internal parameter matrix), and a target value that makes the difference between the diagonal Euclidean distance and the real diagonal distance less than or equal to a preset difference range is calculated through iterative search, so as to optimize the internal parameter matrix. Specifically, the optimal solution of the internal parameter matrix is found iteratively by changing the values of variables in the initial internal parameter matrix.

[0037] Specifically, the iterative constraint added according to the real diagonal distance and the diagonal Euclidean distance is argmin(abs(drel1-d1)+abs(drel2-d2)). Considering that the focal lengths fx and fy of the internal parameter matrix are relatively close, the following further requirement shall be satisfied: abs(f x -f y )<thr, where thr is the maximum threshold of focal length difference defined by the user. By changing f x and / or f y the diagonal Euclidean distances d1 and d2 calculated based on the corrected internal parameter matrix can satisfy argmin(abs(drel1-d1)+abs(drel2-d2)). It is assumed that initially f x =100, f y =100, by increasing or decreasing f x (for example, f x changes within 90 to 110, and satisfies: abs(f x -f y )<thr), a new internal parameter matrix is obtained, and the diagonal Euclidean distances d1 and d2 are recalculated; or by increasing or decreasing f y (for example, f y changes within 90 to 110, and satisfies: abs(f x -f y )<thr), a new internal parameter matrix is obtained, and the diagonal Euclidean distances d1 and d2 are recalculated; or change f x and f y at the same time (for example, f x changes within 90 to 110, f y also changes within 90 to 110, and satisfies: abs(f x -f y )<thr), a new internal parameter matrix is obtained, and the diagonal Euclidean distances d1 and d2 are recalculated.

[0038] In some embodiments, the step of iteratively optimizing the internal parameter matrix according to the difference between the diagonal Euclidean distance and the true diagonal distance further comprises: determining whether the following relational expression holds: argmin(abs(drel1-d1)+abs(drel2-d2))&&abs(fx-fy)<thr, wherein drel1 and drel2 are true diagonal distances, d1 and d2 are diagonal Euclidean distances, and thr is the maximum threshold of a predetermined focal length difference; if the relational expression holds, the current internal parameter matrix is obtained as the optimized internal parameter matrix; if the relational expression does not hold, the values of fx and / or fy are changed, and an internal parameter matrix satisfying the relational expression is iteratively searched as the optimized internal parameter matrix. Wherein, argmin refers to obtaining the values of d1 and d2 when the objective function abs(drel1-d1)+abs(drel2-d2) reaches the minimum value; && represents logical AND, that is, and. When the results of the expressions on both sides of the operator are both true, the result of the entire operation is true; otherwise, if any one side is false, the result is false; && also has a short-circuit function, that is, if the first expression is false, the second expression will not be calculated.

[0039] Regarding step S15, calculating the true distance from each pixel point in the grayscale image to the image sensor of the TOF camera according to the optimized internal parameter matrix. Specifically, according to the optimized internal parameter matrix, the true distance from each pixel point in the grayscale image to the image sensor of the TOF camera can be calculated through the conversion relationship between pixel coordinates and world coordinates. Moreover, since the internal parameter matrix calibrated by lens is used as an initial value, iterative constraint conditions are added according to the true diagonal distance and the diagonal Euclidean distance, and a suitable set of target internal parameter matrix is calculated through iterative search, which improves the accuracy of true distance calculation. The present invention uses the checkerboard for lens calibration as the FPPN calibration board, and can calculate the true distance by using the monocular ranging principle, which can accurately calculate the true distance of the calibration board on the basis of saving the calibration cost of the ToF camera.

[0040] Based on the same inventive concept, the present invention also provides a device for acquiring a true distance of a ToF camera calibration board. The provided device for acquiring a true distance of a ToF camera calibration board can adopt the method as Figure 1 shown method for acquiring a true distance of a ToF camera calibration board to perform FPPN calibration on a ToF camera and then acquire the true distance of the calibration board.

[0041] Please refer to Figure 4 , which is a structural block diagram of the device for acquiring a true distance of a ToF camera calibration board provided by an embodiment of the present invention. As shown in Figure 4As shown, the true distance acquisition device for the ToF camera calibration board includes: a vertex acquisition module 41, a diagonal Euclidean distance acquisition module 42, a diagonal true distance acquisition module 43, an optimization module 44, and a true distance acquisition module 45.

[0042] Specifically, the vertex acquisition module 41 is used to acquire vertices at each corner of the grayscale image of the calibration board in the field of view of the TOF camera; wherein, the checkerboard used for lens calibration is used as the calibration board. The diagonal Euclidean distance acquisition module 42 is used to acquire the corresponding diagonal Euclidean distance based on the vertices, using the intrinsic parameter matrix determined by the lens calibration as the initial value. The diagonal true distance acquisition module 43 is used to acquire the corresponding diagonal true distance based on the actual length and width of the checkerboard. The optimization module 44 is used to iteratively optimize the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the diagonal true distance. The true distance acquisition module 45 is used to calculate the true distance from each pixel in the grayscale image to the image sensor of the TOF camera based on the optimized intrinsic parameter matrix. The working mode of each module can be referred to Figure 1 The descriptions of the corresponding steps in the method for obtaining the true distance of the ToF camera calibration board shown are not repeated here.

[0043] The present invention provides a method and apparatus for obtaining the true distance of a ToF camera calibration board. By using the checkerboard pattern used in lens calibration as the calibration board for FPPN calibration, there is no need to provide an additional whiteboard covering the entire field of view for FPPN calibration, thus eliminating the need to switch between different calibration board patterns during production line calibration. By obtaining the vertices of each corner point of the checkerboard grayscale image in the field of view and obtaining the corresponding diagonal Euclidean distance, the checkerboard can be placed at virtually any position in the ToF camera's field of view. This eliminates the need to limit the parallelism between the ToF camera and the calibration board or to accurately provide the installation tilt angle of the calibration board, and it does not require knowing the distance from the center point of the image sensor to the calibration board, saving calibration time and cost. Based on the true diagonal distance of the checkerboard, the diagonal Euclidean distance is corrected, and the intrinsic parameter matrix obtained from lens calibration is optimized, thereby improving the accuracy of calculating the true distance. Furthermore, by using the checkerboard pattern used in lens calibration as the FPPN calibration board, the true distance can be calculated using the monocular ranging principle, accurately calculating the true distance of the calibration board while saving the calibration cost of the ToF camera.

[0044] Based on the same inventive concept, the present invention also provides an electronic device, including a memory, a processor, and a computer-executable program stored in the memory and executable on the processor; when the processor executes the computer-executable program, it implements as follows: Figure 1 The steps of the method for obtaining the true distance of the ToF camera calibration board are shown.

[0045] Within the scope of this inventive concept, embodiments can be described and illustrated based on modules that perform one or more of the described functions. These modules (also referred to herein as units, etc.) can be physically implemented by analog and / or digital circuitry, such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, etc., and can optionally be driven by firmware and / or software. The circuitry can, for example, be implemented in one or more semiconductor chips. The circuitry constituting a module can be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware performing some functions of the module and a processor performing other functions of the module. Without departing from the scope of this inventive concept, each module of an embodiment can be physically divided into two or more interactive and discrete modules. Similarly, without departing from the scope of this inventive concept, the modules of an embodiment can be physically combined into more complex modules.

[0046] Generally, terms can be understood at least partially from their usage in context. For example, the term "one or more" as used herein depends at least in part on the context and can be used to describe a feature, structure, or characteristic in a singular sense, or in a plural sense to describe a combination of features, structures, or characteristics. Additionally, the term "based on" can be understood not necessarily to express an exclusive set of factors, but rather, alternatively, also depends at least in part on the context, allowing for the presence of other factors that are not necessarily explicitly described.

[0047] It should be noted that the terms "comprising" and "having," and their variations, used in this invention document are intended to cover non-exclusive inclusion. The terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence, unless explicitly indicated by the context. It should be understood that such data used interchangeably where appropriate. Furthermore, embodiments and features within embodiments of this invention can be combined with each other unless otherwise specified. In addition, descriptions of well-known components and technologies have been omitted in the above description to avoid unnecessarily obscuring the concepts of this invention. In the various embodiments described above, each embodiment focuses on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0048] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for obtaining the true distance of a ToF camera calibration plate, characterized in that, The process includes the following steps: obtaining vertices at each corner of the grayscale image of the calibration board in the field of view of the TOF camera, wherein the checkerboard used for lens calibration is used as the calibration board; using the intrinsic parameter matrix determined by the lens calibration as the initial value, obtaining the corresponding diagonal Euclidean distance based on the vertices; obtaining the corresponding diagonal true distance based on the actual length and width of the checkerboard; optimizing the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the diagonal true distance; and calculating the true distance from each pixel in the grayscale image to the image sensor of the TOF camera based on the optimized intrinsic parameter matrix.

2. The method according to claim 1, characterized in that, The step of obtaining the vertices of each corner point of the grayscale image of the calibration board in the field of view of the TOF camera further includes: calculating the brightness value of each pixel in the grayscale image based on the original data of the grayscale image of the calibration board in the field of view of the TOF camera output by the image sensor of the TOF camera; obtaining each corner point of the grayscale image based on the brightness value; and selecting the vertices of each corner point based on the coordinates of each corner point.

3. The method according to claim 2, characterized in that, The brightness value (amplitude) of each pixel in the grayscale image is calculated using the following formula: amplitude = abs (I) + abs (Q), where I = A0 - B0 - (A180 - B180), Q = A90 - B90 - (A270 - B270), and A and B are the original data corresponding to the 0, 90, 180, and 270 phases output by the image sensor.

4. The method according to claim 2, characterized in that, The step of obtaining each corner point of the grayscale image based on the brightness value further includes: obtaining each corner point of the grayscale image using the OpenCV corner detection method based on the brightness value.

5. The method according to claim 1, characterized in that, The step of iteratively optimizing the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the diagonal true distance further includes: using the intrinsic parameter matrix calibrated by the lens as the initial value, and iteratively searching and calculating a target value that makes the difference between the diagonal Euclidean distance and the diagonal true distance less than or equal to a preset difference range, thereby optimizing the intrinsic parameter matrix.

6. The method according to claim 1, characterized in that, The step of obtaining the corresponding diagonal Euclidean distance based on the vertex further includes: calculating the world coordinates of each vertex using the following conversion relationship between pixel coordinates and world coordinates: , , ; where z c M1 represents the scaling factor, u and v are the coordinates of a pixel in the pixel coordinate system, the matrix corresponding to M1 is the intrinsic parameter matrix, u0 and v0 are the center points of the intrinsic parameter matrix, and f x and f y Let M1 be the focal length of the intrinsic parameter matrix, M2 be the corresponding extrinsic parameter matrix, r be the direction vector of the pixel coordinate system axes in the world coordinate system axes, t be the translation vector from the origin of the world coordinate system to the origin of the pixel coordinate system, and x be the direction vector of the pixel coordinate system axes in the world ... w y w z w Let x be the coordinates of the corresponding point in the world coordinate system; and let d1 and d2 be the corresponding diagonal Euclidean distances calculated using the following formula based on the world coordinates of each vertex: d1 = sqrt((x... w11 - x wmn ) 2 + (y w11 - y wmn ) 2 + (z w11 - z wmn ) 2 ), d2=sqrt((x w1n - x wm1 ) 2 + (y w1n - y wm1 ) 2 + (z w1n - z wm1 ) 2 ); where n and m represent the number of columns and rows of the chessboard, respectively, ln represents the vertex of the 1st row and nth column, m1 represents the vertex of the mth row and 1st column, and mn represents the vertex of the mth row and nth column.

7. The method according to claim 6, characterized in that, The step of obtaining the corresponding true diagonal distance based on the actual length and width of the chessboard grid further includes: calculating the corresponding true diagonal distances drel1 and drel2 using the Pythagorean theorem: drel1 = drel2 = sqrt(x 2 +y 2 ); where x is the actual length of the chessboard square and y is the actual width of the chessboard square.

8. The method according to claim 7, characterized in that, The step of iteratively optimizing the internal parameter matrix according to the difference between the diagonal Euclidean distance and the real diagonal distance further comprises: determining whether the following relational expression holds: argmin(abs(drel1-d1)+ abs(drel2-d2)) && abs(f x -f y )<thr, where drel1 and drel2 are real diagonal distances, d1 and d2 are diagonal Euclidean distances, thr is the maximum threshold of a predetermined focal length difference, && represents logical AND, that is, the overall operation result is true only when the results of expressions on both sides of && are true; if the relational expression holds, obtaining a current internal parameter matrix as an optimized internal parameter matrix; if the relational expression does not hold, changing f x and / or f y values, and iteratively searching for an internal parameter matrix satisfying the relational expression as an optimized internal parameter matrix.

9. A device for acquiring the true distance of a ToF camera calibration plate, characterized in that, include: The system includes a vertex acquisition module for acquiring vertices at each corner of the grayscale image of the calibration board in the field of view of the TOF camera, wherein the calibration board is a checkerboard used for lens calibration; a diagonal Euclidean distance acquisition module for acquiring the corresponding diagonal Euclidean distance based on the vertices, using the intrinsic parameter matrix determined by the lens calibration as an initial value; a diagonal true distance acquisition module for acquiring the corresponding diagonal true distance based on the actual length and width of the checkerboard; an optimization module for iteratively optimizing the intrinsic parameter matrix based on the difference between the diagonal Euclidean distance and the diagonal true distance; and a true distance acquisition module for calculating the true distance from each pixel in the grayscale image to the image sensor of the TOF camera based on the optimized intrinsic parameter matrix.

10. An electronic device comprising a memory, a processor, and a computer-executable program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer-executable program, it implements the steps of the method for obtaining the true distance of the ToF camera calibration plate as described in any one of claims 1 to 8.

Images