Depth measurement method, camera calibration result evaluation method, equipment and storage medium

Abstract

The invention discloses a depth measurement method, a camera calibration result evaluation method, equipment and a storage medium, and relates to the technical field of machine vision, and the method comprises the steps: obtaining an object distance and a collection image for a target feature point; calculating a sight line included angle according to the pixel coordinates of the target feature point in the acquired image; acquiring a principal point offset between the center and the optical center of the camera lens; and calculating depth information of the target feature point according to the triangle theorem in combination with the main point offset, the object distance and the sight line included angle. Therefore, the method can dynamically adapt to the spatial relationship between different feature points and the camera, corrects the measurement error caused by the fact that the attitude position of the feature point relative to the optical center is not considered in a traditional method, and improves the depth measurement precision of the camera.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a depth measurement method, a camera calibration result evaluation method, a device, and a storage medium. Background Art

[0002] In 3D ranging and camera calibration applications, accurately measuring the true distance between the camera's optical center (i.e., perspective center) and external spatial feature points is an important step in evaluating the camera's geometric accuracy and measurement system performance.

[0003] Currently, practical engineering often uses the distance between the center of the camera lens and the object as an approximate substitute, using the object distance as an approximate depth measurement. Alternatively, some experts and scholars have proposed applying a fixed compensation to the object distance based on optical design parameters (such as entrance pupil distance or viewpoint depth) to calibrate depth measurements.

[0004] However, although the above-mentioned depth measurement method simplifies the measurement process, it fails to accurately consider the measurement deviation caused by changes in the internal structure of the camera and the relative posture of the optical center and the target point, and is prone to introducing millimeter-level systematic errors. Summary of the Invention

[0005] The embodiments of the present application provide a depth measurement method, a camera calibration result evaluation method, a device, a storage medium, and a program product, which are used to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, an embodiment of the present application provides a depth measurement method, including: obtaining an object distance and a captured image for a target feature point; the object distance is the distance between the target feature point and the center of a camera lens; calculating a line of sight angle based on the pixel coordinates of the target feature point in the captured image; the line of sight angle is the angle between a straight line passing through the target feature point and the optical center and the optical axis; obtaining a principal point offset between the center of the camera lens and the optical center; and calculating the depth information of the target feature point based on the triangle theorem, combining the principal point offset, the object distance, and the line of sight angle.

[0007] In a second aspect, an embodiment of the present application provides a method for evaluating camera calibration results, including: obtaining an object distance and a captured image for a first feature point; the object distance is the distance between the first feature point and the center of the camera lens; calculating a line of sight angle based on the pixel coordinates of the first feature point in the captured image; the line of sight angle is the angle between a straight line passing through the first feature point and the optical center and the optical axis; obtaining a principal point offset between the center of the camera lens and the optical center; calculating a true depth value of the first feature point based on the triangle theorem, combining the principal point offset, the object distance, and the line of sight angle; outputting first depth information corresponding to the first feature point based on a visual ranging model, and evaluating the camera calibration result based on the first depth information and the true depth value.

[0008] In a third aspect, an embodiment of the present application provides a storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the depth measurement method or camera calibration result evaluation method described in any of the above items of the present application.

[0009] In a fourth aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the above-mentioned depth measurement methods or camera calibration result evaluation methods of the present application.

[0010] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned depth measurement methods or camera calibration result evaluation methods.

[0011] The beneficial effects of the embodiments of the present application are: By using the object distance between the camera lens center and the target point as a basis, and further introducing the calculation of the line of sight angle, the spatial orientation changes of feature points in the field of view are captured. Simultaneously, a triangulated geometric model of the true light path is established in combination with the principal point offset, effectively incorporating the effects of optical structure and perspective offset on depth calculation into the measurement process. This allows for dynamic adaptation to the spatial relationship between different feature points and the camera, correcting for measurement errors caused by traditional methods that fail to consider the position of feature points relative to the optical center, ensuring the accuracy and consistency of depth data across different perspectives and working scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A flowchart showing an example of a depth measurement method according to an embodiment of the present application is shown; Figure 2 A schematic diagram showing an optical principle of an example of camera depth measurement; Figure 3 A schematic diagram showing the simulation effect of the relationship between the depth error and the sight angle measured without compensation and with fixed compensation in the current related technology is shown; Figure 4 A flowchart showing an example of a depth measurement method according to an embodiment of the present application; Figure 5 An operational flow chart illustrating an example of calculating a sight line angle based on pixel coordinates of a target feature point according to an embodiment of the present application is shown; Figure 6 A schematic diagram showing a simulation effect of an example of the deviation err of the depth true value measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the sight angle is 0°; Figure 7 A schematic diagram showing the simulation effect of an example of the deviation err of the true depth measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the line of sight angle is 45°; Figure 8 A schematic diagram showing the simulation effect of an example of the deviation err of the depth true value measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the line of sight angle is 80°; Figure 9 A flowchart illustrating an example of a method for evaluating camera calibration results according to an embodiment of the present application is shown; Figure 10 An operational flowchart of an example of evaluating camera calibration results by triangulation according to an embodiment of the present application is shown; Figure 11 A schematic diagram showing the optical principle effect of an example of evaluating camera calibration results based on triangulation is shown; Figure 12 A schematic diagram showing the simulation effect of an example of the influence of OP1 error on P1P2 ranging under a fixed viewing angle; Figure 13 This is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0015] It should also be noted that, in this document, the terms "include" and "comprising" include not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, the elements defined by the phrase "include..." do not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the elements.

[0016] In the technical solutions of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0017] Figure 1 A flowchart of an example of a depth measurement method according to an embodiment of the present application is shown.

[0018] Regarding the executor of the method of the embodiment of the present application, it can be any controller or processor with computing or processing capabilities, such as a camera processor, etc. By combining the image pixel coordinates of the target feature point, object distance information, line of sight angle and internal structural parameters of the camera (such as principal point offset, etc.), a depth measurement method based on a geometric model is established. The depth information obtained directly corresponds to the geometric distance in the actual space, thereby achieving a high-precision estimation of the actual distance between the target feature point and the optical center of the camera.

[0019] In some examples, the method of the embodiments of the present application can be integrated into an electronic device or terminal through software, hardware, or a combination of software and hardware, and the type of terminal or electronic device can be diverse, such as a camera, a mobile phone, a tablet or desktop, a vehicle controller, etc.

[0020] like Figure 1 As shown, in step S110, the object distance and the captured image for the target feature point are obtained, where the object distance is the distance between the target feature point and the center of the camera lens.

[0021] In some embodiments, image data containing target feature points is captured by a camera's imaging component, for example, by invoking an internal image sensor. The target feature points can be natural corners, landmarks, or intersections of structural edges, and must be stable and detectable. Furthermore, a ranging module or device is invoked to measure the spatial distance between the currently imaged target feature point and the center of the camera lens, i.e., the "object distance." Specific distance measurement details can be found in current related technologies.

[0022] In step S120 , the sight line angle is calculated according to the pixel coordinates of the target feature point in the captured image. The sight line angle is the angle between the straight line passing through the target feature point and the optical center and the optical axis.

[0023] Specifically, the two-dimensional coordinates in the image coordinate system are converted into a unit direction vector in the camera coordinate system based on the intrinsic parameters calibrated by the camera (such as focal length, principal point position, pixel size, distortion parameters, etc.). The direction vector starts from the optical center of the camera and points toward the spatial direction of the target feature point in the image. The line of sight direction is defined based on this, and the line of sight angle can be obtained based on the unit direction vector.

[0024] In step S130 , the principal point offset between the center of the camera lens and the optical center is obtained.

[0025] It should be noted that the principal point offset can be pre-calibrated during the manufacturing or deployment phase via an internal camera calibration program. This constant is written into the camera's internal geometric model and can be directly read from the corresponding storage module. In fixed-focus cameras, the principal point offset is a fixed value. However, in some high-end cameras with self-calibration capabilities, the principal point offset parameter can be dynamically estimated and updated using image data and spatial feedback to adapt to lens changes at different zoom states.

[0026] In step S140 , the depth information of the target feature point is calculated based on the triangle theorem, the principal point offset, the object distance, and the sight angle.

[0027] Here, a complete spatial geometric relationship is established based on the previously acquired object distance (the distance from the lens center to the target point), principal point offset (the distance from the lens center to the optical center), and line of sight angle (the angle between the target point's line of sight and the optical axis). Specifically, a triangular model containing these sides and angles is constructed. In this triangle, one side is the modulus of the principal point offset vector, the opposite side is the object distance, and the included angle is the line of sight angle. Based on this known angle and the lengths of the two sides, a preset mathematical model (such as the law of cosines) is used to solve for the third side—the real-world distance from the optical center to the target point—to obtain the measured depth information.

[0028] Compared to traditional "surface approximate ranging," the embodiments of this application utilize "structurally precise ranging," reflecting the true spatial position of the target point relative to the imaging optical center. This provides higher geometric accuracy and system consistency than object distance, achieving a transition from incomplete to complete coupling between image and ranging points. This system achieves highly consistent and accurate depth measurement across the entire field of view, particularly in scenarios with varying viewing angles, posture adjustments, or widely distributed targets.

[0029] Figure 2 A schematic diagram showing the optical principle of an example of camera depth measurement.

[0030] It should be noted that the current method of approximating the distance between the camera's optical center and the target feature point is to measure the distance from the camera's lens center to the target feature point without adding compensation or by adding fixed compensation (selecting lens design dimensions such as entrance pupil distance or viewpoint depth). The optical center is usually located millimeters below the center of the first lens. As the relative posture of the camera and 3D point changes, both the uncompensated and fixed compensation methods use a "hypothetical proxy distance", which can introduce millimeter-level deviations.

[0031] like Figure 2 As shown in the figure, when using the non-compensation method, the object distance measure_l1 is directly used as an approximation. In this case, the true error between the lens optical center and the target feature point is L1 - measure_l1. When using the fixed compensation method, measure_l1 + OM is used to approximate L1. According to the triangle theorem, the actual measure_l1 + OM > L1. Furthermore, the error range will adjust accordingly as the viewing angle changes.

[0032] Figure 3 The figure shows a schematic diagram of the simulation effect of the relationship between the depth error and the sight angle measured without compensation and with fixed compensation in the current related technology.

[0033] like Figure 3 As shown in the figure, assuming that measure_l1=10m and OM=0.007m are fixed, the angle theta between the target point and the optical axis ranges from [0,80°]. When no compensation is used, the maximum error at 0° is 0.007m. When a fixed compensation scheme is used, the maximum error at 80° is -0.0058m, resulting in a maximum measurement error of 5~7mm.

[0034] Unlike the methods of no compensation and using fixed value compensation, a dynamic compensation method is proposed in the embodiment of the present application. Specifically, the dynamic changes in depth measurement errors caused by changes in the relative posture between the camera and the target feature points are comprehensively considered, and dynamic optical center compensation is proposed by combining the relative viewing angle of the feature points with the camera intrinsic parameter calibration results.

[0035] Figure 4 A flowchart illustrating an example of a depth measurement method according to an embodiment of the present application is shown.

[0036] like Figure 4 As shown, in step S410, the distance measure_l1 from the center of the first lens to the target feature point is measured.

[0037] In step S420 , an image is collected and pixel coordinates (u, v) of target feature points are extracted.

[0038] In step S430 , based on the pixel coordinates (u, v) of the target feature point and in combination with the camera intrinsic parameter matrix and the distortion parameter, the sight angle theta1 corresponding to the target feature point is calculated.

[0039] In one example of an embodiment of the present application, the sight angle can be measured using various known measurement methods, such as by calling a sight angle measurement component. In another example of an embodiment of the present application, the sight angle can also be measured using the novel solution proposed herein. More details will be provided below in conjunction with other examples.

[0040] In step S440 , the distance OP1 from the target feature point to the optical center is calculated according to the triangle theorem.

[0041] More specifically, based on the known parameters and the calculated line of sight angle, the cosine theorem is used to accurately solve the distance OP1 between the target feature point and the optical center of the camera.

[0042] refer to Figure 2 In the geometric relationship shown in , in the triangle OMP1, point O represents the camera optical center, point M represents the center of the first lens, and point P1 is the position of the target feature point in space. The known side lengths in this triangle include: Distance from optical center O to center M of first lens: ; The distance from the center M of the first lens to the target feature point P1: ; The angle between the optical axis (OM) and the line of sight direction (OP1) of the target feature point: theta1 or , which is obtained by combining pixel coordinates with camera intrinsic parameters.

[0043] According to the law of cosines, the following mathematical relationship can be established: , formula (1) Where, is the measured distance from the center of the first lens to the target feature point, is the known distance from the camera's optical center to the center of the first lens, is the distance from the target feature point to the optical center of the camera, Theta1 is the angle between the feature point direction and the optical axis.

[0044] The above formula can be organized into The quadratic equation of : , Formula (2) By solving the quadratic equation, we can get Since the distance must be a non-negative real number, the positive solution with physical meaning is selected as the final result.

[0045] In some examples of the embodiments of the present application, the principal point offset between the center of the camera lens and the optical center is determined based on one or more of the measurement distance, the entrance pupil distance and the viewpoint depth, where the measurement distance is the measurement distance between the center of the camera lens and the optical center measured by optical measurement equipment.

[0046] More specifically, the distance from the optical center to the center of the first lens This can be obtained through precise measurement or design parameter approximation. Precise measurement involves direct measurement using an optical coordinate measuring instrument (such as a three-dimensional coordinate measuring machine or a dedicated optical calibration device); approximate estimation is performed using parameters such as the "entrance pupil distance" or "viewpoint depth" provided in the lens design documentation. By introducing parameters such as the measurement distance, entrance pupil distance, or viewpoint depth to derive the principal point offset between the camera lens center and the optical center, the principal point offset is transformed from a static assumption to a dynamically measurable one, meeting the depth measurement accuracy requirements of various cameras (for example, zoom cameras) and improving the geometric accuracy and system adaptability of depth measurement.

[0047] Through the embodiments of this application, the law of cosines is applied based on the line-of-sight angle between the feature point, actual distance measurement data, and known camera structural parameters to derive the precise distance from the feature point to the optical center, achieving geometric compensation of the optical center position. This fusion of physical measurement and geometric modeling enhances the accuracy and consistency of measurement results, avoids the uncertainty caused by parameter estimation, and effectively improves the stability and accuracy of the camera system in spatial positioning and visual measurement.

[0048] Figure 5 An operational flowchart of an example of calculating the sight line angle based on the pixel coordinates of a target feature point according to an embodiment of the present application is shown.

[0049] like Figure 5 As shown, in step S510, the pixel coordinates are converted into normalized coordinates in the camera coordinate system according to the camera intrinsic parameters.

[0050] Here, the pixel coordinates of the target feature point on the two-dimensional image are converted into a normalized direction vector in the three-dimensional camera coordinate system, which is also called "back projection" to remap the coordinates on the image plane back to the three-dimensional direction that the light passes through during imaging.

[0051] Specifically, let the pixel coordinates of the target feature point on the image plane be , the camera intrinsic parameter matrix is: , Formula (3) Then the pixel coordinates are converted to the normalized image coordinate system (ie, the camera coordinate system The coordinates on the plane are: , Formula (4) Where, is the normalized coordinate under the distorted image.

[0052] In step S520, the normalized coordinates are iteratively dedistorted according to the distortion parameters to obtain corresponding undistorted normalized coordinates.

[0053] It should be noted that since camera lenses, especially wide-angle or fisheye lenses, inevitably introduce image distortion during the imaging process, the normalized coordinates generated in the previous step are corrected through the dedistortion algorithm to restore the true imaging path direction.

[0054] Specifically, let the camera distortion coefficient be ,in Indicates the The radial distortion coefficients, Indicates the The tangential distortion coefficient.

[0055] From the initial normalized coordinates Starting from, the distortion-free normalized coordinates are estimated iteratively Each iteration includes the following steps: Calculate radial distortion correction , Formula (5) , Formula (6) , Formula (7) Calculate the tangential distortion correction term , formula (8) , formula (9) Update the distorted coordinate model , formula (10) , formula (11) Will As an observation value, Newton iteration or other numerical methods are used to continuously adjust , until the error is less than the set threshold, and finally the normalized coordinates after dedistortion are obtained.

[0056] Therefore, the pixel coordinates are converted into normalized image coordinates using the camera intrinsic parameters. On this basis, an iterative dedistortion calculation is introduced to systematically remove the nonlinear error introduced by lens distortion, thereby obtaining the true line of sight direction under the ideal imaging model.

[0057] In step S530, the sight angle is calculated based on the undistorted normalized coordinates.

[0058] Here, the optical axis direction is defined as the positive direction of the z-axis in the camera coordinate system, and the distortion-free normalized coordinates express the spatial direction from the optical center to the target feature point. The line of sight angle can be obtained by solving the angle between the two direction vectors.

[0059] Specifically, once the undistorted normalized coordinates are obtained , the angle between the feature point's sight line and the optical axis can be calculated based on the directional relationship between the feature point and the optical axis (i.e., the z-axis). The angle can be calculated using the following formula: , formula (12) The angle can be further obtained by inverse solution: , Formula (13) Here, by deriving the three-dimensional angle between the normalized and distortion-free line of sight vector and the optical axis direction, the problem of angle misjudgment caused by ignoring the distortion factor is avoided, and a more realistic mapping relationship between the image plane and the spatial geometry is established.

[0060] Through the embodiments of the present application, a method for calculating the line of sight angle based on the distortion parameters involved in the camera is adopted, which can introduce a compensation mechanism for the actual imaging distortion effect at the image pixel coordinate level, making the spatial direction estimation of the feature point more accurate and reliable.

[0061] In some business application scenarios, the distance between different feature points can be measured by a depth camera. Specifically, the depth information of the reference feature point is obtained, and based on the depth information of the target feature point and the depth information of the reference feature point, combined with the line of sight angle between the target feature point and the reference feature point, the distance between the reference feature point and the target feature point is calculated. Exemplarily, the three-dimensional position of each feature point is obtained by multiplying its undistorted normalized coordinates by the depth information to form a complete three-dimensional point position representation. Subsequently, by calculating the Euclidean distance between the three-dimensional position vectors of the reference feature point and the target feature point, the spatial distance between the two points, that is, the distance between the reference feature point and the target feature point, can be obtained. In this way, the imaging geometry model of the camera and the optical distortion compensation result are fully combined to ensure that the final feature point spacing result can more accurately reflect the spatial relationship in the real physical scene.

[0062] It should be understood that the method of obtaining depth information for reference feature points can be diverse, such as pre-calibration or obtaining through conventional methods (such as no compensation method or fixed compensation method). It can also be obtained through the same dynamic depth compensation measurement method as the target feature point, which is not limited here.

[0063] Since the depth information measured by the uncompensated method or the fixed compensation method has large errors, especially the error will change dynamically as the orientation of the target point changes, the distance calculated using the depth information measured by the uncompensated method or the fixed compensation method further amplifies the error and may seriously deviate from the actual true value of the feature point distance.

[0064] In contrast, by adopting the perspective-based dynamic optical center compensation method provided in the embodiment of the present application, more accurate depth measurement results can be achieved for target feature points or different feature points, thereby reducing depth measurement errors and supporting more accurate feature point spacing measurement results.

[0065] In order to further verify the stability and error control ability of the dynamic optical center compensation method based on the line of sight angle proposed in this application, the following will be combined with Figures 6 to 8 This paper demonstrates the impact of line-of-sight angle measurement errors (i.e., theta1 estimation bias) on depth estimation results under different viewing angles. Specifically, the error levels at 0°, 45°, and 80° are analyzed to verify the error level of dynamic compensation from different angles and poses.

[0066] Figure 6 The figure shows an example of the simulation effect of the deviation err of the depth true value measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the line of sight angle is 0°.

[0067] like Figure 6As shown in the figure, when the true value of the line of sight angle is 0°, if the angle error of the line of sight angle estimated due to dynamic compensation is within the range of ±2°, relative to the true value error of 0°, it can be seen that the error in the [-2,0] interval gradually decreases, and the error in the [0,2] interval gradually increases. The error of the y coordinate shows that the maximum error amplitude is approximately , which is on the order of 0.004 mm.

[0068] Figure 7 The figure shows an example of the simulation effect of the deviation err of the depth true value measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the line of sight angle is 45°.

[0069] like Figure 7 As shown, if the true theta1 is 45°, if the angle error estimated due to dynamic compensation is in the range of ±2°, relative to the true value error of 45°, it can be seen that the error in the [-2,0] interval gradually decreases, and the error in the [0,2] interval gradually increases. The y coordinate shows that the error level is within 0.00016m, that is, within 0.16mm.

[0070] Figure 8 The figure shows an example of the simulation effect of the deviation err of the depth true value measurement result caused by the view angle measurement theta1 error during dynamic compensation when the true value of the line of sight angle is 80°.

[0071] like Figure 8 As shown, if the true theta1 is 80°, if the angle error estimated due to dynamic compensation is in the range of ±2°, relative to the true value error of 80°, it can be seen that the error in the [-2,0] interval gradually decreases, and the error in the [0,2] interval gradually increases. The y coordinate shows that the error level is within 0.00025m, that is, within 0.25mm.

[0072] pass Figures 6 to 8 It can be seen that the impact of the line of sight angle error on the depth solution accuracy tends to increase with the size of the angle, but the dynamic compensation method of the embodiment of the present application has extremely high stability at a small viewing angle, and the error is controlled at the micron level; and at a large viewing angle (such as 80°), although the error increases slightly, it can be maintained at the sub-millimeter level (on the order of 0.1mm), with good angle adaptability and ranging robustness, and can achieve high-precision depth estimation within different viewing angle ranges.

[0073] Figure 9 A flowchart of an example of a method for evaluating camera calibration results according to an embodiment of the present application is shown.

[0074] It should be noted that before a camera is shipped out of the factory, it is usually necessary to evaluate the camera's calibration results to ensure that the spatial measurement data provided by the camera has sufficient accuracy and repeatability, thereby guaranteeing the performance of the camera's depth imaging system in actual applications.

[0075] like Figure 9 As shown, in step S910, the object distance and the captured image for the first feature point are obtained, where the object distance is the distance between the first feature point and the center of the camera lens.

[0076] In step S920, the sight line angle is calculated according to the pixel coordinates of the first feature point in the captured image. The sight line angle is the angle between the straight line passing through the first feature point and the optical center and the optical axis.

[0077] In step S930, the principal point offset between the center of the camera lens and the optical center is obtained.

[0078] In step S940 , the true depth value of the first feature point is calculated based on the triangle theorem, combined with the principal point offset, object distance, and sight angle.

[0079] In step S950, first depth information corresponding to the first feature point is output based on the visual odometry model, and the camera calibration result is evaluated according to the calculated first depth information and the true depth value.

[0080] Here, the visual odometry model can employ a monocular or binocular odometry model to calculate the first depth information of the first feature point through visual depth estimation. Regardless of the visual odometry method employed, the calculated first depth information is an estimate dependent on calibration parameters (particularly the camera intrinsic parameters). Therefore, this first depth information is compared with the true dynamically compensated depth value calculated using the triangle theorem to quantitatively assess the applicability and accuracy of the camera intrinsic parameters within the visual odometry model. This comparison provides a more accurate evaluation mechanism for camera calibration results.

[0081] In some embodiments, the object distance of the first feature point is pre-acquired in a controlled experimental environment using a high-precision measurement device (such as an industrial-grade laser rangefinder, a standard three-dimensional position calibration system, or a high-precision multi-camera array), and true value compensation is performed in combination with the above-mentioned dynamic compensation method to serve as an absolute benchmark for evaluation reference.

[0082] Through the embodiments of the present application, the depth true value calculated based on dynamic compensation is introduced into the calibration evaluation process, which reduces the difficulty of measuring the depth true value while improving the accuracy of the depth true value. Compared with the traditional depth true value measurement method without compensation or fixed compensation, a spatial true value distance with higher geometric accuracy can be achieved. This distance is used as an input parameter for camera calibration evaluation to evaluate the calibration results, thereby achieving more targeted geometric error control and system accuracy improvement.

[0083] Specifically, during camera imaging, the position of the target feature point in the image affects its relative orientation to the optical center. Ignoring the geometric changes introduced by the line-of-sight angle can easily lead to systematic deviations in the true depth value at different viewing angles, thereby affecting the accuracy of the camera calibration parameter assessment results. This embodiment calculates the angle between the feature point's line of sight and the optical axis in real time, and dynamically adjusts the principal point offset compensation based on this angle. This effectively incorporates the structural errors caused by changes in imaging angle into the true depth value compensation model, making the depth result of each feature point closer to its actual spatial position.

[0084] This provides a more accurate depth truth, which not only improves the accuracy of recovering the spatial coordinates of feature points during the calibration process, but also enhances the model's error control capabilities within the entire field of view, significantly improving the evaluation accuracy of the camera intrinsic calibration results.

[0085] As described in the above embodiment, single-point depth measurement without compensation may introduce a maximum measurement error of 5-7mm. By incorporating this error into the calculation of the "true value measurement site error" in the camera calibration evaluation process, its impact on the overall site reconstruction accuracy evaluation can be analyzed. For example, in a typical ranging scenario, the distance between multiple feature points or reference structures is 3.33 meters. If the overall ranging accuracy requirement of the system is 0.15%, the upper limit of the allowable error is: , formula (14) According to engineering practice, it is usually necessary to divide the total error tolerance into multiple system components and algorithm modules. Assume that the single-point depth error budget should not exceed 1 / 5 of the tolerance band, which is about 1 mm. If the dynamic true value measurement method based on viewing angle provided by this application is not adopted, the actual depth error introduced will reach 5~7 mm, which will exceed the target error control range by 5~7 times. In order to ensure the resolution of the test equipment itself, the error of the measuring equipment itself increases, which requires the distance between multiple feature points or reference structures to increase. For every 1mm increase in the measurement error, the corresponding spatial distance will be required to increase by 3.33m, and the cumulative spatial distance will increase to: , formula (15) Therefore, if the depth estimation method without dynamic compensation is used, the single-point measurement error alone can lead to large error in the result evaluation, and the required measurement scene area is too large and the calibration cost is high.

[0086] In one example of an embodiment of the present application, the depth true value is the depth true value between the first feature point and the optical center, and then the true value deviation information between the calculated first depth information and the depth true value is determined, and the camera calibration result is evaluated based on the true value deviation information.

[0087] In contrast, the depth measurement method proposed in the embodiment of the present application corrects the offset caused by the difference in image viewing angle and internal structure by introducing the line of sight angle and principal point offset parameters in real time, so that the depth measurement method can be solved. It is closer to the real geometric path, ensuring the physical consistency and spatial accuracy of the final depth value, thereby effectively suppressing the transmission of system errors, and demonstrating stronger accuracy assurance and robustness in calibration evaluation and precision ranging tasks.

[0088] Specifically, by comparing the calculated first depth information with the true depth value, the difference between the two or a relative error index is usually used to quantify the difference, and the evaluation method adopted can be diverse. In one example, the evaluation is performed by judging the absolute error threshold. If the depth difference between the two exceeds the preset error tolerance, the calibration result is judged to have a deviation. In another example, the evaluation is performed by the root mean square error statistics, and the systematic deviation trend is obtained based on the error accumulation evaluation of multiple points. In another example, the error consistency at different feature points is judged by multi-point discreteness evaluation, reflecting the stability of the calibration model within the field of view.

[0089] In another example of the embodiment of the present application, the true depth value is the true value distance between the first feature point and the second feature point, and the camera calibration result is evaluated by triangulation.

[0090] Figure 10 An operational flowchart of an example of evaluating camera calibration results through triangulation according to an embodiment of the present application is shown.

[0091] like Figure 10 As shown, in step S1010, second depth information for a second feature point is obtained.

[0092] Regarding the depth information of the second feature point, it can be obtained based on a dynamic compensation depth measurement method, or the true depth value of the second feature point can be directly determined by other methods, such as outputting the second depth information corresponding to the second feature point based on a visual ranging model that has passed calibration evaluation, and all of these fall within the scope of implementation of the embodiments of the present application.

[0093] In step S1020 , a feature point distance between the first feature point and the second feature point is calculated based on the first depth information and the second depth information.

[0094] Specifically, based on the pixel coordinates and corresponding depths of the two feature points, they are projected into the three-dimensional camera coordinate system to construct two complete spatial position vectors, and then the Euclidean distance between the two three-dimensional coordinate points is used as the spatial spacing of the feature point pair (i.e., feature point spacing).

[0095] In step S1030, the true value error information of the distance between the calculated feature point distance and the true value distance is determined, and the camera calibration result is evaluated according to the true value error of the distance.

[0096] Specifically, the calculated feature point spacing is compared with its preset or measured "true spacing," and by analyzing the difference between the two, "spacing error information" is obtained, reflecting the camera calibration accuracy. It should be understood that the true spacing can be provided by a high-precision distance measurement system, a standard physical fixture, or an experimental calibration environment, and is characterized by high reliability and controllable errors.

[0097] Through the embodiments of the present application, the true value of the feature point spacing is used as the basis for calibration evaluation. Compared to using the true value of a single-point depth, the measurement error of the evaluation benchmark itself can be significantly reduced. This is because in actual measurement, the distance between two points can usually be stably obtained using high-precision tools (such as laser interferometers and precision jigs), and the measurement error and difficulty are far less than the measurement error and difficulty of a single feature point relative to the optical center. As a result, the true value of the spacing has higher reliability and repeatability. When used as a comparison benchmark for calibration results, it can effectively reduce the uncertainty of error sources, thereby improving the accuracy and credibility of the overall evaluation.

[0098] Figure 11 A schematic diagram of the optical principle effect of an example of evaluating camera calibration results based on triangulation is shown.

[0099] like Figure 11 As shown in the figure, the two target feature points P1 and P2 and the camera optical center O together form a spatial triangle. Based on the triangular geometric relationship, the system can use the known lengths of the two sides OP1 and OP2 and the angle ∠P1OP2 to estimate the distance P1P2 between the two feature points using the cosine theorem, thereby indirectly evaluating the accuracy of depth measurement and camera calibration. Specifically, the relationship between the triangle side lengths can be expressed as: , formula (16) In this measurement model, OP1 and OP2 represent the spatial distances from the camera's optical center, O, to the two feature points, P1 and P2, respectively. ∠P1OP2 is the angle between the two feature points and the optical center, calculated from the angle between the corresponding vectors of the normalized coordinates in the image. P1P2 is the final estimate of the feature point spacing, which can be compared with the known physical distance to determine the measurement accuracy of OP1 and OP2 and the accuracy of the derived calibration results.

[0100] Figure 12 The figure shows a simulation effect diagram of an example of the influence of OP1 error on P1P2 ranging under a fixed viewing angle.

[0101] The figure shows that under a fixed line of sight angle of 45°, single-point depth errors (such as inaccurate OP1 distance measurement) will be structurally amplified, ultimately affecting the accuracy of multi-point distance calculations. In practical applications, if OP1 is not dynamically compensated for the optical center, this error transmission path will lead to unreliable P1P2 estimates, especially in calibration or spatial modeling tasks, causing a decrease in system accuracy. Therefore, combined with Figure 12 The results can further verify that the use of a dynamic compensation mechanism to correct the error of OP1 can help significantly reduce the error propagation effect in the estimation of the distance between key feature points, thereby ensuring the spatial consistency and accuracy stability of the overall ranging system.

[0102] To further illustrate the impact of depth measurement error on the final distance estimation, assume that ∠P1OP2 is a fixed value of 45°, and the true values ​​of OP1 and OP2 are both 10 meters. Under ideal conditions, substituting the formula into the calculation yields: , formula (17) Assume that due to measurement error, OP1 exists If the deviation of OP1 is -7mm to +7mm, and OP2 keeps the true value of 10 meters, the measured value of OP1 may be Within this error range, the estimated error range of P1P2 can be calculated by substituting the cosine theorem: When OP1 = 10.007 meters, , formula (18) When OP1 = 9.993 meters, , formula (19) As can be seen, the maximum error in the P1P2 estimation is approximately ±0.0026 meters, or 2.6 millimeters. This error is only introduced by the measurement error of OP1, and when the angle is fixed, the error transmission is stable and controllable.

[0103] In addition, for 3D measurement tasks involving multi-point distance accumulation in real-world scenarios, if there are multiple feature point pairs with distances ranging from several meters (e.g., 3.33 meters) and the overall system ranging accuracy requirement is 0.15%, the acceptable measurement error for the device is: , formula (20) Considering the safety control range within the actual measurement error tolerance band (such as controlling the error budget to 1 / 5 of the allowable range), the single measurement error should be controlled within 1 mm. At this time, if the traditional depth measurement method without compensation or fixed compensation is adopted, it is easy to produce a larger error when the feature point angle deviates, causing the final P1P2 distance estimation to deviate from the upper limit. However, the dynamic optical center compensation method based on the line of sight angle and the principal point offset provided in the embodiment of the present application is used to measure the depth of OP1 and OP2. This can effectively suppress the error transmission caused by the change in viewing angle, ensure that the depth measurement value is more accurate and close to the actual physical distance, and thus make the feature point spacing estimated by triangulation more reliable.

[0104] The triangulation ranging method provided by the embodiments of this application provides a stable and geometrically consistent evaluation mechanism for calibration accuracy. Combined with a depth measurement method using dynamic optical center compensation, this significantly reduces the amplification effect of single-point errors on overall ranging results, improving the system's overall precision control capabilities and calibration verification reliability.

[0105] It should be noted that, considering the overall accuracy requirement of the ranging system is 0.15%, the error tolerance should be controlled at 1 / 5 of the target site reference distance. Taking a typical ranging site length of 3.33m as an example, when using the above-mentioned uncompensated or fixed-compensated depth measurement method and introducing a true value error of 2-7mm, in order to meet the error tolerance, the equivalent error tolerance distance range needs to be increased: , formula (21) Therefore, a larger site space is needed to meet the ranging accuracy requirements, which significantly increases deployment costs and equipment requirements.

[0106] In contrast, the camera optical center depth true value measurement method based on relative posture dynamic compensation provided in the embodiment of the present application introduces the angle theta1 between the feature point and the optical axis, combines the camera internal parameters to accurately estimate the current imaging line of sight direction, and constructs a dynamic geometric compensation model from the lens center to the optical center. It can adapt to the position changes of the feature point in the field of view in real time and realize dynamic correction of the optical center offset.

[0107] Regarding the key variable angle theta1 of the measured application, it is the angle between the normalized light direction vector and the optical axis calculated based on the camera calibration result and pixel coordinate transformation. Figure 6-8 Simulation analysis shows that even with an angle estimation error of ±2°, the maximum depth error introduced is only 0.25mm, which is much lower than the millimeter-level error level of traditional methods.

[0108] Under the dynamic compensation method, the length of the site only needs to be increased as follows: , formula (22) In this way, while maintaining the accuracy requirements, the requirements for the length of the test site are significantly reduced, effectively improving the space utilization efficiency of the measurement system, and helping to improve the overall measurement resolution and equipment deployment flexibility.

[0109] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of combined actions, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0110] In some embodiments, the embodiments of the present application also provide a depth measurement device, including: an acquisition unit, used to acquire the object distance and captured image for a target feature point; the object distance is the distance between the target feature point and the center of the camera lens; a line of sight angle calculation unit, used to calculate the line of sight angle based on the pixel coordinates of the target feature point in the captured image; the line of sight angle is the angle between the straight line passing through the target feature point and the optical center and the optical axis; a principal point offset acquisition unit, used to acquire the principal point offset between the center of the camera lens and the optical center; a depth calculation unit, used to calculate the depth information of the target feature point based on the triangle theorem, combined with the principal point offset, the object distance and the line of sight angle.

[0111] In some embodiments, the embodiments of the present application also provide a device for evaluating camera calibration results, including: an acquisition unit, used to acquire an object distance and a captured image for a first feature point; the object distance is the distance between the first feature point and the center of the camera lens; a line of sight angle calculation unit, used to calculate the line of sight angle based on the pixel coordinates of the first feature point in the captured image; the line of sight angle is the angle between the straight line passing through the first feature point and the optical center and the optical axis; a principal point offset acquisition unit, used to acquire the principal point offset between the center of the camera lens and the optical center; a depth calculation unit, used to calculate the true depth value of the first feature point based on the triangle theorem, combined with the principal point offset, the object distance and the line of sight angle; a camera calibration evaluation unit, used to output first depth information corresponding to the first feature point based on a visual ranging model, and evaluate the camera calibration result based on the first depth information and the true depth value.

[0112] The specific implementation process of the above-mentioned device can be referred to the content of the aforementioned method embodiment, which will not be repeated here.

[0113] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned depth measurement methods or camera calibration result evaluation methods.

[0114] In some embodiments, the embodiments of the present application also provide a storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by electronic devices (including but not limited to computers, servers, or network devices, etc.) to execute any of the depth measurement methods or camera calibration result evaluation methods described above.

[0115] In some embodiments, embodiments of the present application further provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a depth measurement method or a camera calibration result evaluation method. The electronic device may include a camera or be connected to a camera.

[0116] The apparatus of the embodiment of the present application described above can be used to implement the depth measurement method or the camera calibration result evaluation method of the embodiment of the present application, and accordingly achieve the technical effects achieved by the depth measurement method or the camera calibration result evaluation method of the embodiment of the present application, which will not be described in detail here. In the embodiment of the present application, the relevant functional modules can be implemented by a hardware processor.

[0117] Figure 13 This is a hardware structure diagram of an electronic device for performing a depth measurement method or a camera calibration result evaluation method provided in another embodiment of the present application, such as Figure 13 As shown, the device includes: One or more processors 1310 and memory 1320, Figure 13 A processor 1310 is taken as an example.

[0118] The device for executing the depth measurement method or the camera calibration result evaluation method may further include: an input device 1330 and an output device 1340 .

[0119] The processor 1310, the memory 1320, the input device 1330 and the output device 1340 may be connected via a bus or other means. Figure 13 The bus connection is taken as an example.

[0120] Memory 1320, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the depth measurement method or the camera calibration result evaluation method in the embodiments of the present application. Processor 1310 executes the non-volatile software programs, instructions, and modules stored in memory 1320 to execute various server functional applications and data processing, thereby implementing the depth measurement method or camera calibration result evaluation method in the above-mentioned method embodiments.

[0121] The memory 1320 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the device, etc. In addition, the memory 1320 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1320 may optionally include a memory remotely located relative to the processor 1310, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The input device 1330 may receive input digital or character information and generate signals related to user settings and function control of the device. The output device 1340 may include a display device such as a display screen.

[0123] The one or more modules are stored in the memory 1320 , and when executed by the one or more processors 1310 , perform the depth measurement method or camera calibration result evaluation method in any of the above method embodiments.

[0124] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0125] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0126] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0127] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0128] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0129] (5) Other electronic devices with data interaction functions.

[0130] In some embodiments, this application further provides a mobile platform equipped with the computer device described in any embodiment of this application. Mobile platforms include, but are not limited to, vehicles, tracked robots, bipedal robots, quadrupedal robots, etc., where the vehicles may be passenger cars, pickup trucks, and vans. It should be noted that the above are merely examples, and this application does not limit the specific form of the mobile platform.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0132] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A depth measurement method, comprising: Obtaining the object distance and collected images for target feature points; The object distance is the distance between the target feature point and the center of the camera lens; Calculating a sight angle according to the pixel coordinates of the target feature point in the captured image; the sight angle is the angle between a straight line passing through the target feature point and the optical center and the optical axis; Obtaining a principal point offset between the center of the camera lens and the optical center; According to the triangle theorem, the depth information of the target feature point is calculated by combining the principal point offset, the object distance and the sight angle.

2. The method according to claim 1, wherein The calculating the sight angle according to the pixel coordinates of the target feature point in the acquired image includes: According to the camera intrinsic parameters, the pixel coordinates are converted into normalized coordinates in the camera coordinate system; Performing iterative dedistortion processing on the normalized coordinates according to the distortion parameters to obtain corresponding undistorted normalized coordinates; The sight angle is calculated according to the undistorted normalized coordinates.

3. The method according to claim 1, wherein The obtaining of the principal point offset between the center of the camera lens and the optical center includes: The principal point offset between the center of the camera lens and the optical center is determined according to one or more of a measurement distance, an entrance pupil distance, and a viewpoint depth; the measurement distance is a measurement distance between the center of the camera lens and the optical center measured by optical measurement equipment.

4. The method according to claim 1, further comprising: Obtain depth information of reference feature points; The distance between the reference feature point and the target feature point is calculated according to the depth information of the target feature point and the depth information of the reference feature point and in combination with the sight angle between the target feature point and the reference feature point.

5. A method for evaluating camera calibration results, comprising: Obtaining an object distance and a captured image for a first feature point; The object distance is the distance between the first feature point and the center of the camera lens; Calculating a sight angle according to the pixel coordinates of the first feature point in the captured image; the sight angle is the angle between a straight line passing through the first feature point and the optical center and the optical axis; Obtaining a principal point offset between the center of the camera lens and the optical center; Calculate the true depth value of the first feature point based on the triangle theorem and the principal point offset, the object distance, and the sight angle; First depth information corresponding to the first feature point is output based on a visual ranging model, and a camera calibration result is evaluated according to the first depth information and the true depth value.

6. The method according to claim 5, wherein: The depth truth value is the dynamic compensation depth truth value between the first feature point and the optical center, The evaluating the camera calibration result according to the first depth information and the true depth value includes: Determine true value deviation information between the first depth information and the dynamic compensation depth true value, and evaluate a camera calibration result according to the true value deviation information.

7. The method according to claim 5, wherein: The depth truth value is the distance between the first feature point and the second feature point. The evaluating the camera calibration result according to the first depth information and the true depth value includes: Acquire second depth information for the second feature point; Calculating a feature point distance between the first feature point and the second feature point according to the first depth information and the second depth information; Determine the true value error information of the distance between the calculated feature point distance and the true value distance, and evaluate the camera calibration result according to the true value error of the distance.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, wherein: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, wherein: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images