Monocular laser speckle projection system's external parameter calibration method and related device

Abstract

The application discloses a monocular laser speckle projection system external parameter calibration method and related device. The monocular laser speckle projection system external parameter calibration method calculates a spatial curved surface equation by using a curved surface calibration board, and calculates three-dimensional coordinates of homonymous speckle points according to the spatial curved surface equation; then, a light center and an optical axis position of the laser speckle projector are estimated according to the three-dimensional coordinates of the homonymous speckle points; finally, a pose relationship between the camera and the laser speckle projector is calculated according to a preset laser speckle projector coordinate system, a homonymous speckle straight line and a projector virtual image plane intersection point are calculated by using the pose relationship, and a virtual speckle image of the laser speckle projector is generated, so that the monocular laser speckle projection system external parameter calibration is realized. The monocular laser speckle projection system external parameter calibration method is beneficial to significantly improving the measurement efficiency of the external parameter calibration, and can significantly improve the measurement accuracy.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method and apparatus for calibrating the extrinsic parameters of a monocular laser speckle projection system. Background Technology

[0002] High-precision depth measurement is one of the important research topics in the field of computer vision. Traditional depth measurement methods mainly include time-of-flight (ToF) methods and binocular stereo vision methods.

[0003] The Time-of-Flight (ToF) method obtains the depth information of a target by measuring the time of flight or phase transformation of modulated light. ToF methods typically have advantages such as being less affected by ambient light, high measurement speed, and the ability to perform long-distance measurements. However, the measurement accuracy of ToF methods is only at the centimeter level, which cannot meet the requirements of some high-precision measurement tasks.

[0004] Binocular stereo vision methods acquire disparity maps of target regions by matching image pairs captured by two cameras at different locations, thereby obtaining depth information of the target. This method typically uses block matching or semi-global matching algorithms to search for similar regions in image pairs, achieving sub-pixel level matching accuracy. However, because binocular stereo vision methods perform image matching based on visual features, they encounter difficulties in scenes with significant changes in ambient light or lack of texture features, leading to large matching errors or even matching failures. Furthermore, the massive computational cost of image feature extraction and matching limits its application in real-time measurement. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and apparatus for calibrating the external parameters of a monocular laser speckle projection system.

[0006] In a first aspect, embodiments of this application provide a method for calibrating the extrinsic parameters of a monocular laser speckle projection system, the method including:

[0007] Acquire camera calibration images using a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector;

[0008] The camera parameters are calibrated based on the camera calibration image;

[0009] Speckle images were acquired using the monocular laser speckle projection system.

[0010] Based on the camera parameters, the characteristic features of the speckle image are extracted, and the spatial surface equation of the calibration plate used in the monocular laser speckle projection system is calculated. The calibration plate has at least three characteristic features.

[0011] Obtain speckle spots with the same name from the speckle image;

[0012] Calculate the three-dimensional coordinates of the corresponding speckle based on the spatial surface equation of the calibration plate; estimate the optical center and optical axis position of the laser speckle projector based on the three-dimensional coordinates of the corresponding speckle.

[0013] The pose relationship between the camera and the laser speckle projector is calculated based on the preset laser speckle projector coordinate system.

[0014] The intersection point of the corresponding speckle line and the virtual image plane of the projector is calculated based on the pose relationship to generate the virtual image of the laser speckle projector.

[0015] In some possible implementations, the marking feature is a diagonal marking, the calibration plate is a spherical calibration plate, and the calculation of the spatial surface equation of the calibration plate includes:

[0016] A diagonal mark is placed in the central area of ​​the calibration plate to define two mutually perpendicular closed curves with an arc length equal to the circumference of the complete sphere passing through the diagonal mark. On the finite-area spherical calibration plate, several diagonal marks are arranged along each of the two closed curves, with at least three diagonal marks on the spherical calibration plate. A corresponding world coordinate system is established on the calibration plate, with the diagonal mark in the central area as the origin. The x-axis and y-axis extend tangent to the closed curves, and the z-axis extends outward perpendicular to the spherical calibration plate. Let L be the arc length between the diagonal mark on the closed curve corresponding to the positive x-axis and the central diagonal mark, and r be the radius of the spherical calibration plate. The three-dimensional coordinates (X, Y, Z) of the diagonal mark in the world coordinate system are given. w ,Y w Z w ) is represented as:

[0017]

[0018] Wherein, the pixel coordinates (u,v) of the diagonal marker and the three-dimensional coordinates (X,V) of the world coordinate system w ,Y w Z w The relationship between ) is expressed as:

[0019]

[0020] Among them, K c Let R be the intrinsic parameter matrix of the camera, s be the scaling factor, R be the rotation matrix, and T be the translation vector, where R... 3×2 Includes only the elements from the first and third columns of R;

[0021] Among them, the three-dimensional coordinates (X, Y, Z) of the diagonal marker in the camera coordinate system c,Y c Z c ):

[0022]

[0023] The spatial spherical equation of the calibration plate in the camera coordinate system is obtained by fitting using the least squares method.

[0024] In some possible implementations, calculating the three-dimensional coordinates of the corresponding speckle points includes:

[0025] Based on the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X, V) of the speckle image point in the camera coordinate system are calculated. c ,Y c Z c )for:

[0026]

[0027] Among them, (C) x C y ) represents the principal point coordinates of the camera, d x d represents the pixel dimension in the x-direction. y represents the pixel dimension in the y-direction, and f represents the camera focal length;

[0028] The equation of the spatial straight line between the speckle image points and the origin of the camera coordinate system is expressed as:

[0029]

[0030] Based on the spatial straight line equation and the spatial surface equation of the calibration plate, the corresponding three-dimensional coordinate set including the three-dimensional coordinates of the same-named speckles is calculated.

[0031] In some possible implementations, calculating the pose relationship between the camera and the laser speckle projector includes: the normalized direction vector of the laser speckle projector's optical axis in the camera coordinate system is V. c =[v x ,v y ,v z ] T Wherein, in the coordinate system of the laser speckle projector, it is represented as V p =[0,0,1] T The relationship between the two can be expressed as:

[0032]

[0033] Calculate A x and A y :

[0034]

[0035] Select a preset A z The value is used to calculate the rotation matrix R, where A x A y A z The Euler angles are the rotation matrix R;

[0036] The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x... p ,y p ,z p The optical center of the laser speckle projector has coordinates (0,0,0) in the laser speckle projector coordinate system, and the relationship between the two can be expressed as:

[0037]

[0038] The translation vector T is calculated, where,

[0039] In some possible implementations, generating the virtual image of the laser speckle projector includes: calculating the intersection points of all corresponding speckle lines with the virtual image plane of the laser speckle projector, and converting the intersection points into image point coordinates, thereby generating the virtual image of the laser speckle projector.

[0040] In some possible implementations, the method further includes: after generating the virtual image of the laser speckle projector, using the laser speckle projector as a second camera, the second camera forming a binocular camera system together with the camera in the monocular laser speckle projection system.

[0041] Secondly, embodiments of this application also provide an external parameter calibration device for a monocular laser speckle projection system, comprising:

[0042] A camera calibration image acquisition module is used to acquire camera calibration images under a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector.

[0043] A camera parameter calibration module is used to calibrate camera parameters based on the camera calibration image.

[0044] A speckle image acquisition module is used to acquire speckle images under the monocular laser speckle projection system;

[0045] The spatial surface equation calculation module is used to extract the marker features of the speckle image based on the camera parameters and calculate the spatial surface equation of the calibration plate used by the monocular laser speckle projection system. The calibration plate has at least three marker features.

[0046] A speckle image acquisition module is used to acquire speckles with the same name based on the speckle image.

[0047] The corresponding speckle calculation module is used to calculate the three-dimensional coordinates of the corresponding speckles based on the spatial surface equation of the calibration plate;

[0048] The optical center and optical axis estimation module is used to estimate the optical center and optical axis positions of the laser speckle projector based on the three-dimensional coordinates of the speckle with the same name.

[0049] The pose relationship calculation module is used to calculate the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system.

[0050] The virtual speckle image generation module is used to calculate the intersection point of the corresponding speckle line and the virtual image plane of the projector based on the pose relationship, and generate the virtual image of the laser speckle projector.

[0051] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores computer-readable instructions executable on the processor, and the processor, when executing the computer-readable instructions, can perform the steps of the extrinsic parameter calibration method for the monocular laser speckle projection system as described in the first aspect.

[0052] Fourthly, this application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a processor, can implement the steps of the extrinsic parameter calibration method for a monocular laser speckle projection system as described in any of the first aspects.

[0053] In this embodiment, a calibration plate is used to calculate the equation of a spatial surface, and the three-dimensional coordinates of the corresponding speckle lines are calculated based on this equation. Then, the optical center and optical axis positions of the laser speckle projector are estimated based on the three-dimensional coordinates of the speckle lines. Finally, the pose relationship between the camera and the laser speckle projector is calculated based on a preset laser speckle projector coordinate system. This pose relationship is used to calculate the intersection of the corresponding speckle lines and the virtual image plane of the projector, generating a virtual image of the laser speckle projector, thereby achieving the external parameter calibration of the monocular laser speckle projection system. This application eliminates the need for precise ranging devices to capture corresponding speckle images at different standard distances, significantly improving the measurement efficiency and accuracy of external parameter calibration. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic flowchart of an external parameter calibration method for a monocular laser speckle projection system provided in an embodiment of this application.

[0056] Figure 2 This is a schematic diagram of the calibration plate in the embodiments of this application.

[0057] Figure 3 This is a schematic diagram of the optical axis and optical center of the laser speckle projector in the embodiments of this application.

[0058] Figure 4 This is a schematic diagram of an external parameter calibration device for a monocular laser speckle projection system provided in an embodiment of this application.

[0059] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0060] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0061] It should be clearly stated that the described embodiments are merely some, not all, of the embodiments described in this application. All other embodiments obtained by those skilled in the art based on the embodiments described in this application without inventive effort are within the scope of protection of this application.

[0062] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application.

[0063] For example, the singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0064] It should be understood that the term "and / or" used in this article is merely a description of the same field in the related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0065] It should be understood that although terms such as first, second, third, etc., may be used to describe preset ranges in the embodiments of this application, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from one another. For example, without departing from the scope of the embodiments of this application, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0066] Furthermore, depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)," etc.

[0067] To overcome the limitations of Time-of-Flight (ToF) and binocular stereo vision methods, some solutions have proposed projection systems based on laser speckle images. In a laser speckle projector, an infrared laser emits light that passes through a diffraction grating (e.g., frosted glass) to form a highly random speckle image. This system uses an infrared camera to capture the speckle image of the target surface, significantly reducing the impact of ambient light on the measurement. Furthermore, based on the randomness of the speckle image, the measurement time of the laser speckle projection system can be shortened to the time of a single exposure, thus enabling real-time dynamic measurement.

[0068] Based on the number of cameras in a speckle projection system, they can be divided into two categories: binocular speckle systems and monocular speckle systems. A binocular laser speckle projection system is equivalent to a binocular stereo vision system with speckle images. The highly random speckle images provide rich feature information to textureless regions, significantly improving the image matching and measurement accuracy of the binocular stereo vision system. However, binocular speckle systems are more expensive to manufacture, and the system calibration process is more complex. A monocular laser speckle projection system contains only one infrared camera and one laser speckle projector, making it more compact and less expensive.

[0069] Because laser speckle projectors lack a standard speckle image, and the speckle image distorts to varying degrees as the distance between the projector and the target increases, monocular laser speckle projection systems require high-precision rangefinders to capture corresponding speckle images at different standard distances before leaving the factory or returning for repair. This makes the external parameter calibration of conventional monocular laser speckle projection systems relatively complex and cumbersome. The following examples will explore and discuss how to solve the problem of overly complex and cumbersome external parameter calibration for conventional monocular laser speckle projection systems.

[0070] See Figure 1 , Figure 1 This is a flowchart illustrating an external parameter calibration method for a monocular laser speckle projection system according to an embodiment of this application. Figure 1 As shown, the external parameter calibration method for this monocular laser speckle projection system specifically includes the following steps:

[0071] S10: Acquire camera calibration images using a monocular laser speckle projection system, which includes a camera and a laser speckle projector.

[0072] In one embodiment, an infrared camera and a laser speckle projector are fixed at a suitable angle to a tripod to form a monocular laser speckle projection system. Then, a appropriately sized checkerboard calibration board is selected and placed within the camera's field of view. The camera is then turned on and the focus is adjusted to capture an image of the checkerboard calibration board. During this process, the position and orientation of the checkerboard calibration board need to be continuously adjusted.

[0073] S20: Calibrate camera parameters based on camera calibration images.

[0074] In one embodiment, the camera's intrinsic parameters, including focal length and principal point coordinates, as well as radial and tangential distortion, are calculated using the Zhang Zhengyou calibration method based on the set of checkerboard calibration board images captured by the camera.

[0075] S30: Equipped with a calibration plate, which acquires speckle images under a monocular laser speckle projection system, wherein the calibration plate has at least three marker features.

[0076] The calibration plate includes at least three marking features, which are set on the spherical calibration plate in the form of markings or specific features. This application does not limit the marking features.

[0077] Specifically, the calibration plate can be marked with diagonal markings to facilitate the calculation of the spatial spherical equation of the calibration plate. Understandably, three or more markings can be set on the calibration plate, and the more markings set (to aid calculation and verification), the more effective the calibration of the monocular laser speckle projection system's extrinsic parameters will be, thereby improving the measurement accuracy of the monocular laser speckle projection system.

[0078] In one embodiment, several diagonal marks are pre-printed. First, a diagonal mark is placed in the central area of ​​the calibration plate, defining two mutually perpendicular closed curves passing through this mark, each with an arc length equal to the circumference of a complete sphere. Then, on a spherical calibration plate with a limited area, several diagonal marks are arranged along each of the two closed curves, resulting in a spherical calibration plate with diagonal marks. The calibration plate is placed within the field of view of a monocular laser speckle system, the laser speckle projector is activated, and speckle images are projected onto the surface of the calibration plate. The corresponding speckle images are then recorded by a camera. During this process, the position and orientation of the calibration plate need to be continuously adjusted.

[0079] S40: Extract the signature features of the speckle image based on the camera parameters and calculate the spatial spherical equation of the calibration plate.

[0080] In one embodiment, the marking features in the calibration plate may specifically be diagonal markings, that is, markings set at diagonal positions on the calibration plate.

[0081] In one embodiment, the distortion of the speckle image can be corrected based on the camera distortion coefficients (including radial and tangential distortion) obtained in S20. The Harris corner detection algorithm is used to identify diagonal markers in the speckle image and extract them, thereby calculating the spatial spherical equation of the calibration plate based on the diagonal markers.

[0082] Furthermore, a corresponding world coordinate system is established on the calibration plate, with the diagonal mark of the central region as the origin of the coordinate system. The x-axis and y-axis extend along the tangent direction of the closed curve mentioned above, and the z-axis extends outward perpendicular to the spherical calibration plate. Figure 2 This is a schematic diagram of the calibration plate in an embodiment of this application. Figure 2 As can be seen, there are 5 diagonal marks on the calibration plate. The world coordinate system established based on these diagonal marks is as follows: Figure 2 The coordinate axes are shown in the diagram.

[0083] Let L be the arc length between the diagonal mark and the central diagonal mark on the closed curve corresponding to the positive x-axis, and let r be the radius of the spherical calibration plate. Let the three-dimensional coordinates (X, Y, R) of the diagonal mark in the world coordinate system be... w ,Y w Z w This can be represented as:

[0084]

[0085] Following the above process, the three-dimensional coordinates of all diagonal markers in the world coordinate system can be calculated. The pixel coordinates (u,v) of the diagonal markers and their three-dimensional coordinates (X,V) in the world coordinate system are then compared. w ,Y w Z w The relationship between them can be expressed as:

[0086]

[0087] Where K c Let be the intrinsic parameter matrix of the camera, s be the scaling factor, R be the rotation matrix, T be the translation vector, and R be the rotation vector. 3×2 It contains only the elements from the first and third columns of R.

[0088] This allows us to calculate the three-dimensional coordinates (X, Y, Z) of the diagonal marker in the camera coordinate system. c ,Y c Z c):

[0089]

[0090] The equation of the spatial sphere of the calibration plate in the camera coordinate system is obtained by fitting using the least squares method.

[0091] S50: Obtain speckle spots with the same name from the speckle image.

[0092] In one embodiment, the first speckle image is used as a reference image for speckle matching. The digital image correlation method is used to determine the optimal matching position of the speckle image points by solving the displacement shape function containing first-order and second-order displacement gradient parameters, thereby obtaining the speckle image's corresponding speckle set.

[0093] S60: Calculate the three-dimensional coordinates of the corresponding speckle points based on the spatial spherical equation of the calibration plate.

[0094] Understandably, in a lossless imaging model, light rays entering the camera must pass through the camera's optical center, i.e., the origin of the camera coordinate system. After obtaining the spatial spherical equation of the calibration plate, the three-dimensional coordinates of the corresponding speckle pattern can be calculated based on this equation. Understandably, this corresponding speckle pattern represents the three-dimensional coordinates on the calibration plate, which are the intersection points of the speckle rays emitted by the laser speckle projector and the spherical calibration plate.

[0095] Specifically, based on the speckle image point coordinates (u,v) and camera parameters, the equivalent three-dimensional coordinates (X,V) of the speckle image point in the camera coordinate system are calculated. c ,Y c Z c )for:

[0096]

[0097] Among them (C) x C y ) represents the principal point coordinates of the camera, d x d represents the pixel dimension in the x-direction. y Let f represent the pixel dimension in the y-direction and f be the camera focal length; the equation of the spatial straight line passing through the speckle image point and the origin of the camera coordinate system can be expressed as:

[0098]

[0099] Based on the equation of a straight line in space and the equation of a spherical surface in space of the calibration plate, the three-dimensional coordinates of the corresponding speckles are calculated, and the three-dimensional coordinate set corresponding to the speckles is obtained.

[0100] S70: Estimate the optical center and optical axis position of the laser speckle projector based on the three-dimensional coordinates of the speckle with the same name.

[0101] In one embodiment, during the manufacturing process of the laser speckle projector, the optical axis of the laser speckle projector is designed to be strictly perpendicular to the diffraction grating and pass through its center. Therefore, the center point of the speckle image on the calibration plate at different positions and orientations is located on or near the optical axis of the laser speckle projector. Based on the three-dimensional coordinate set corresponding to the center point of the speckle image obtained in step S60, the optical axis position of the laser speckle projector can be determined by straight line fitting. Furthermore, the straight line fitted by the same speckle points corresponds to the light rays emitted from the laser speckle projector, where the light source point passing through the laser is the optical center of the laser speckle projector. Understandably, under the influence of factors such as camera calibration error, image matching error, and fitting error, the fitted straight line set will not intersect at a single point, but will exhibit varying degrees of offset. Therefore, the spatial point closest to the fitted straight line set is calculated and used as the optimal optical center of the projector.

[0102] Figure 3 This is a schematic diagram of the optical axis and optical center of the laser speckle projector in an embodiment of this application. From Figure 3 The physical spatial relationship between the speckle, optical axis and (laser speckle) projector on the calibration plate can be seen.

[0103] S80: Calculate the pose relationship between the camera and the laser speckle projector based on the preset laser speckle projector coordinate system.

[0104] In one embodiment, a laser speckle projector coordinate system can be established based on the obtained optical center and optical axis positions of the laser speckle projector, and the pose relationship between the camera and the laser speckle projector can be calculated in the laser speckle projector coordinate system.

[0105] Specifically, the laser speckle projector coordinate system has its origin at the laser speckle projector's optical center, with the z-axis coinciding with the laser speckle projector's optical axis, and the direction facing the target as the positive direction. The normalized direction vector of the laser speckle projector's optical axis in the camera coordinate system is V. c =[v x ,v y ,v z ] T In the laser speckle projector coordinate system, it is represented as V p =[0,0,1] T The relationship between the two can be expressed as:

[0106]

[0107] Where A x A y A z For Euler angles of the rotation matrix, it should be noted that the rotation order of Euler angles can be xyz, or other rotation orders are also acceptable.

[0108] The above formula can be simplified to:

[0109]

[0110] A can be calculated x and A y :

[0111]

[0112] A z The directions of the x and y axes of the laser speckle projector coordinate system, as well as the speckle coordinates of the speckle image, are determined. Since the optical center and optical axis positions of the laser speckle projector are fixed, the absolute physical position of the speckle image within the laser speckle projector is fixed and independent of Euler angles. To ensure that most or all of the speckle image can be located within the virtual image of the laser speckle projector, a suitable A is selected. z The value is obtained, and the rotation matrix R is calculated.

[0113] The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x... p ,y p ,z p The coordinates of the given information in the laser speckle projector coordinate system are (0,0,0), and the relationship between them can be expressed as:

[0114]

[0115] Then the translation vector can be obtained.

[0116] S90: Calculate the intersection point of the same speckle line and the virtual image plane of the projector based on the pose relationship, and generate the virtual image of the laser speckle projector.

[0117] Specifically, let the coordinates of the speckle on the calibration plate in the camera coordinate system be (x1, y1, z1). Based on the pose relationship between the camera and the laser speckle projector, its coordinates in the laser speckle projector coordinate system can be calculated as (x2, y2, z2):

[0118]

[0119] Following the above process, the three-dimensional coordinate set of the same speckle set on the calibration plate at different poses is obtained in the coordinate system of the laser speckle projector, and the same speckle line is plotted. Assume that the same speckle line passes through the point (a1, a2, a3), and its direction vector v... L For (v1, v2, v3), the equation of the straight line in space can be expressed as:

[0120]

[0121] Where (X) L ,Y L Z L Let be any point on the same speckle line, and t be the scaling factor.

[0122] Since the virtual image plane is perpendicular to the z-axis of the laser speckle projector coordinate system, the intersection point (X) of the corresponding speckle line and the image plane can be calculated. P ,Y P Z P ):

[0123]

[0124] Based on the intrinsic parameter matrix of the laser speckle projector, the image point coordinates (u, v) of the intersection point in the virtual image are:

[0125]

[0126] Calculate the intersection points of all corresponding speckle lines with the virtual image plane of the laser speckle projector, and convert them into virtual image point coordinates to generate a virtual image of the laser speckle projector.

[0127] This application establishes a connection between the camera and the laser speckle projector using a spatial sphere, and generates a virtual speckle image corresponding to the laser speckle projector. Furthermore, after reconstructing the virtual speckle image, the monocular laser speckle projection system in this application is equivalent to a binocular camera system, and can perform stereoscopic correction and online calibration of the camera and the laser speckle projector (which can recover the virtual speckle image, equivalent to the function of a camera) using binocular stereo vision.

[0128] Understandably, traditional monocular laser speckle projection systems require precise ranging devices to capture corresponding speckle images at different standard distances for further extrinsic parameter calibration. However, the monocular laser speckle projection system of this application can calculate the pose relationship between the camera and the laser speckle projector using a calibration plate with marked features. This allows the laser speckle projector to reconstruct a virtual speckle image, giving the monocular laser speckle projection system capabilities equivalent to binocular stereo vision. The dual-view constraint method can improve image matching speed, thereby increasing measurement accuracy and reducing calibration complexity. Furthermore, if the measurement accuracy decreases due to prolonged use, the user can quickly recalibrate the extrinsic parameters of the monocular laser speckle system without needing to return it to the factory for recalibration.

[0129] In this embodiment, compared to traditional monocular laser speckle projection systems, this application uses a calibration plate to calibrate the extrinsic parameters of the monocular laser speckle projection system, eliminating the need to use a precise ranging device to capture corresponding speckle images at different standard distances. This significantly improves measurement efficiency and reduces measurement costs. The monocular laser speckle projection system in this application is equivalent to a binocular camera system with speckle images, improving measurement accuracy. Furthermore, this application can also calibrate the optical center and optical axis positions of the laser speckle projector, allowing users to correct optical axis misalignment that occurs during the use of the monocular laser speckle projection system online.

[0130] It should be noted that, besides the spherical calibration plate used in this application for calibrating the extrinsic parameters of a monocular laser speckle projection system, other types of curved surface calibration plates can also be used. Specifically, referring to this application, for other types of curved surface calibration plates, the spatial surface equation of each surface can be calculated through the marking features on the surface. Then, by changing the position of the surface, corresponding speckle points are found, and the three-dimensional coordinates of the corresponding speckle points are further calculated. This allows estimation of the optical center and optical axis position of the laser speckle projector, determination of the intersection points of all corresponding speckle lines with the virtual image plane of the laser speckle projector, and conversion of these into virtual image point coordinates. Finally, a virtual image of the laser speckle projector is generated. This monocular laser speckle projection system using a curved surface calibration plate for extrinsic parameter calibration has binocular stereo vision capabilities equivalent to a binocular camera system, and can conveniently and efficiently complete the extrinsic parameter calibration. It should be understood that calibration methods implemented using other types of curved surface calibration plates should also be included within the scope of protection of this application.

[0131] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0132] See Figure 4 , Figure 4 This application provides an external parameter calibration device for a monocular laser speckle projection system. The device includes:

[0133] The camera calibration image acquisition module 410 is used to acquire camera calibration images under a monocular laser speckle projection system, which includes a camera and a laser speckle projector.

[0134] Camera parameter calibration module 420 is used to calibrate camera parameters based on camera calibration images;

[0135] The speckle image acquisition module 430 is used to acquire speckle images under a monocular laser speckle projection system;

[0136] The spatial surface equation calculation module 440 is used to extract the marker features of the speckle image based on the camera parameters and calculate the spatial surface equation of the calibration plate of the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marker features.

[0137] The same-name speckle acquisition module 450 is used to acquire same-name speckles based on the speckle image;

[0138] The corresponding speckle calculation module 460 is used to calculate the three-dimensional coordinates of the corresponding speckles based on the spatial surface equation of the calibration plate.

[0139] The optical center and optical axis estimation module 470 is used to estimate the position of the optical center and optical axis of the laser speckle projector based on the three-dimensional coordinates of the speckle with the same name.

[0140] The pose relationship calculation module 480 is used to calculate the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system.

[0141] The virtual speckle image generation module 490 is used to calculate the intersection point of the same speckle line and the virtual image plane of the projector according to the pose relationship, and generate the virtual image of the laser speckle projector.

[0142] It is understood that the implementation of the relevant functions of the external parameter calibration device of the monocular laser speckle projection system in this embodiment can refer to the above method embodiment, and the parts not described in detail can refer to the relevant descriptions in the above method embodiment.

[0143] In this embodiment, compared to traditional monocular laser speckle projection systems, this application uses a calibration plate to calibrate the extrinsic parameters of the monocular laser speckle projection system, eliminating the need to use a precise ranging device to capture corresponding speckle images at different standard distances. This significantly improves measurement efficiency and reduces measurement costs. The monocular laser speckle projection system in this application is equivalent to a binocular camera system with speckle images, improving measurement accuracy. Furthermore, this application can also calibrate the optical center and optical axis positions of the laser speckle projector, allowing users to correct optical axis misalignment that occurs during the use of the monocular laser speckle projection system online.

[0144] See Figure 5 , Figure 5 This application provides a schematic diagram of a computer device, which includes a memory 510 and a processor 520. Computer-readable instructions are stored in the memory and can be executed on the processor. When the processor executes the computer-readable instructions, it performs the steps of the external parameter calibration method of the monocular laser speckle projection system as described in the embodiment.

[0145] This application provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the external parameter calibration method for a monocular laser speckle projection system as described in the embodiments.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0147] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for calibrating the external parameters of a monocular laser speckle projection system, characterized in that, include: Acquire camera calibration images using a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector; The camera parameters are calibrated based on the camera calibration image; Speckle images were acquired using the monocular laser speckle projection system. Based on the camera parameters, the characteristic features of the speckle image are extracted, and the spatial surface equation of the calibration plate used in the monocular laser speckle projection system is calculated. The calibration plate has at least three characteristic features. Obtain speckle spots with the same name from the speckle image; Calculate the three-dimensional coordinates of the corresponding speckle based on the spatial surface equation of the calibration plate; The optical center and optical axis positions of the laser speckle projector are estimated based on the three-dimensional coordinates of the speckle with the same name. The pose relationship between the camera and the laser speckle projector is calculated based on the preset laser speckle projector coordinate system. The intersection point of the same speckle line and the virtual image plane of the projector is calculated based on the pose relationship to generate the virtual image of the laser speckle projector. in, The calculation of the three-dimensional coordinates of the corresponding speckle points includes: Based on the coordinates of the speckle points Based on the camera parameters, the equivalent three-dimensional coordinates of the speckle image points in the camera coordinate system are calculated. for: in, Indicates the principal point coordinates of the camera. Indicates the pixel size in the x-direction. represents the pixel dimension in the y-direction, and f represents the camera focal length; The equation of the spatial straight line between the speckle image points and the origin of the camera coordinate system is expressed as: Based on the spatial straight line equation and the spatial surface equation of the calibration plate, the corresponding three-dimensional coordinate set including the three-dimensional coordinates of the same-named speckles is calculated; The calculation of the pose relationship between the camera and the laser speckle projector includes: the normalized direction vector of the laser speckle projector's optical axis in the camera coordinate system is... Wherein, in the coordinate system of the laser speckle projector, it is represented as The relationship between the two can be expressed as: Calculate and : Select a preset The value is used to calculate the rotation matrix R, where, The Euler angles are the rotation matrix R; The coordinates of the optical center of the laser speckle projector in the camera coordinate system are: The coordinates of the optical center of the laser speckle projector in the laser speckle projector coordinate system are: The relationship between the two can be expressed as: The translation vector T is calculated, where, .

2. The method according to claim 1, characterized in that, The marking feature is a diagonal marking, the calibration plate is a spherical calibration plate, and the calculation of the spatial surface equation of the calibration plate includes: A diagonal mark is placed in the central region of the calibration plate to define two mutually perpendicular closed curves with an arc length equal to the circumference of the complete sphere. On the spherical calibration plate with a finite area, several diagonal marks are arranged along each of the two closed curves, with at least three diagonal marks on the spherical calibration plate. A corresponding world coordinate system is established on the calibration plate, with the diagonal mark in the central region as the origin. The x-axis and y-axis extend tangent to the closed curves, and the z-axis extends outward perpendicular to the spherical calibration plate. Let L be the arc length between the diagonal mark on the closed curve corresponding to the positive x-axis and the central diagonal mark, and let r be the radius of the spherical calibration plate. The three-dimensional coordinates of the diagonal mark in the world coordinate system are... Represented as: Wherein, the pixel coordinates of the diagonal marker Three-dimensional coordinates of the world coordinate system The relationship between them is represented as follows: in, Let be the intrinsic parameter matrix of the camera, s be the scaling factor, R be the rotation matrix, and T be the translation vector, where Includes only the elements from the first and third columns of R; Wherein, the three-dimensional coordinates of the diagonal markers in the camera coordinate system : The spatial spherical equation of the calibration plate in the camera coordinate system is obtained by fitting using the least squares method.

3. The method according to any one of claims 1 to 2, characterized in that, The step of generating the virtual image of the laser speckle projector includes: calculating the intersection points of all speckle lines with the same name with the virtual image plane of the laser speckle projector, and converting the intersection points into image point coordinates, thereby generating the virtual image of the laser speckle projector.

4. The method according to any one of claims 1-2, characterized in that, The method further includes: after generating the virtual image of the laser speckle projector, using the laser speckle projector as a second camera, wherein the second camera and the camera in the monocular laser speckle projection system form a binocular camera system.

5. A device for calibrating external parameters of a monocular laser speckle projection system, characterized in that, include: A camera calibration image acquisition module is used to acquire camera calibration images under a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector. A camera parameter calibration module is used to calibrate camera parameters based on the camera calibration image. A speckle image acquisition module is used to acquire speckle images under the monocular laser speckle projection system; The spatial surface equation calculation module is used to extract the marker features of the speckle image based on the camera parameters and calculate the spatial surface equation of the calibration plate used by the monocular laser speckle projection system. The calibration plate has at least three marker features. A speckle image acquisition module is used to acquire speckles with the same name based on the speckle image. The corresponding speckle calculation module is used to calculate the three-dimensional coordinates of the corresponding speckles based on the spatial surface equation of the calibration plate; The optical center and optical axis estimation module is used to estimate the optical center and optical axis positions of the laser speckle projector based on the three-dimensional coordinates of the speckle with the same name. The pose relationship calculation module is used to calculate the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system. The virtual speckle image generation module is used to calculate the intersection point of the same speckle line and the virtual image plane of the projector according to the pose relationship, and generate the virtual image of the laser speckle projector. in, The calculation of the three-dimensional coordinates of the corresponding speckle points includes: Based on the coordinates of the speckle points Based on the camera parameters, the equivalent three-dimensional coordinates of the speckle image points in the camera coordinate system are calculated. for: in, Indicates the principal point coordinates of the camera. Indicates the pixel size in the x-direction. represents the pixel dimension in the y-direction, and f represents the camera focal length; The equation of the spatial straight line between the speckle image points and the origin of the camera coordinate system is expressed as: Based on the spatial straight line equation and the spatial surface equation of the calibration plate, the corresponding three-dimensional coordinate set including the three-dimensional coordinates of the same-named speckles is calculated; The calculation of the pose relationship between the camera and the laser speckle projector includes: the normalized direction vector of the laser speckle projector's optical axis in the camera coordinate system is... Wherein, in the coordinate system of the laser speckle projector, it is represented as The relationship between the two can be expressed as: Calculate and : Select a preset The value is used to calculate the rotation matrix R, where, The Euler angles are the rotation matrix R; The coordinates of the optical center of the laser speckle projector in the camera coordinate system are: The coordinates of the optical center of the laser speckle projector in the laser speckle projector coordinate system are: The relationship between the two can be expressed as: The translation vector T is calculated, where, .

6. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions executable on the processor, characterized in that, When the processor executes the computer-readable instructions, it performs the steps of the extrinsic parameter calibration method for the monocular laser speckle projection system as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product enables the processor to implement the method as described in any one of claims 1 to 4.

Images