External parameter calibration method and device for monocular laser speckle projection system

Abstract

The present application discloses a method and device for calibrating the extrinsic parameters of a monocular laser speckle projection system. The extrinsic parameter calibration method of the monocular laser speckle projection system uses a calibration plate to calculate a spatial plane equation, and calculates the three-dimensional coordinates of the same-name speckle image points according to the spatial plane equation; then estimates the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the same-name speckle image points; finally, according to a preset laser speckle projector coordinate system, calculates the posture relationship between the camera and the laser speckle projector, and can use the posture relationship to generate a plane homography matrix between the camera image and the laser speckle projector virtual image, thereby realizing the extrinsic parameter calibration of the monocular laser speckle projection system. The extrinsic parameter calibration method of the monocular laser speckle projection system can significantly improve the measurement efficiency of the extrinsic parameter calibration, and can significantly improve the measurement accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method and device for calibrating external parameters of a monocular laser speckle projection system. Background Art

[0002] High-precision depth measurement is one of the important research topics in the field of computer vision. Traditional depth measurement methods mainly include Time-of-Flight (ToF) and binocular stereo vision methods. ToF obtains the depth information of the target by measuring the flight time or phase change of modulated light. It has the advantages of being less affected by ambient light, fast measurement speed, and being able to achieve long-distance measurement. However, the measurement accuracy of ToF can only reach the centimeter level, which cannot meet the requirements of some high-precision measurement tasks.

[0003] The binocular stereo vision method obtains the disparity map of the target area by matching the image pairs taken by two cameras at different positions, thereby obtaining the depth information of the target to be measured. This method usually uses block matching or semi-global matching algorithms to search for similar areas of the image pairs, which can achieve sub-pixel matching accuracy. Since the binocular stereo vision method performs image matching based on visual features, it will be difficult to match scenes with obvious changes in ambient light or lack of texture features, resulting in large matching errors or even matching failures. In addition, the huge amount of computation caused by the image feature extraction and matching process limits its application in the field of real-time measurement.

[0004] In order to overcome the defects of the above-mentioned ToF and binocular stereo vision methods, researchers invented a projection system based on laser speckle images. The laser in the laser speckle projector emits infrared laser light, which passes through the diffraction grating (ground glass) to form a highly random speckle image. The system uses an infrared camera to capture the speckle image of the surface of the target to be measured, which greatly reduces the impact of ambient light on the measurement. At the same time, based on the randomness of the speckle image, the measurement time of the laser speckle projection system can be shortened to a single exposure time, thereby realizing real-time dynamic measurement.

[0005] According to the number of cameras in the speckle projection system, it can be divided into two categories: binocular speckle system and monocular speckle system. The binocular laser speckle projection system is equivalent to a binocular stereo vision system with speckle images. The highly random speckle image gives rich feature information to the texture-free area, which significantly improves the image matching accuracy and measurement accuracy of the binocular stereo vision system. However, the manufacturing cost of the binocular speckle system is high, and the system calibration steps are relatively complicated. The monocular laser speckle projection system only contains an infrared camera and a laser speckle projector, and its system is more compact and has a lower cost.

[0006] Since there is no standard speckle image for the laser speckle projector, and the speckle image will be deformed to varying degrees as the distance between the projector and the target increases, the monocular laser speckle projection system needs to use a high-precision ranging instrument to capture the corresponding speckle images in advance at different standard distances before leaving the factory or after being returned to the factory for repair. This makes the external parameter calibration of the monocular laser speckle projection system complicated and cumbersome. Summary of the invention

[0007] In view of this, an embodiment of the present application provides a method and device for calibrating the extrinsic parameters of a monocular laser speckle projection system, so as to solve the problem that the extrinsic parameter calibration of a monocular laser speckle projection system in the prior art is too complicated and cumbersome.

[0008] In a first aspect, an embodiment of the present application provides a method for calibrating external parameters of a monocular laser speckle projection system, the method comprising:

[0009] Collecting a camera calibration image under a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector;

[0010] Calibrate camera parameters according to the camera calibration image;

[0011] A calibration plate is prepared to collect speckle images under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

[0012] Extracting the signature features of the speckle image according to the camera parameters, and calculating the spatial plane equation of the calibration plate;

[0013] Acquire speckle image points with the same name according to the speckle image;

[0014] Calculating the three-dimensional coordinates of the speckle image points with the same name according to the spatial plane equation of the calibration plate;

[0015] Estimate the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name;

[0016] Calculating the position relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system;

[0017] The plane homography matrix between the camera image and the laser speckle projector virtual image is calculated according to the posture relationship to generate the laser speckle projector virtual image.

[0018] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the mark feature is a diagonal mark, and the step of calculating the spatial plane equation of the calibration plate includes:

[0019] A corresponding world coordinate system is established on the calibration plate, with the diagonal mark on the upper left as the origin of the coordinate system, the x-axis passes vertically downward through the diagonal mark on the lower left, the y-axis passes horizontally to the right through the diagonal mark on the upper right, and the z-axis is perpendicular to the calibration plate and outward, wherein the coordinate of the diagonal mark in the world coordinate system is expressed as (X w ,Y w , 1), the pixel coordinates (u, v) of the diagonal mark and the three-dimensional coordinates (X w ,Y w ,1) is expressed as:

[0020]

[0021] Among them, K c is the intrinsic parameter matrix of the camera, s is the scale factor, R is the rotation matrix, and T is the translation vector; the coordinates (X c ,Y c ,Z c ):

[0022]

[0023] The spatial plane equation of the calibration plate in the camera coordinate system is obtained by least square fitting.

[0024] According to the above aspect and any possible implementation manner, an implementation manner is further provided, wherein the calculating the three-dimensional coordinates of the speckle image points with the same name comprises:

[0025] According to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X c ,Y c ,Z c )for:

[0026]

[0027] Among them (C x ,C y ) is the principal point coordinate of the camera, d x and d y are the physical sizes of the pixel in the x-axis and y-axis directions, respectively, and f is the focal length of the camera;

[0028] The equation of the space line passing through the speckle image point and the origin of the camera coordinate system can be expressed as:

[0029]

[0030] The three-dimensional coordinates of the speckle image points with the same name are calculated according to the spatial straight line equation and the spatial plane equation of the calibration plate to obtain a three-dimensional coordinate set corresponding to the speckle image points with the same name.

[0031] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the calculating the posture relationship between the camera and the laser speckle projector includes:

[0032] Among them, the normalized direction vector of the laser speckle projector optical axis in the camera coordinate system is V c =[v x ,v y ,v z ] T , which is represented as V in the laser speckle projector coordinate system p =[0,0,1] T , the relationship between the two can be expressed as:

[0033]

[0034] Calculate A x and A y :

[0035] Select a preset A z Value, calculate the rotation matrix R, where A x , A y , A z is the Euler angle of the rotation matrix R;

[0036] The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x p ,y p ,z p ), the coordinate in the laser speckle projector coordinate system is (0,0,0), and the relationship between the two can be expressed as:

[0037]

[0038] Get the translation vector T:

[0039]

[0040] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the calculating of the plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the posture relationship comprises:

[0041] Assume that the coordinates of the speckle image point in the camera coordinate system are (x1, y1, z1) and in the laser speckle projector coordinate system are (x2, y2, z2). The relationship between the two can be expressed as:

[0042]

[0043] According to the fact that the speckle image point is located on the surface of the calibration plate, the coordinates of the speckle image point in the camera coordinate system satisfy the following equation:

[0044]

[0045] Wherein, n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate;

[0046] Combining the above two equations, we get:

[0047]

[0048] Then the plane homography matrix between the camera image and the projector virtual image is expressed as:

[0049]

[0050] Among them, K c and K p is the intrinsic parameter matrix of the camera and the laser speckle projector, and K p =K c .

[0051] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein generating the laser speckle projector virtual image comprises:

[0052] According to the plane homography matrix between the camera image and the laser speckle projector virtual image, the camera image is mapped to the virtual projector plane to obtain the laser speckle projector virtual image.

[0053] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein after the virtual image of the laser speckle projector is generated, the laser speckle projector is used as a second camera to form a binocular camera system together with the camera in the monocular laser speckle projection system.

[0054] In a second aspect, an embodiment of the present application provides an external parameter calibration device for a monocular laser speckle projection system, the device comprising:

[0055] A camera calibration image acquisition module, used for acquiring a camera calibration image under a monocular laser speckle projection system, wherein the monocular laser speckle projection system comprises a camera and a laser speckle projector;

[0056] A camera parameter calibration module, used to calibrate camera parameters according to the camera calibration image;

[0057] A speckle image acquisition module, which is provided with a calibration plate and is used to acquire a speckle image under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

[0058] A space plane equation calculation module, used to extract the signature features of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate;

[0059] A module for acquiring speckle image points with the same name, used for acquiring speckle image points with the same name according to the speckle image;

[0060] A same-name speckle image point calculation module, used for calculating the three-dimensional coordinates of the same-name speckle image points according to the spatial plane equation of the calibration plate;

[0061] An optical center and optical axis estimation module, used for estimating the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name;

[0062] A posture relationship calculation module, used for calculating the posture relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system;

[0063] The virtual speckle image generation module is used to calculate the plane homography matrix between the camera image and the laser speckle projector virtual image according to the posture relationship, so as to generate the laser speckle projector virtual image.

[0064] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the processor executes the computer-readable instructions, the processor performs the steps of the method for calibrating external parameters of a monocular laser speckle projection system as described in the first aspect.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the method for calibrating external parameters of a monocular laser speckle projection system as described in any one of the first aspects are implemented.

[0066] In the embodiment of the present application, the present application uses a calibration plate to calculate a spatial plane equation, and calculates the three-dimensional coordinates of the same-name speckle image points according to the spatial plane equation; then estimates the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the same-name speckle image points; finally, according to the preset laser speckle projector coordinate system, calculates the posture relationship between the camera and the laser speckle projector, and can use the posture relationship to generate a plane homography matrix between the camera image and the laser speckle projector virtual image, thereby realizing the external parameter calibration of the monocular laser speckle projection system. The present application does not need to use an accurate distance measuring device to shoot the corresponding speckle image at different standard distances, which can significantly improve the measurement efficiency of the external parameter calibration and the measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0068] Figure 1 It is a flow chart of an external parameter calibration method of a monocular laser speckle projection system in an embodiment of the present application;

[0069] Figure 2 is a schematic diagram of a calibration plate in an embodiment of the present application;

[0070] Figure 3 It is a schematic diagram of the optical axis and optical center of a laser speckle projector in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0072] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0073] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0074] It should be understood that the term "and / or" used in this article is only a description of the same field of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0075] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present application, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present application, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0076] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0077] Figure 1 FIG. 1 is a flow chart of a method for calibrating external parameters of a monocular laser speckle projection system in an embodiment of the present application. Figure 1 As shown, the external parameter calibration method of the monocular laser speckle projection system specifically includes the following steps:

[0078] S10: collecting a camera calibration image in a monocular laser speckle projection system, where the monocular laser speckle projection system includes a camera and a laser speckle projector.

[0079] In one embodiment, an infrared camera and a laser speckle projector are fixed on a tripod at a suitable angle to form a monocular laser speckle projection system. Afterwards, a checkerboard calibration plate of a suitable size is selected and placed within the camera field of view, and the camera is turned on and the focus is adjusted to shoot the checkerboard calibration plate image. During this period, the position and posture of the checkerboard calibration plate need to be continuously adjusted.

[0080] S20: Calibrate camera parameters according to the camera calibration image.

[0081] In one embodiment, the Zhang Zhengyou calibration method is used to calculate the intrinsic parameters of the camera, including the focal length and the principal point coordinates, as well as the radial distortion and the tangential distortion, according to the checkerboard calibration plate image set captured by the camera.

[0082] S30: preparing a calibration plate, and collecting a speckle image under a monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features.

[0083] The calibration plate includes at least three marking features, which are arranged on the plane of the calibration plate in the form of markings or specific features. The present application does not limit the marking features.

[0084] Specifically, a diagonal mark can be used to set a mark feature on the calibration plate that can be used to calculate the spatial plane equation of the calibration plate. It can be understood that three or more mark features can be set on the calibration plate, and the more they are set (to help calculation and verification), the more helpful it is to improve the accuracy of the external parameter calibration of the monocular laser speckle projection system, so as to improve the measurement accuracy of the monocular laser speckle projection system.

[0085] In one embodiment, a diagonal mark calibration paper is printed with a diagonal mark on each of the four corners and the rest of the positions are blank, and the diagonal mark calibration paper is placed closely on a flat plate to obtain a calibration plate with diagonal marks. The calibration plate is placed within the field of view of the monocular laser speckle system, the laser speckle projector is turned on, a speckle image is projected onto the surface of the calibration plate, and the corresponding speckle image is recorded with a camera. During this period, the position and posture of the calibration plate need to be continuously adjusted.

[0086] S40: extracting the signature features of the speckle image according to the camera parameters, and calculating the spatial plane equation of the calibration plate.

[0087] In one embodiment, the marking feature in the calibration plate may specifically be a diagonal mark, that is, a mark set at a diagonal position of the calibration plate.

[0088] In one embodiment, the distortion of the speckle image can be corrected according to the camera distortion coefficient (including radial distortion and tangential distortion) obtained in S20. The Harris corner point detection algorithm is used to identify the diagonal marks in the speckle image, and the diagonal marks are extracted, so as to calculate the spatial plane equation of the calibration plate according to the diagonal marks.

[0089] Furthermore, a corresponding world coordinate system can be established on the calibration plate, with the upper left diagonal mark as the origin of the coordinate system, the x-axis vertically downward through the lower left diagonal mark, the y-axis horizontally to the right through the upper right diagonal mark, and the z-axis perpendicular to the calibration plate outward. Figure 2 is a schematic diagram of a calibration board in an embodiment of the present application, Figure 2 It can be seen that there is a pair of diagonal marks on each of the four corners of the calibration plate. The world coordinate system established based on the diagonal marks is as follows: Figure 2 The coordinate axes are shown in .

[0090] It can be understood that since the physical distance between the diagonal marks is known, the coordinates of the diagonal marks in the world coordinate system are known, and the coordinates of the diagonal marks in the world coordinate system are expressed as (X w ,Y w,1), the pixel coordinates (u,v) of the diagonal mark and the three-dimensional coordinates (X w ,Y w ,1) is expressed as:

[0091]

[0092] Among them, K c is the camera's intrinsic parameter matrix, s is the scale factor, R is the rotation matrix, T is the translation vector, R and T describe the pose relationship between the camera coordinate system and the world coordinate system; and the four sets of diagonal mark coordinate pairs are sufficient to solve R and T, then the coordinates of the diagonal mark on the calibration plate surface in the camera coordinate system (X c ,Y c ,Z c ):

[0093]

[0094] Finally, the least squares method is used to fit the spatial plane equation of the calibration plate in the camera coordinate system.

[0095] S50: Acquire speckle image points with the same name according to the speckle image.

[0096] In one embodiment, the first speckle image can be used as a reference image for speckle matching, and a digital image correlation method is used to determine the best matching position of the speckle image points by solving a displacement shape function including first-order and second-order displacement gradient parameters, thereby obtaining a set of speckle image points with the same name in the speckle image.

[0097] S60: Calculate the three-dimensional coordinates of the speckle image points with the same name according to the spatial plane equation of the calibration plate.

[0098] It can be understood that in the distortion-free camera imaging model, the light entering the camera must pass through the optical center of the camera, that is, the origin of the camera coordinate system. After obtaining the spatial plane equation of the calibration plate, the three-dimensional coordinates of the same-name speckle image point can be calculated according to the spatial plane equation. It can be understood that the same-name speckle image point is the three-dimensional coordinate on the calibration plate plane, and the three-dimensional coordinate is the intersection of the speckle ray emitted by the laser speckle projector and the calibration plate plane.

[0099] Specifically, according to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X c ,Y c ,Z c )for:

[0100]

[0101] Among them (C x ,C y) is the principal point coordinate of the camera, d x and d y are the physical dimensions of the pixel in the x-axis and y-axis directions, respectively, and f is the focal length of the camera; the equation of the spatial straight line passing through the speckle image point and the origin of the camera coordinate system can be expressed as:

[0102]

[0103] According to the spatial straight line equation and the spatial plane equation of the calibration plate, the three-dimensional coordinates of the speckle image points with the same name are calculated to obtain the three-dimensional coordinate set corresponding to the speckle image points with the same name.

[0104] S70: Estimate the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name.

[0105] In one embodiment, during the manufacturing process of the laser speckle projector, the optical axis of the laser speckle projector is designed to be strictly perpendicular to the diffraction grating and pass through its center position. Therefore, the center points of the speckle images on the calibration plate at different positions and postures are all located on the optical axis of the laser speckle projector or in the vicinity. According to the three-dimensional coordinate set corresponding to the center point of the speckle image obtained in step S60, the position of the optical axis of the laser speckle projector can be determined by straight line fitting. In addition, the straight line fitted by the speckle spots of the same name corresponds to the light emitted by the laser speckle projector, wherein the light source point passing through the laser is the optical center of the laser speckle projector. It can be understood that under the influence of factors such as camera calibration error, image matching error, and fitting error, the set of fitted straight lines will not intersect at one point, but will be offset to varying degrees. Therefore, the spatial point closest to the set of fitted straight lines is calculated and used as the optimal optical center of the projector.

[0106] Figure 3 Schematic diagram of the optical axis and optical center of a laser speckle projector in an embodiment of the present application. Figure 3 The physical spatial relationship between the same-named speckle spots, the optical axis and the (laser speckle) projector on the calibration plate can be seen.

[0107] S80: Calculate the position relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system.

[0108] In one embodiment, a laser speckle projector coordinate system may be established according to the obtained positions of the optical center and optical axis of the laser speckle projector, and the positional relationship between the camera and the laser speckle projector may be calculated in the laser speckle projector coordinate system.

[0109] Specifically, the laser speckle projector coordinate system takes the optical center of the laser speckle projector as the origin, the z-axis coincides with the optical axis of the laser speckle projector, and the direction facing the target is the positive direction. The normalized direction vector of the optical axis of the laser speckle projector in the camera coordinate system is V c =[vx ,v y ,v z ] T , which is expressed as V in the laser speckle projector coordinate system p =[0,0,1] T , the relationship between the two can be expressed as:

[0110]

[0111] Among them A x , A y , A z is the Euler angle of the rotation matrix. It should be noted that the rotation order of the Euler angle can adopt the rotation order of xyz, and the Euler angle of other rotation orders is also possible.

[0112] The above formula can be simplified as:

[0113]

[0114] A can be calculated x and A y :

[0115] A z The direction of the x and y axes of the laser speckle projector coordinate system and the speckle coordinates of the speckle image are determined. Since the optical center and optical axis position of the laser speckle projector have been determined, the absolute physical position of the speckle image in the laser speckle projector is fixed and has nothing to do with the Euler angle. Under the condition that most or the entire speckle image can be in the virtual image of the laser speckle projector, select a suitable A z Value, and calculate the rotation matrix R.

[0116] The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x p ,y p ,z p ), the coordinate in the laser speckle projector coordinate system is (0,0,0), and the relationship between the two can be expressed as:

[0117]

[0118] Then we can get the translation vector:

[0119]

[0120] S90: Calculate a plane homography matrix between the camera image and the laser speckle projector virtual image according to the posture relationship, and generate the laser speckle projector virtual image.

[0121] Specifically, the coordinates of the speckle image point in the camera coordinate system are (x1, y1, z1), and in the laser speckle projector coordinate system are (x2, y2, z2). The relationship between the two can be expressed as:

[0122]

[0123] Where R and T are the rotation matrix and translation vector calculated by S80, respectively, which describe the position relationship between the camera coordinate system and the laser speckle projector coordinate system. Since the speckle image points are located on the surface of the calibration plate, the coordinates of the speckle image points in the camera coordinate system satisfy the following equation:

[0124]

[0125] Where n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate;

[0126] Combining the above two equations, we get:

[0127]

[0128] Then the plane homography matrix between the camera image and the projector virtual image is expressed as:

[0129]

[0130] Among them, K c and K p is the intrinsic parameter matrix of the camera and laser speckle projector, and K p =K c Since the speckle image is generated by laser light passing through a diffraction grating without passing through a lens group, the distortion of the projector is not considered.

[0131] In specific implementation, a calibration plate is arranged on the target to be measured, the laser speckle projector projects the speckle image onto the surface of the calibration plate, and the camera captures the corresponding speckle image. By calculating the spatial plane equation of the calibration plate, the plane homography matrix H can be obtained. It can be understood that with the plane homography matrix H, the image captured by the camera can be converted to the laser speckle projector, so that the monocular laser speckle projection system of the present application has the binocular stereoscopic vision capability equivalent to that of the binocular camera system, and the calibration of the external parameters can be completed conveniently and efficiently.

[0132] Furthermore, in step S90, i.e., the step of generating a laser speckle projector virtual image, it specifically includes: mapping the camera image to the virtual projector plane to obtain the laser speckle projector virtual image according to a plane homography matrix between the camera image and the laser speckle projector virtual image.

[0133] In one embodiment, the plane homography matrix describes the mapping relationship between points in the same plane between different images, and the plane may be the plane where the calibration plate is located or the plane where the corresponding image of the target to be measured is located. It can be understood that when a binocular camera is used for shooting, the two cameras will obtain images of different shooting angles on the plane, and the plane homography matrix describes the mapping relationship between the images of different shooting angles. Combined with the scheme of the present application, the present application establishes a connection between the camera and the laser speckle projector by means of a spatial plane, and restores the virtual speckle image corresponding to the laser speckle projector through the plane homography matrix. Furthermore, after restoring the virtual speckle image, the monocular laser speckle projection system in the present application is equivalent to a binocular camera system, and the camera and the laser speckle projector (the virtual speckle image can be restored, which is equivalent to the role of the camera) can be stereoscopically corrected, online calibrated, and other operations can be performed using binocular stereo vision.

[0134] It can be understood that the traditional monocular laser speckle projection system needs to use a precise distance measuring device to capture the corresponding speckle images at different standard distances, so as to further complete the external parameter calibration. The monocular laser speckle projection system in the present application can calculate the plane homography matrix of the image conversion relationship between the camera and the laser speckle projector through a calibration plate with marking features, so that the laser speckle projector can restore the virtual speckle image, so that the monocular laser speckle projection system in the present application has the ability equivalent to binocular stereo vision, and can improve the image matching speed by means of dual-view constraints, thereby improving the measurement accuracy and reducing the complexity of calibration; further, if the measurement accuracy of the equipment decreases due to long-term use, the user can re-calibrate the external parameters of the monocular laser speckle system quickly without returning to the factory for re-calibration.

[0135] In the embodiment of the present application, compared with the traditional monocular laser speckle projection system, the present application realizes the external parameter calibration of the monocular laser speckle projection system through the calibration plate, and does not need to use an accurate distance measuring device to shoot the corresponding speckle image at different standard distances. This significantly improves the measurement efficiency and reduces the measurement cost. The monocular laser speckle projection system in the present application is equivalent to a binocular camera system with a speckle image, which improves the measurement accuracy. In addition, the present application can also calibrate the optical center and optical axis position of the laser speckle projector, so that the user can correct the optical axis offset of the monocular laser speckle projection system during use online.

[0136] It should be noted that, in addition to the use of a single plane calibration plate to realize the external parameter calibration of the monocular laser speckle projection system in this application, a stereoscopic calibration piece composed of multiple planes can also be used to realize the external parameter calibration of the monocular laser speckle projection system. Specifically, referring to the present application, for a stereoscopic calibration piece including multiple planes, the spatial plane equation of each plane can be calculated by the mark features on the plane, and then the same-name speckle image points can be found by changing the position of the plane, and the three-dimensional coordinates of the same-name speckle image points can be further calculated to estimate the optical center and optical axis position of the laser speckle projector, determine the homography matrix of different planes of the camera and the laser speckle projector, and finally generate a virtual image of the laser speckle projector according to these homography matrices. The monocular laser speckle projection system that uses a stereoscopic calibration piece composed of multiple planes for external parameter calibration has a binocular stereoscopic vision capability equivalent to that of a binocular camera system, and can complete the calibration of external parameters conveniently and efficiently. It should be understood that other calibration methods implemented by multiple planes should also be included in the protection scope of this application.

[0137] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0138] The present application embodiment provides an external parameter calibration device for a monocular laser speckle projection system. The device comprises:

[0139] A camera calibration image acquisition module, used for acquiring a camera calibration image under a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector;

[0140] A camera parameter calibration module, used to calibrate camera parameters according to a camera calibration image;

[0141] A speckle image acquisition module is used to acquire a speckle image under a monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

[0142] A space plane equation calculation module is used to extract the signature features of the speckle image according to the camera parameters and calculate the space plane equation of the calibration plate;

[0143] A module for acquiring speckle image points with the same name, used for acquiring speckle image points with the same name according to the speckle image;

[0144] The same-name speckle image point calculation module is used to calculate the three-dimensional coordinates of the same-name speckle image points according to the spatial plane equation of the calibration plate;

[0145] An optical center and optical axis estimation module is used to estimate the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name;

[0146] A posture relationship calculation module, used to calculate the posture relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system;

[0147] The virtual speckle image generation module is used to calculate the plane homography matrix between the camera image and the laser speckle projector virtual image according to the posture relationship, and generate the laser speckle projector virtual image.

[0148] In the embodiment of the present application, compared with the traditional monocular laser speckle projection system, the present application realizes the external parameter calibration of the monocular laser speckle projection system through the calibration plate, and does not need to use an accurate distance measuring device to shoot the corresponding speckle image at different standard distances. This significantly improves the measurement efficiency and reduces the measurement cost. The monocular laser speckle projection system in the present application is equivalent to a binocular camera system with a speckle image, which improves the measurement accuracy. In addition, the present application can also calibrate the optical center and optical axis position of the laser speckle projector, so that the user can correct the optical axis offset of the monocular laser speckle projection system during use online.

[0149] The present application provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the method for calibrating external parameters of a monocular laser speckle projection system as described in the embodiment are performed.

[0150] The present application provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the steps of the method for calibrating the external parameters of a monocular laser speckle projection system as described in the embodiment are implemented.

[0151] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0152] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for calibrating the external parameters of a monocular laser speckle projection system, characterized in that: include: Collecting a camera calibration image under a monocular laser speckle projection system, wherein the monocular laser speckle projection system includes a camera and a laser speckle projector; Calibrate camera parameters according to the camera calibration image; A calibration plate is prepared to collect speckle images under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features; Extracting the signature features of the speckle image according to the camera parameters, and calculating the spatial plane equation of the calibration plate; The marking feature is a diagonal marking, and the calculating of the spatial plane equation of the calibration plate includes: A corresponding world coordinate system is established on the calibration plate, with the diagonal mark on the upper left as the origin of the coordinate system, the x-axis passes vertically downward through the diagonal mark on the lower left, the y-axis passes horizontally to the right through the diagonal mark on the upper right, and the z-axis is perpendicular to the calibration plate and outward, wherein the coordinate of the diagonal mark in the world coordinate system is expressed as (X w ,Y w , 1), the pixel coordinates (u, v) of the diagonal mark and the three-dimensional coordinates (X w ,Y w ,1) is expressed as: Among them, K c is the intrinsic parameter matrix of the camera, s is the scale factor, R is the rotation matrix, and T is the translation vector; the coordinates (X c ,Y c ,Z c ): The spatial plane equation of the calibration plate in the camera coordinate system is obtained by using the least squares method to fit; Acquire speckle image points with the same name according to the speckle image; Calculating the three-dimensional coordinates of the speckle image points with the same name according to the spatial plane equation of the calibration plate; The calculating the three-dimensional coordinates of the speckle image points with the same name comprises: According to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X c ,Y c ,Z c )for: Among them (C x ,C y ) is the principal point coordinate of the camera, d x and d y are the physical sizes of the pixel in the x-axis and y-axis directions, respectively, and f is the focal length of the camera; The equation of the space line passing through the speckle image point and the origin of the camera coordinate system can be expressed as: Calculating the three-dimensional coordinates of the speckle image points with the same name according to the spatial straight line equation and the spatial plane equation of the calibration plate to obtain a three-dimensional coordinate set corresponding to the speckle image points with the same name; Estimate the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name; Calculating the position relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system; The plane homography matrix between the camera image and the laser speckle projector virtual image is calculated according to the posture relationship to generate the laser speckle projector virtual image.

2. The method according to claim 1, characterized in that The calculating the posture relationship between the camera and the laser speckle projector includes: The normalized direction vector of the laser speckle projector optical axis in the camera coordinate system is V c =[v x ,v y ,v z ] T , which is represented as V in the laser speckle projector coordinate system p =[0,0,1] T , the relationship between the two can be expressed as: Calculate A x and A y : Select a preset A z Value, calculate the rotation matrix R, where A x , A y , A z is the Euler angle of the rotation matrix R; The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x p ,y p ,z p ), the coordinate in the laser speckle projector coordinate system is (0,0,0), and the relationship between the two can be expressed as: Get the translation vector T:

3. The method according to claim 1, characterized in that The calculating of the plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the posture relationship comprises: Assume that the coordinates of the speckle image point in the camera coordinate system are (x1, y1, z1) and in the laser speckle projector coordinate system are (x2, y2, z2). The relationship between the two can be expressed as: According to the fact that the speckle image point is located on the surface of the calibration plate, the coordinates of the speckle image point in the camera coordinate system satisfy the following equation: Wherein, n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate; Combining the above two equations, we get: Then the plane homography matrix between the camera image and the projector virtual image is expressed as: Among them, K c and K p is the intrinsic parameter matrix of the camera and the laser speckle projector, and K p =K c .

4. The method according to claim 1, characterized in that: The generating the laser speckle projector virtual image comprises: According to the plane homography matrix between the camera image and the laser speckle projector virtual image, the camera image is mapped to the virtual projector plane to obtain the laser speckle projector virtual image.

5. The method according to any one of claims 1 to 4, characterized in that: After the laser speckle projector virtual image is generated, the laser speckle projector is used as a second camera to form a binocular camera system together with the camera in the monocular laser speckle projection system.

6. An external parameter calibration device for a monocular laser speckle projection system, characterized in that: include: A camera calibration image acquisition module, used for acquiring a camera calibration image under a monocular laser speckle projection system, wherein the monocular laser speckle projection system comprises a camera and a laser speckle projector; A camera parameter calibration module, used to calibrate camera parameters according to the camera calibration image; A speckle image acquisition module, which is provided with a calibration plate and is used to acquire a speckle image under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features; A space plane equation calculation module, used to extract the signature features of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate; The marking feature is a diagonal marking, and the calculating of the spatial plane equation of the calibration plate includes: A corresponding world coordinate system is established on the calibration plate, with the diagonal mark on the upper left as the origin of the coordinate system, the x-axis passes vertically downward through the diagonal mark on the lower left, the y-axis passes horizontally to the right through the diagonal mark on the upper right, and the z-axis is perpendicular to the calibration plate and outward, wherein the coordinate of the diagonal mark in the world coordinate system is expressed as (X w ,Y w , 1), the pixel coordinates (u, v) of the diagonal mark and the three-dimensional coordinates (X w ,Y w ,1) is expressed as: Among them, K c is the intrinsic parameter matrix of the camera, s is the scale factor, R is the rotation matrix, and T is the translation vector; the coordinates (X c ,Y c ,Z c ): The spatial plane equation of the calibration plate in the camera coordinate system is obtained by using the least squares method to fit; A module for acquiring speckle image points with the same name, used for acquiring speckle image points with the same name according to the speckle image; A same-name speckle image point calculation module, used for calculating the three-dimensional coordinates of the same-name speckle image points according to the spatial plane equation of the calibration plate; The calculating the three-dimensional coordinates of the speckle image points with the same name comprises: According to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X c ,Y c ,Z c )for: Among them (C x ,C y ) is the principal point coordinate of the camera, d x and d y are the physical sizes of the pixel in the x-axis and y-axis directions, respectively, and f is the focal length of the camera; The equation of the space line passing through the speckle image point and the origin of the camera coordinate system can be expressed as: Calculating the three-dimensional coordinates of the speckle image points with the same name according to the spatial straight line equation and the spatial plane equation of the calibration plate to obtain a three-dimensional coordinate set corresponding to the speckle image points with the same name; An optical center and optical axis estimation module, used for estimating the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name; A posture relationship calculation module, used for calculating the posture relationship between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system; The virtual speckle image generation module is used to calculate the plane homography matrix between the camera image and the laser speckle projector virtual image according to the posture relationship, so as to generate the laser speckle projector virtual image.

7. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the processor performs the steps of the method for calibrating the external parameters of the monocular laser speckle projection system according to any one of claims 1 to 5.

8. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the method for calibrating the external parameters of a monocular laser speckle projection system according to any one of claims 1 to 5 are implemented.

Images