Laser sensor calibration method, device, calibration equipment and computer medium

Abstract

The present application discloses a laser sensor calibration method, device, calibration equipment and computer medium, and relates to the field of laser sensor technology. The laser sensor calibration method of the present application is applied to a calibration equipment equipped with a laser sensor, a double-sided step and a tilt-shift camera, and includes: determining the target camera internal parameters and target camera external parameters corresponding to the tilt-shift camera; controlling the laser sensor to emit a test line laser to the double-sided step to form a line laser plane; controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters to obtain the light plane parameters corresponding to the laser sensor. The present application can achieve the technical effect of quickly obtaining light plane parameters.

Description

Technical Field

[0001] The present application relates to the technical field of laser sensors, and in particular to a calibration method, apparatus, calibration device and computer-readable storage medium for a laser sensor. Background Art

[0002] In order to ensure that the laser sensor can generate accurate detection results during the detection of the object to be detected, technicians often need to calibrate the laser sensor in advance. Detecting the light plane parameters corresponding to the line laser plane generated by the laser sensor is the most important part of the laser sensor calibration process.

[0003] However, in the related technology, technicians need to use calibration equipment to control the laser sensor to generate multiple line laser planes, and use the tilt-shift camera configured on the calibration equipment to take multiple shots of the multiple line laser planes to obtain the light plane parameters corresponding to the line laser planes, thereby completing the calibration of the laser sensor. As a result, technicians often need to waste a lot of time and energy on calibrating laser sensors. Summary of the invention

[0004] The main purpose of this application is to provide a laser sensor calibration method, apparatus, calibration equipment and computer-readable storage medium, aiming to quickly obtain light plane parameters, thereby increasing the calibration speed of the laser sensor and improving the accuracy of the laser sensor.

[0005] To achieve the above object, the present application provides a laser sensor calibration method, which is applied to a calibration device equipped with a laser sensor, a double-sided step and a tilt-shift camera, and the laser sensor calibration method comprises the following steps:

[0006] Determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera;

[0007] Controlling the laser sensor to emit a test line laser toward the double-sided step to form a line laser plane;

[0008] The tilt-shift camera is controlled to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor.

[0009] Furthermore, the step of determining the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera includes:

[0010] Controlling the tilt-shift camera to shoot the calibration checkerboard to capture a first target image including the calibration checkerboard;

[0011] A target camera intrinsic parameter and a target camera extrinsic parameter corresponding to the tilt-shift camera are determined according to the first target image.

[0012] Furthermore, the step of determining target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera according to the first target image includes:

[0013] Determine the initial pixel coordinates of the checkerboard corner points in the calibration checkerboard in a pixel coordinate system according to the first target image;

[0014] Establishing a first tilt-shift camera model corresponding to the tilt-shift camera, and determining the world coordinates corresponding to the checkerboard corner points in a world coordinate system based on the first tilt-shift camera model and the initial pixel coordinates;

[0015] Determine an initial camera intrinsic parameter and an initial camera extrinsic parameter corresponding to the tilt-shift camera according to the initial pixel coordinates and the world coordinates, and determine a first pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the first tilt-shift camera model, the initial camera intrinsic parameter, the initial camera extrinsic parameter and the world coordinates;

[0016] Calculating a first coordinate error between the first pixel coordinate and the initial pixel coordinate, and constructing a first optimization equation according to the first coordinate error;

[0017] The initial camera intrinsic parameters and the initial camera extrinsic parameters are optimized by using the first optimization equation to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera;

[0018] A second tilt-shift camera model is constructed by using preset distortion parameters, and a second pixel coordinate of the checkerboard corner point in the pixel coordinate system is determined based on the optimized camera intrinsic parameters, the optimized camera extrinsic parameters, the world coordinates and the second tilt-shift camera model;

[0019] Calculating a second coordinate error between the second pixel coordinate and the first pixel coordinate, and constructing a second optimization equation according to the second coordinate error;

[0020] The optimized camera intrinsic parameters and the optimized camera extrinsic parameters are optimized by using the second optimization equation to obtain the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera.

[0021] Furthermore, the step of establishing a first tilt-shift camera model corresponding to the tilt-shift camera includes:

[0022] Acquire fixed camera intrinsic parameters corresponding to the tilt-shift camera, and construct an intrinsic parameter matrix corresponding to the tilt-shift camera based on the fixed camera intrinsic parameters;

[0023] Constructing an inclined plane coordinate system based on the vertical plane coordinate system corresponding to the tilt-shift camera, and constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system;

[0024] A first tilt-shift camera model corresponding to the tilt-shift camera is established according to the intrinsic parameter matrix, the rotation matrix and a preset Sham matrix.

[0025] Furthermore, the step of constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system includes:

[0026] Determine the rotation angle value between each first coordinate axis included in the vertical plane coordinate system and the corresponding second coordinate axis in the inclined plane coordinate system;

[0027] A rotation matrix corresponding to the tilt-shift camera is constructed based on each of the rotation angle values.

[0028] Furthermore, the step of controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters to obtain the light plane parameters corresponding to the laser sensor includes:

[0029] Controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain a second target image including the line laser plane and the double-sided steps;

[0030] Extracting each feature point in the line laser plane through the second target image, and constructing a target function based on each feature point;

[0031] The light plane parameters corresponding to the line laser plane are determined according to the objective function.

[0032] Furthermore, the step of constructing an objective function based on each of the feature points includes:

[0033] Determine the camera coordinates corresponding to each of the feature points in the vertical plane coordinate system and the step coordinates corresponding to each of the feature points in the step coordinate system;

[0034] A coordinate conversion matrix between the vertical plane coordinate system and the step coordinate system is constructed based on each of the camera coordinates and the step coordinates corresponding to each of the camera coordinates, and an objective function is constructed according to the coordinate conversion matrix.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a laser sensor calibration device, which is applied to a calibration device equipped with a laser sensor, a double-sided step and a tilt-shift camera, and the device includes:

[0036] A camera calibration module, used to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera;

[0037] A laser emission module, used for controlling the laser sensor to emit a test line laser toward the double-sided step to form a line laser plane;

[0038] The laser calibration module is used to control the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a calibration device, which includes: a memory and a processor, wherein the memory stores a calibration program for a laser sensor that can be run on the processor, and when the calibration program for the laser sensor is executed by the processor, the steps of the laser sensor calibration method as described above are implemented.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a laser sensor calibration program is stored. When the laser sensor calibration program is executed by a processor, the steps of the laser sensor calibration method as described above are implemented.

[0041] The laser sensor calibration method, apparatus, calibration device and computer-readable storage medium provided in the embodiments of the present application are applied to a calibration device equipped with a laser sensor, a bilateral step and a tilt-shift camera, by determining the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera; controlling the laser sensor to emit a test line laser to the bilateral step to form a line laser plane; and controlling the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameters and the target camera extrinsic parameters to obtain the light plane parameters corresponding to the laser sensor.

[0042] In this embodiment, when calibrating the laser sensor, the calibration device first calibrates the tilt-shift camera configured by itself to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera. After that, the calibration device controls the laser sensor configured by itself to emit a test line laser to the double-sided step used for calibration to form a line laser plane, and makes the line laser plane intersect with the double-sided step. Finally, the calibration device controls the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameters and target camera extrinsic parameters to determine the light plane parameters corresponding to the line laser plane, thereby completing the calibration of the laser sensor.

[0043] In this way, the present application controls the calibration device to calibrate the tilt-shift camera configured by itself to obtain the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera, and controls the laser sensor to emit a test laser to the double-sided steps to form a line laser plane intersecting the double-sided steps, and then controls the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameters and target camera extrinsic parameters to obtain light plane parameters, thereby ensuring that the calibration device can determine the light plane parameters of the line laser plane generated by the laser sensor when controlling the tilt-shift camera to take only one picture of the double-sided steps, thereby achieving the technical effect of quickly acquiring the light plane parameters, thereby improving the calibration speed of the laser sensor and improving the accuracy of the laser sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural diagram of a calibration device for a hardware operating environment involved in an embodiment of the present application;

[0045] Figure 2 This is a flow chart of a first embodiment of a calibration method for a laser sensor of the present application;

[0046] Figure 3 A schematic diagram of a vertical plane coordinate system and an inclined plane coordinate system involved in an embodiment of a laser sensor calibration method of the present application;

[0047] Figure 4 A schematic diagram of the positions of the line laser plane and the characteristic points involved in an embodiment of the laser sensor calibration method of the present application;

[0048] Figure 5 A schematic diagram of a scene in which a laser sensor emits a test line laser according to an embodiment of a laser sensor calibration method of the present application;

[0049] Figure 6 This is a schematic diagram of the structure of a calibration device involved in an embodiment of a calibration method for a laser sensor of the present application;

[0050] Figure 7 This is a schematic diagram of the functional modules involved in an embodiment of a calibration device for a laser sensor of the present application.

[0051] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] Reference Figure 1 , Figure 1 This is a schematic diagram of the calibration equipment structure of the hardware operating environment involved in the embodiment of the present application.

[0054] Please note that, please refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a calibration device according to an embodiment of a calibration method for a laser sensor of the present application. Figure 6 As shown, the calibration device of the embodiment of the present application can be an automated integrated calibration device which internally includes: a laser sensor 1, a bracket 2, a bilateral step 3, a fixed platform 4, a displacement platform 5, a light source 6, and a calibration checkerboard 7, wherein the light source 6, the calibration plate 7 and the bilateral step 3 are installed on the fixed platform 4, and the laser sensor 1 is installed on the bracket 2. The automated integrated calibration device can control the movement of the laser sensor 1 relative to the bilateral step 3 and the calibration plate 7 through the displacement platform 5, thereby calibrating the laser sensor 1.

[0055] like Figure 1 As shown, the calibration device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the calibration device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0057] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and a calibration program for the laser sensor.

[0058] exist Figure 1In the calibration device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the calibration device of the present application can be set in the calibration device, and the calibration device calls the calibration program of the laser sensor stored in the memory 1005 through the processor 1001, and executes the calibration method of the laser sensor provided in the embodiment of the present application.

[0059] Based on the above calibration equipment, the overall concept of the calibration method of the laser sensor of the present application is provided.

[0060] In the related art, technicians often control the laser sensor configured on the detection device to project a light plane that intersects with the object to be measured, and make the light plane form a light strip on the surface of the object to be measured, so as to obtain an image formed by the light strip projected into the tilt-shift camera, and determine the three-dimensional coordinates of the center of the surface light strip of the object to be measured relative to the camera based on the relative position relationship between the tilt-shift camera and the light plane. At the same time, the technician moves the object to be measured so that the light plane scans the entire surface of the object to be measured, so as to complete the detection of the object to be measured and obtain the complete three-dimensional morphology of the surface of the object to be measured.

[0061] However, before using a laser sensor to detect an object to be detected, the laser sensor must be calibrated to ensure that the laser sensor can generate accurate detection results during the detection of the object to be detected. Detecting the light plane parameters corresponding to the line laser plane generated by the laser sensor is the most important step in calibrating the laser sensor. In related technologies, technicians need to control the laser sensor to generate multiple line laser planes through a calibration device, and use a tilt-shift camera configured on the calibration device to shoot multiple line laser planes multiple times to obtain the light plane parameters corresponding to the line laser planes, thereby completing the calibration of the laser sensor. As a result, technicians often need to waste a lot of time and energy on calibrating laser sensors.

[0062] In view of the above phenomenon, the present application proposes a laser sensor calibration method, which is applied to a calibration device equipped with a laser sensor, a bilateral step and a tilt-shift camera. The laser sensor calibration method includes the steps of: determining a target camera intrinsic parameter and a target camera extrinsic parameter corresponding to the tilt-shift camera; controlling the laser sensor to emit a test line laser to the bilateral step to form a line laser plane; and controlling the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameter and the target camera extrinsic parameter to obtain light plane parameters corresponding to the laser sensor.

[0063] In this way, the present application controls the calibration device to calibrate the tilt-shift camera configured by itself to obtain the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera, and controls the laser sensor to emit a test laser to the double-sided steps to form a line laser plane intersecting the double-sided steps, and then controls the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameters and target camera extrinsic parameters to obtain light plane parameters, thereby ensuring that the calibration device can determine the light plane parameters of the line laser plane generated by the laser sensor when controlling the tilt-shift camera to take only one picture of the double-sided steps, thereby achieving the technical effect of quickly acquiring the light plane parameters, thereby improving the calibration speed of the laser sensor and improving the accuracy of the laser sensor.

[0064] Based on the above-mentioned calibration device and the overall concept of the calibration method of the laser sensor of the present application, various embodiments of the calibration method of the laser sensor of the present application are further proposed.

[0065] Please refer to Figure 2 , Figure 2 This is a flow chart of the first embodiment of the laser sensor calibration method of the present application.

[0066] It should be understood that although a logical order is shown in the flowchart, in some cases, the calibration method of the laser sensor of the present application can certainly perform the steps shown or described in an order different from that shown here.

[0067] like Figure 2 As shown, in this embodiment, the calibration method of the laser sensor of the present application may include the following steps:

[0068] Step S10: Determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera;

[0069] In this embodiment, when the calibration device needs to calibrate the laser sensor configured by itself, it first calibrates the tilt-shift camera configured by itself, so as to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera.

[0070] Furthermore, in a feasible embodiment, the above step S10 may specifically include:

[0071] Step S101: controlling the tilt-shift camera to shoot a calibration checkerboard to capture a first target image including the calibration checkerboard;

[0072] In this embodiment, the calibration device determines the position of the calibration checkerboard used for calibration, and then controls the shift camera configured therein to photograph the calibration checkerboard in different postures, thereby capturing a first target image containing the calibration checkerboard corresponding to multiple postures.

[0073] Step S102: determining target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera according to the first target image;

[0074] In this embodiment, the calibration device reads the first target image, thereby determining the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera calculated based on the target first image.

[0075] Exemplarily, for example, the automated integrated calibration device first determines the position of the calibration checkerboard 7 used for calibration, and based on the position, photographs the calibration checkerboard 7 in multiple postures, thereby obtaining multiple first target images containing the calibration checkerboard 7 corresponding to the multiple postures, and then the automated integrated calibration device analyzes the first target image to determine the pixel coordinates of the checkerboard corner points formed between the checkerboards in the calibration checkerboard 7 in the first target image in the pixel coordinate system, and then calculates the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera based on the pixel coordinates.

[0076] Furthermore, in a feasible embodiment, the above step S102 may specifically include:

[0077] Step S1021: determining initial pixel coordinates of corner points of the checkerboard in the calibration checkerboard in a pixel coordinate system according to the first target image;

[0078] Step S1022: establishing a first tilt-shift camera model corresponding to the tilt-shift camera, and determining the world coordinates corresponding to the checkerboard corner points in the world coordinate system based on the first tilt-shift camera model and the initial pixel coordinates;

[0079] Step S1023: determining initial camera intrinsic parameters and initial camera extrinsic parameters corresponding to the tilt-shift camera according to the initial pixel coordinates and the world coordinates, and determining first pixel coordinates of the checkerboard corner point in the pixel coordinate system based on the first tilt-shift camera model, the initial camera intrinsic parameters, the initial camera extrinsic parameters and the world coordinates;

[0080] Step S1024: calculating a first coordinate error between the first pixel coordinate and the initial pixel coordinate, and constructing a first optimization equation according to the first coordinate error;

[0081] Step S1025: optimizing the initial camera intrinsic parameters and the initial camera extrinsic parameters by using the first optimization equation to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera;

[0082] Step S1026: constructing a second tilt-shift camera model by using preset distortion parameters, and determining a second pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the optimized camera intrinsic parameters, the optimized camera extrinsic parameters, the world coordinates and the second tilt-shift camera model;

[0083] Step S1027: calculating a second coordinate error between the second pixel coordinate and the first pixel coordinate, and constructing a second optimization equation according to the second coordinate error;

[0084] Step S1028: optimizing the optimized camera intrinsic parameters and the optimized camera extrinsic parameters by using the second optimization equation to obtain target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera;

[0085] Exemplarily, for example, the automated integrated device first reads each first target image to determine the initial pixel coordinates of each checkerboard corner point formed by the intersection of each checkerboard contained in each first target image in the pixel coordinate system. At the same time, the automated integrated calibration device constructs a first tilt-shift camera model corresponding to the tilt-shift camera:

[0086]

[0087] Based on the first tilt-shift camera model and the initial pixel coordinates, the world coordinates corresponding to each checkerboard corner point in the world coordinates are determined. The automated integrated calibration device further determines the initial camera intrinsic parameters and initial camera extrinsic parameters corresponding to the tilt-shift camera based on the initial pixel coordinates and the world coordinates. After that, the automated integrated calibration device first determines the first pixel coordinates corresponding to each checkerboard corner point in the pixel coordinate system based on the initial camera intrinsic parameters, the initial camera extrinsic parameters, the first tilt-shift camera model and the world coordinates without considering distortion. The automated integrated calibration device calculates the first coordinate error between the first pixel coordinates corresponding to each checkerboard corner point and the initial pixel coordinates, and uses the Levenberg-Marquardt algorithm to establish a first optimization equation based on the first coordinate error:

[0088]

[0089] Afterwards, the automated integrated calibration device iteratively optimizes the initial camera intrinsic parameters and the initial camera extrinsic parameters based on the first optimization equation, and records the number of iterations. When the automated integrated calibration device determines that the number of iterations reaches a preset iteration number threshold, it completes convergence to obtain the optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera. At the same time, the automated integrated calibration device reads the storage device configured therein to obtain the radial distortion and thin prism distortion parameters pre-stored by the technician:

[0090]

[0091] And, the initial tilt-shift camera model corresponding to the tilt-shift camera without considering the distortion:

[0092]

[0093] The automated integrated calibration device updates the initial tilt-shift camera model based on the radial distortion and thin prism distortion parameters to obtain a second tilt-shift camera model:

[0094]

[0095] Then, the automated integrated calibration device determines the second pixel coordinates corresponding to each checkerboard corner point in the pixel coordinate system based on the second tilt-shift camera model, the optimized camera intrinsic parameters, the optimized camera extrinsic parameters and the world coordinate system, and calculates the second coordinate error between the second pixel coordinates corresponding to each checkerboard corner point and the initial pixel coordinates. The automated integrated calibration device then constructs a second optimization equation based on the second coordinate error, and iterates the optimized camera intrinsic parameters and the optimized camera extrinsic parameters again through the second optimization equation, while recording the number of iterations in the update iterative process. When the automated integrated calibration device determines that the number of iterations reaches a preset number threshold, convergence is completed, thereby obtaining the target camera intrinsic parameters and the target camera extrinsic parameters.

[0096] It should be noted that, in this embodiment, λ2 in the above-mentioned tilt-shift camera model represents a scale factor, the size of which is determined by the normalization of the coordinates, (u, v) represents the pixel coordinates corresponding to the checkerboard corner points in the checkerboard corner points, K is the internal parameter matrix corresponding to the tilt-shift camera, S τ is the Scheimpflug matrix, R τ It is a rotation matrix generated when an ideal vertical plane coordinate system rotates around the principal point to become an inclined plane coordinate system. R is the external parameter rotation matrix corresponding to the tilt-shift camera, and T is the external parameter translation vector corresponding to the tilt-shift camera. (X w , Y w , Z w ) is the world coordinate of the chessboard corner point in the world coordinate system.

[0097] Similarly, in this embodiment, m in the first optimization equation above represents the total number of shooting postures, n represents the total number of corner points of the calibration plate, and p (i,j) (u,v) represents the initial pixel coordinates corresponding to the i-th corner point in the calibration chessboard at the j-th pose, Represents the first pixel coordinate corresponding to the i-th corner point in the calibration chessboard at the j-th pose.

[0098] Similarly, in this embodiment, the radial distortion and thin prism distortion parameters are k1 and k2 are the first two order coefficients of radial distortion, s1~s4, X c and Y c is the thin prism distortion coefficient, λ3 and λ4 also represent scale factors. In addition, (X c ,Y c , 1) is the normalized coordinate corresponding to the vertical plane coordinate system, and (X' d ,Y d ',1) is the normalized coordinate corresponding to the vertical plane coordinate system under the condition of considering distortion.

[0099] Further, in a feasible embodiment, the step of “establishing a first tilt-shift camera model corresponding to the tilt-shift camera” in the above step S1022 may specifically include:

[0100] Step A10: acquiring fixed camera intrinsic parameters corresponding to the tilt-shift camera, and constructing an intrinsic parameter matrix corresponding to the tilt-shift camera based on the fixed camera intrinsic parameters;

[0101] Step A20: constructing an inclined plane coordinate system based on the vertical plane coordinate system corresponding to the tilt-shift camera, and constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system;

[0102] Step A30: establishing a first tilt-shift camera model corresponding to the tilt-shift camera according to the intrinsic parameter matrix, the rotation matrix and a preset Sham matrix;

[0103] Exemplarily, for example, the automated integrated calibration device first reads the storage device configured therein to obtain the fixed camera intrinsic parameters corresponding to the above-mentioned tilt-shift camera pre-stored by the technician, and constructs the intrinsic parameter matrix K corresponding to the tilt-shift camera based on the fixed camera intrinsic parameters:

[0104]

[0105] Afterwards, please refer to Figure 3 , Figure 3 Schematic diagram of a vertical plane coordinate system and an inclined plane coordinate system involved in an embodiment of a laser sensor calibration method of the present application, such as Figure 3 As shown, the automated integrated calibration device determines that the z-axis corresponding to the tilt-shift camera is parallel to the optical axis of the lens of the tilt-shift camera, and the distance between the origin of the coordinate and the rotation center on the z-axis is z cThe automatic integrated calibration device controls the vertical plane coordinate system to rotate with the rotation center on the optical axis of the tilt-shift camera lens as the reference, so as to obtain an inclined imaging coordinate system with the origin at the focal length f behind the camera coordinate system, the x-axis being parallel to the x-axis of the camera coordinate system, and the z-axis being parallel to the optical axis of the lens of the tilt-shift camera. The automatic integrated calibration device then constructs a rotation matrix R corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system. τ :

[0106]

[0107] Finally, the automated integrated calibration device obtains the preset Scheimpflug matrix S τ :

[0108]

[0109] The automated integrated calibration device is based on the internal parameter matrix K and the rotation matrix R τ and the Scheimpflug matrix S τ The first tilt-shift camera model mentioned above is established:

[0110]

[0111] It should be noted that, in this embodiment, the internal parameter matrix K Indicates the x coordinates of the tilt-shift camera along the camera coordinate system c and c The equivalent focal length in the direction. Similarly, u0 and v0 in the intrinsic parameter matrix K represent the position of the origin of the camera coordinate system, and f is the focal length of the lens of the tilt-shift camera.

[0112] In addition, in this embodiment, the rotation matrix R τ and the Scheimpflug matrix S τ The τ contained in x represents the rotation angle between the first x-axis in the vertical plane coordinate system and the second x-axis corresponding to the first x-axis in the inclined plane coordinate system. Similarly, τ y Represents the rotation angle between the first y-axis in the straight plane coordinate system and the second y-axis corresponding to the first y-axis in the inclined plane coordinate system.

[0113] Further, in a feasible embodiment, the step of “constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system” in the above step A20 may specifically include:

[0114] Step A201: determining a rotation angle value between each first coordinate axis included in the vertical plane coordinate system and a corresponding second coordinate axis in the inclined plane coordinate system;

[0115] Step A202: constructing a rotation matrix corresponding to the tilt-shift camera based on each of the rotation angle values;

[0116] For example, after the automated integrated calibration device controls the vertical plane coordinate system to rotate along the rotation center point on the lens optical axis of the tilt-shift camera to generate the inclined plane coordinate system, it first determines the rotation angle value τ formed between the first x-axis in the vertical plane coordinate system and the second x-axis corresponding to the first x-axis in the inclined plane coordinate system. x , and the rotation angle value τ formed between the first y-axis in the vertical plane coordinate system and the second y-axis corresponding to the first y-axis in the inclined plane coordinate system y , the automatic integrated calibration equipment is then based on the rotation angle value τ x and the rotation angle value τ y Construct the rotation matrix R corresponding to the tilt-shift camera τ :

[0117]

[0118] Step S20: controlling the laser sensor to emit a test line laser toward the double-sided step to form a line laser plane;

[0119] In this embodiment, the calibration device controls the laser sensor configured therein to emit a test line laser to the double-sided step configured therein, so that the test line laser intersects with the double-sided step to form a line laser plane.

[0120] For example, see Figure 4 , Figure 5 and Figure 6 ,in, Figure 4 Schematic diagram of the position of the line laser plane and the characteristic points involved in an embodiment of the laser sensor calibration method of the present application, Figure 5 FIG. 1 is a schematic diagram of a scene in which a laser sensor emits a test line laser according to an embodiment of a laser sensor calibration method of the present application. Figure 4 , Figure 5 and Figure 6 As shown, the automated integrated calibration device controls the laser sensor 1 configured therein to emit a test line laser to the double-sided step 3 configured therein, so that the test line laser intersects with the double-sided step 3, thereby forming a line laser plane.

[0121] Step S30: controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor;

[0122] In this embodiment, the calibration device controls the tilt-shift camera to shoot the line laser plane according to the above-mentioned target camera intrinsic parameters and target camera extrinsic parameters, thereby obtaining a second target image including the line laser plane and the double-sided steps. The calibration device then analyzes the second target image to obtain the light plane parameters corresponding to the line laser plane, thereby completing the calibration of the laser sensor.

[0123] Furthermore, in a feasible embodiment, the above step S30 may specifically include:

[0124] Step S301: controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain a second target image including the line laser plane and the double-sided steps;

[0125] In this embodiment, after controlling the laser sensor to emit a test line laser to the double-sided steps, the calibration device controls the above-mentioned tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters to capture a second target image including the line laser plane and the double-sided steps.

[0126] Step S302: extracting each feature point in the line laser plane through the second target image, and constructing an objective function based on each feature point;

[0127] In this embodiment, the calibration device reads the second target image to extract multiple feature points contained in the second target image formed by the intersection of the line laser and the double-sided steps, and determines the coordinate data of each feature point, and then constructs the target function based on the coordinate data of each feature point.

[0128] Step S303: determining the light plane parameters corresponding to the line laser plane according to the objective function;

[0129] In this embodiment, the calibration device solves the objective function to determine the light plane parameters corresponding to the line laser plane.

[0130] For example, the automated integrated calibration device controls the laser sensor 1 configured on the device to emit a test line laser to the double-sided step 3 configured on the device, and then calls the above-mentioned tilt-shift camera to shoot the line laser plane formed by the test line laser to capture a second target image containing k feature points formed by the intersection of the test line laser and the double-sided step 3. The automated integrated calibration device then reads the second target image to determine the camera coordinates (x ) corresponding to each feature point in the camera coordinate system.(i) ,y (i) ,z (i) ), the automated integrated calibration device then constructs the objective function based on the camera coordinates, and solves the objective function through SDV decomposition and other methods to obtain the light plane parameters:

[0131]

[0132] After obtaining the light plane parameters, the automated integrated calibration device determines that the calibration of the laser sensor 1 is completed.

[0133] It should be noted that, in this embodiment, A, B, C, and D included in the above light plane parameters are all light plane parameters.

[0134] Further, in a feasible embodiment, the step of "constructing an objective function based on each of the feature points" in the above step S302 may specifically include:

[0135] Step S3021: determining the camera coordinates corresponding to each of the feature points in the vertical plane coordinate system and the step coordinates corresponding to each of the feature points in the step coordinate system;

[0136] In this embodiment, the calibration device reads the second target image, thereby extracting multiple feature points contained in the second target image that are formed by the intersection of the test line laser and the bilateral steps, and determining the camera coordinates corresponding to each feature point in the vertical plane coordinate system. At the same time, the calibration device constructs a step coordinate system corresponding to the bilateral steps, and determines the step coordinates corresponding to each feature point in the step coordinate system.

[0137] Step S3022: constructing a coordinate transformation matrix between the vertical plane coordinate system and the step coordinate system based on the camera coordinates and the step coordinates corresponding to the camera coordinates, and constructing an objective function according to the coordinate transformation matrix;

[0138] In this embodiment, the calibration device constructs a coordinate transformation matrix between the vertical plane coordinate system and the step coordinate system based on the camera coordinates and the step coordinates corresponding to each feature point, and the calibration device further constructs the target function based on the coordinate transformation matrix.

[0139] For example, the automated integrated calibration device reads the second target image, thereby extracting k feature points formed by the intersection of the test line laser and the double-sided step 3 contained in the second target image, and determining the camera coordinates (x (i) ,y (i) ,z (i)), at the same time, the automated integrated calibration device takes any position in the bilateral step 3 as the origin, constructs a step coordinate system corresponding to the bilateral step 3, and determines the step coordinates corresponding to each feature point in the step coordinate system. After that, the automated integrated calibration device constructs a coordinate conversion matrix between the vertical plane coordinate system and the step coordinate system based on each camera coordinate and each step coordinate:

[0140]

[0141] The automated integrated calibration device then constructs an objective function based on the coordinate transformation matrix, and solves the objective function through methods such as SDV decomposition to obtain the light plane parameters:

[0142]

[0143] It should be noted that, in this embodiment, the coordinate transformation matrix (u ud ,v ud ) is the camera coordinate corresponding to each feature point in the absence of distortion, (X p ,Y p ) is the step coordinate corresponding to each feature point, H is the homography matrix, R p and T p is the extrinsic parameter matrix corresponding to the step coordinate system.

[0144] In this embodiment, when the calibration device needs to calibrate the laser sensor configured by itself, it first calibrates the tilt-shift camera configured by itself, so as to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera. Then, the calibration device controls the laser sensor configured by itself to emit a test line laser to the double-sided step configured by itself, so that the test line laser intersects with the double-sided step to form a line laser plane. Finally, the calibration device controls the tilt-shift camera to shoot the line laser plane according to the above-mentioned target camera intrinsic parameters and target camera extrinsic parameters, so as to obtain a second target image including the line laser plane and the double-sided steps. The calibration device then analyzes the second target image to obtain the light plane parameters corresponding to the line laser plane, and then completes the calibration of the laser sensor.

[0145] In this way, the present application controls the calibration device to calibrate the tilt-shift camera configured by itself to obtain the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera, and controls the laser sensor to emit a test laser to the double-sided steps to form a line laser plane intersecting the double-sided steps, and then controls the tilt-shift camera to shoot the line laser plane according to the target camera intrinsic parameters and target camera extrinsic parameters to obtain light plane parameters, thereby ensuring that the calibration device can determine the light plane parameters of the line laser plane generated by the laser sensor when controlling the tilt-shift camera to take only one picture of the double-sided steps, thereby achieving the technical effect of quickly acquiring the light plane parameters, thereby improving the calibration speed of the laser sensor and improving the accuracy of the laser sensor.

[0146] In addition, this application also provides a laser sensor calibration device, please refer to Figure 7 , Figure 7 This is a schematic diagram of the functional modules involved in an embodiment of a calibration device for a laser sensor of the present application. Figure 7 As shown, the device comprises:

[0147] A camera calibration module 10, used to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera;

[0148] The laser emitting module 20 is used to control the laser sensor to emit a test line laser to the double-sided step to form a line laser plane;

[0149] The laser calibration module 30 is used to control the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor.

[0150] Furthermore, the camera calibration module 10 includes:

[0151] An image acquisition unit, used for controlling the tilt-shift camera to shoot the calibration checkerboard to capture a first target image including the calibration checkerboard;

[0152] A parameter calculation unit is used to determine the target camera intrinsic parameters and target camera extrinsic parameters corresponding to the tilt-shift camera according to the first target image.

[0153] Furthermore, the parameter calculation unit includes:

[0154] A first coordinate confirmation subunit, used to determine the initial pixel coordinates of the checkerboard corner points in the calibration checkerboard in a pixel coordinate system according to the first target image;

[0155] a second coordinate confirmation subunit, configured to establish a first tilt-shift camera model corresponding to the tilt-shift camera, and determine the world coordinates corresponding to the checkerboard corner points in a world coordinate system based on the first tilt-shift camera model and the initial pixel coordinates;

[0156] a third coordinate confirmation subunit, configured to determine an initial camera intrinsic parameter and an initial camera extrinsic parameter corresponding to the tilt-shift camera according to the initial pixel coordinate and the world coordinate, and determine a first pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the first tilt-shift camera model, the initial camera intrinsic parameter, the initial camera extrinsic parameter and the world coordinate;

[0157] a first equation constructing subunit, configured to calculate a first coordinate error between the first pixel coordinate and the initial pixel coordinate, and construct a first optimization equation according to the first coordinate error;

[0158] A first parameter calculation subunit, configured to optimize the initial camera intrinsic parameters and the initial camera extrinsic parameters by using the first optimization equation to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera;

[0159] a fourth coordinate confirmation subunit, configured to construct a second tilt-shift camera model by using preset distortion parameters, and determine a second pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the optimized camera intrinsic parameters, the optimized camera extrinsic parameters, the world coordinates and the second tilt-shift camera model;

[0160] a second equation constructing subunit, configured to calculate a second coordinate error between the second pixel coordinate and the first pixel coordinate, and construct a second optimization equation according to the second coordinate error;

[0161] The second parameter calculation subunit is used to optimize the optimized camera intrinsic parameters and the optimized camera extrinsic parameters by using the second optimization equation to obtain the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera.

[0162] In addition, the present application also provides a calibration device having a laser sensor calibration program that can be run on a processor, and when the calibration device executes the laser sensor calibration program, the steps of the laser sensor calibration method described in any of the above embodiments are implemented.

[0163] The specific embodiments of the calibration device of the present application are basically the same as the embodiments of the calibration method of the laser sensor described above, and will not be described in detail here.

[0164] In addition, the present application also provides a computer-readable storage medium, on which a laser sensor calibration program is stored. When the laser sensor calibration program is executed by a processor, the steps of the laser sensor calibration method described in any of the above embodiments are implemented.

[0165] The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the embodiments of the laser sensor calibration method described above, and will not be described in detail here.

[0166] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0167] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, including a number of instructions for enabling a calibration device (which can be an automated integrated calibration device that internally includes: laser sensor 1, bracket 2, bilateral step 3, fixed platform 4, displacement platform 5, light source 6, calibration chessboard 7) to execute the methods described in each embodiment of the present application.

[0169] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A laser sensor calibration method, characterized in that: The laser sensor calibration method is applied to a calibration device equipped with a laser sensor, a double-sided step and a shift camera. The laser sensor calibration method comprises the following steps: Controlling the tilt-shift camera to shoot the calibration checkerboard to capture a first target image including the calibration checkerboard; Determine the initial pixel coordinates of the checkerboard corner points in the calibration checkerboard in a pixel coordinate system according to the first target image; Establishing a first tilt-shift camera model corresponding to the tilt-shift camera, and determining the world coordinates corresponding to the checkerboard corner points in a world coordinate system based on the first tilt-shift camera model and the initial pixel coordinates; Determine an initial camera intrinsic parameter and an initial camera extrinsic parameter corresponding to the tilt-shift camera according to the initial pixel coordinates and the world coordinates, and determine a first pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the first tilt-shift camera model, the initial camera intrinsic parameter, the initial camera extrinsic parameter and the world coordinates; Calculating a first coordinate error between the first pixel coordinate and the initial pixel coordinate, and constructing a first optimization equation according to the first coordinate error; The initial camera intrinsic parameters and the initial camera extrinsic parameters are optimized by using the first optimization equation to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera; A second tilt-shift camera model is constructed by using preset distortion parameters, and a second pixel coordinate of the checkerboard corner point in the pixel coordinate system is determined based on the optimized camera intrinsic parameters, the optimized camera extrinsic parameters, the world coordinates and the second tilt-shift camera model; Calculating a second coordinate error between the second pixel coordinate and the first pixel coordinate, and constructing a second optimization equation according to the second coordinate error; The optimized camera intrinsic parameters and the optimized camera extrinsic parameters are optimized by using the second optimization equation to obtain the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera; Controlling the laser sensor to emit a test line laser toward the double-sided step to form a line laser plane; The tilt-shift camera is controlled to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor.

2. The laser sensor calibration method according to claim 1, characterized in that: The step of establishing a first tilt-shift camera model corresponding to the tilt-shift camera comprises: Acquire fixed camera intrinsic parameters corresponding to the tilt-shift camera, and construct an intrinsic parameter matrix corresponding to the tilt-shift camera based on the fixed camera intrinsic parameters; Constructing an inclined plane coordinate system based on the vertical plane coordinate system corresponding to the tilt-shift camera, and constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system; A first tilt-shift camera model corresponding to the tilt-shift camera is established according to the intrinsic parameter matrix, the rotation matrix and a preset Sham matrix.

3. The laser sensor calibration method according to claim 2, characterized in that: The step of constructing a rotation matrix corresponding to the tilt-shift camera based on the vertical plane coordinate system and the inclined plane coordinate system comprises: Determine the rotation angle value between each first coordinate axis included in the vertical plane coordinate system and the corresponding second coordinate axis in the inclined plane coordinate system; A rotation matrix corresponding to the tilt-shift camera is constructed based on each of the rotation angle values.

4. The laser sensor calibration method according to claim 1, characterized in that: The step of controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters to obtain the light plane parameters corresponding to the laser sensor includes: Controlling the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain a second target image including the line laser plane and the double-sided steps; Extracting each feature point in the line laser plane through the second target image, and constructing a target function based on each feature point; The light plane parameters corresponding to the line laser plane are determined according to the objective function.

5. The laser sensor calibration method according to claim 4, characterized in that: The step of constructing an objective function based on each of the feature points comprises: Determine the camera coordinates corresponding to each of the feature points in the vertical plane coordinate system and the step coordinates corresponding to each of the feature points in the step coordinate system; A coordinate conversion matrix between the vertical plane coordinate system and the step coordinate system is constructed based on each of the camera coordinates and the step coordinates corresponding to each of the camera coordinates, and an objective function is constructed according to the coordinate conversion matrix.

6. A laser sensor calibration device, characterized in that: The laser sensor calibration device is applied to a calibration device equipped with a laser sensor, a double-sided step and a shift camera, and the device comprises: A camera calibration module, used for controlling the tilt-shift camera to shoot a calibration checkerboard to capture a first target image including the calibration checkerboard; Determine the initial pixel coordinates of the checkerboard corner points in the calibration checkerboard in a pixel coordinate system according to the first target image; Establishing a first tilt-shift camera model corresponding to the tilt-shift camera, and determining the world coordinates corresponding to the checkerboard corner points in a world coordinate system based on the first tilt-shift camera model and the initial pixel coordinates; Determine an initial camera intrinsic parameter and an initial camera extrinsic parameter corresponding to the tilt-shift camera according to the initial pixel coordinates and the world coordinates, and determine a first pixel coordinate of the checkerboard corner point in the pixel coordinate system based on the first tilt-shift camera model, the initial camera intrinsic parameter, the initial camera extrinsic parameter and the world coordinates; Calculating a first coordinate error between the first pixel coordinate and the initial pixel coordinate, and constructing a first optimization equation according to the first coordinate error; Optimizing the initial camera intrinsic parameters and the initial camera extrinsic parameters by using the first optimization equation to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters corresponding to the tilt-shift camera; A second tilt-shift camera model is constructed by using preset distortion parameters, and a second pixel coordinate of the checkerboard corner point in the pixel coordinate system is determined based on the optimized camera intrinsic parameters, the optimized camera extrinsic parameters, the world coordinates and the second tilt-shift camera model; Calculating a second coordinate error between the second pixel coordinate and the first pixel coordinate, and constructing a second optimization equation according to the second coordinate error; The optimized camera intrinsic parameters and the optimized camera extrinsic parameters are optimized by using the second optimization equation to obtain the target camera intrinsic parameters and the target camera extrinsic parameters corresponding to the tilt-shift camera; A laser emission module, used for controlling the laser sensor to emit a test line laser toward the double-sided step to form a line laser plane; The laser calibration module is used to control the tilt-shift camera to shoot the line laser plane according to the target camera internal parameters and the target camera external parameters, so as to obtain the light plane parameters corresponding to the laser sensor.

7. A calibration device, characterized in that: The calibration device includes: a memory and a processor, wherein the memory stores a laser sensor calibration program that can be run on the processor, and when the laser sensor calibration program is executed by the processor, the steps of the laser sensor calibration method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a laser sensor calibration program, and when the laser sensor calibration program is executed by a processor, the steps of the laser sensor calibration method according to any one of claims 1 to 5 are implemented.

Images