Structured light module calibration method, chip and calibration device

By utilizing the pinhole imaging principle and the method of motor-driven calibration plate sliding, a structured light ranging formula is constructed, which simplifies the calibration process of the structured light module, solves the problem of complex and inefficient calibration in existing technologies, and realizes efficient and automated camera calibration.

CN119022825APending Publication Date: 2024-11-26AMICRO SEMICONDUCTOR CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202310605693.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The camera calibration process of existing structured light vision systems is complex and inefficient, requiring specially made calibration targets to assist in calibration, which makes the calibration process cumbersome.

Method used

By adopting the pinhole imaging principle, the calibration plate is displaced relative to the structured light module to construct the structured light ranging formula, which simplifies the calibration process. Only a light strip needs to be projected and the image is acquired by the camera. The parameter calibration is performed by using a motor to drive the calibration plate to slide, which reduces the amount of calculation and simplifies the camera intrinsic parameter correction.

Benefits of technology

It improves the efficiency and accuracy of structured light module calibration, simplifies the computation, reduces the steps for correcting camera intrinsic parameters, and achieves efficient automated calibration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119022825A_ABST
    Figure CN119022825A_ABST
Patent Text Reader

Abstract

The invention discloses a calibration method of a structured light module, a chip and a calibration device. The calibration method comprises the following steps: step 1, setting positions of a calibration plate, a structured light emitter and a camera; 2, controlling a structured light emitter to emit structured light, projecting the structured light to the calibration plate to form a light strip, and controlling a camera to collect an image of the calibration plate; the calibration plate is controlled to displace relative to the structured light module, so that the camera obtains calibration plate images at different distances; and step 3, calculating modeling parameters required for constructing the structured light distance measurement relational expression based on the structured light distance measurement relational expression constructed under the pinhole imaging principle and the calibration plate images at different distances. Therefore, the calibration efficiency of the structured light module is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and in particular to a calibration method, chip and calibration device for a structured light module. Background Art

[0002] The structured light vision system is a measurement system that combines a line laser and a monocular camera. For example, a line laser sensor is composed of a camera and a line laser. Before using the structured light vision system, camera calibration and laser plane calibration are required to achieve three-dimensional measurement through imaging principles and laser plane parameters. This calibration method is applicable to most cameras. The line laser projects laser light onto the surface of the object to be measured. The laser enters the camera's receiving lens after diffuse reflection and forms an image on the camera's photosensitive element. The position of the image changes with the distance of the target. By imaging on the camera's photosensitive element and solving the relative position of the image, the relative height of the object to be measured can be obtained. However, a special calibration target is required to calibrate the camera's internal parameters first, which results in a complex calibration process and relatively low calibration efficiency. Summary of the invention

[0003] The present application discloses a calibration method, chip and calibration device for a structured light module. The specific technical solution is as follows: A calibration method for a structured light module, the structured light module comprising a structured light emitter and a camera; the calibration method comprising the following steps: step 1, setting the positions of a calibration plate, a structured light emitter and a camera; step 2, controlling the structured light emitter to emit structured light, projecting it onto the calibration plate and forming a light strip, and controlling the camera to collect an image of the calibration plate; and controlling the calibration plate to be displaced relative to the structured light module, so that the camera obtains images of the calibration plate at different distances; step 3, calculating modeling parameters required for constructing the structured light ranging relationship based on a structured light ranging relationship constructed under the pinhole imaging principle and images of the calibration plate at different distances.

[0004] In summary, the calibration method does not require additional target auxiliary calibration on the calibration plate. It only needs to project a light bar and be captured by the camera. Moreover, the parameters can be calibrated by constructing a structured light ranging relationship only by controlling the displacement of the calibration plate relative to the structured light module, thereby improving the efficiency of the calibration of the structured light module while ensuring the accuracy of the modeling parameters. In addition, based on the structured light ranging relationship constructed in this application, the ranging modeling method between the structured light and the camera is simplified to simply calculating and obtaining the modeling parameters, so that the calibration method has less calculation amount and improves the calibration efficiency; due to the simplification of the calibration method, the required calculation efficiency is also improved in the actual application process. Specifically, the mathematical formula constructed by the triangle ranging is simplified, and there is no need to calibrate the internal parameters of the camera. The amount of calculation is less, and the calibration efficiency of the structured light module is improved; in the actual application process, after obtaining the modeling parameters, the modeling and ranging operations can be performed without calibrating the internal parameters of the camera, and the calculation efficiency is improved.

[0005] Furthermore, the step 1 specifically includes: adjusting the emission direction of the structured light emitter to be set at an angle with the optical axis direction of the camera, setting the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate; setting the emission direction of the structured light emitter to intersect with the calibration plate, so that the structured light emitted by the structured light emitter in step 2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera; wherein the camera collects the image of the reflected light of the structured light in the photosensitive element of the camera, and uses the image of the light strip in the imaging plane of the camera to represent it; the image of the light strip in the imaging plane of the camera is a part of the image of the calibration plate. Thus, the relative position between the camera and the structured light emitter is kept unchanged, similar triangles are stably established for triangle distance measurement calculation, calibration is accelerated, and the accuracy and efficiency of the camera collecting images including the light strip are improved.

[0006] Further, step 2 includes: driving the calibration plate to slide on the slide rail; each time the calibration plate slides a certain distance along the slide rail, the camera collects the calibration plate image once, the light bar is offset once on the calibration plate, the camera detects the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane in real time, and the structured light module detects the distance between the baseline and the calibration plate in real time; wherein, the line between the center of the structured light emitter and the optical center of the camera is set as the baseline; wherein, the relative position between the structured light emitter and the camera is fixed, and the structured light module is installed at one end of the slide rail. Therefore, the calibration plate calibrates the structured light module under the drive of the motor, and there is no need to specially set up targets such as checkerboards. The calibration can be performed by controlling the calibration plate to slide along the slide rail by the motor, which is easy to realize automation; thereby, the efficiency of the structured light module calibration is improved while ensuring the reliability of the calibration result of the structured light module. The calibration plate, the slide rail, and the fixture for fixing the sensor in the structured light module can form a calibration device for calibrating the structured light module under the drive of the motor.

[0007] Furthermore, in step 2, when the calibration plate slides along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures the image of the light strip, and the imaging position of the light strip in the imaging plane is offset by a preset fixed height in the height direction, and the corresponding projected light spot pixel coordinates are offset once in the opposite direction of the height direction in the pixel plane; wherein, when the calibration plate slides to a position on the slide rail, the offset formed by the imaging position of the light strip in the imaging plane relative to the reference position is marked as the light strip offset. When the calibration plate slides along the slide rail for the same distance, the reflected light on the calibration plate can project the light strip on the imaging plane of the camera, and at least the light spot pixel coordinates can be obtained so as to construct multiple sets of the structured light ranging relationship in step 3, so as to solve the modeling parameters required to construct the structured light ranging relationship.

[0008] Furthermore, when the calibration plate slides along the slide rail to the upper limit of the range of the slide rail, the imaging position of the light bar in the imaging plane is set as a reference position, wherein the upper limit of the range of the slide rail is the position farthest from the structured light module to which the calibration plate is allowed to slide on the slide rail; or, the reference position is the imaging position in the camera of the reflected light of the structured light emitted by the structured light emitter on the measured surface of the measured object farthest from the structured light module within the detectable area of ​​the structured light module; wherein the measured object includes the calibration plate. In summary, the reference position is used to represent the offset reference point of the projection position of the light bar in the imaging plane coordinate system, and provides an imaging position offset starting point for the measured object or calibration plate at any distance, so as to facilitate the measurement and description of the displacement of the light bar in the imaging plane or in the photosensitive element from the reference position, thereby improving the calibration efficiency.

[0009] Furthermore, in step 3, the method for constructing a structured light ranging relationship according to the pinhole imaging principle includes: setting the length of the baseline as the baseline distance, which is predetermined; setting the distance between the baseline and the calibration plate as the test distance; the constructed structured light ranging relationship is: Px=k0 / d+k1, where d is the test distance and Px is the pixel coordinate of the light point; the modeling parameters required to construct the structured light ranging relationship include k0 and k1; where the emission angle of the structured light emitter, the focal length of the camera and the optical axis direction of the camera are all predetermined to determine the reflection light path of the single-point structured light from the perspective of the two-dimensional plane. Then the triangulated ranging relationship constructed by a pair of similar triangles is simplified, especially in the case of converting from the imaging plane coordinate system to the pixel coordinate system, reducing the type and number of parameters to be calculated, which is equivalent to reducing the number of camera intrinsic parameters to be calibrated, simplifying the camera calibration method. Therefore, it is only necessary to determine the two parameters k0 and k1 in the above structured light ranging relationship to complete the modeling of the structured light module, that is, to establish a model that uses structured light to measure the distance between the baseline and the object being measured, which is equivalent to completing the camera calibration and has simplified the camera calibration method. Then, on the basis of simplifying the camera internal parameter calibration operation, the functional relationship between the test distance and the light point pixel coordinates d=k0 / (Px-k1) is determined to calculate the test distance when the light point pixel coordinates are known, as the ranging information of the structured light module.

[0010] Further, in step 3, based on the structured light ranging relationship constructed under the pinhole imaging principle and the calibration plate images at different distances, the method for calculating the modeling parameters required to construct the structured light ranging relationship includes: in the process of the calibration plate sliding from the upper limit of the range to the lower limit of the range along the slide rail, or in the process of the calibration plate sliding from the lower limit of the range to the upper limit of the range along the slide rail, there is: the calibration plate slides along the slide rail to the first position, then the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as P x1, and determine the test distance as d1; then construct equation 1: Px1=k0 / d1+k1; the calibration plate slides to the second position along the slide rail, then the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as Px2, and determines the test distance as d2; then construct equation 2: Px2=k0 / d2+k1; then combine equation 1 and equation 2 to solve the modeling parameters k0 and k1; wherein, the distance between the second position and the baseline is not equal to the distance between the first position and the baseline. In summary, the pixel coordinates of the light spots disclosed in the technical solution do not need to be converted into an internal parameter matrix according to the internal parameter calibration method disclosed in the prior art, but rely on a motor to drive the calibration plate to slide to different distances to establish a set of equations for the structured light ranging relationship, solve the modeling parameters, and simplify the calculation amount of the transformation relationship from the camera coordinate system to the pixel coordinate system; compared with the prior art, the type and number of camera internal parameters that need to be calculated based on the structured light ranging relationship can be reduced, thereby improving the calibration efficiency of the structured light module; when it is subsequently actually applied to ranging, the calculation efficiency is also improved.

[0011] Furthermore, the light bar offset is equal to the sum of the product of the light spot pixel coordinates and the size of a single pixel and the projection error; the size of a single pixel and the projection error are both predetermined; the imaging of the light bar in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light bar offset are both set in the imaging plane coordinate system; the line connecting the reference position and the optical center of the camera is parallel to the structured light emitted by the structured light emitter; the light spot pixel coordinates are set in the pixel coordinate system, and the pixel coordinate system is the coordinate system of the pixel plane of the camera. Therefore, this technical solution not only simplifies the distance measurement relationship that requires coordinate system conversion under the pinhole imaging principle, but also reduces the parameters that need to be calibrated. The functional relationship between the light spot pixel coordinates in the photosensitive element and the test distance (the distance between the measured surface (target surface) of the calibration plate and the baseline) is simplified.

[0012] A chip is used to execute the calibration method. The chip can drive the calibration plate on the slide rail to move relative to the structured light module by controlling the motor, control the light strips formed on the surface of the object to be measured by the structured light emitted by the structured light module, and control the image of the camera including the light strips so that the coordinates of the pixel positions projected by the light strips on the pixel plane of the structured light module are detected by the camera in real time. The chip is equivalent to a controller for the structured light module and the motor.

[0013] A calibration device is provided, and the calibration device is used to calibrate the structured light module. The calibration device includes a calibration plate, a slide rail and the chip, wherein the chip is used to control the calibration plate to slide along the slide rail, and control the structured light emitter to emit structured light to the measured surface of the calibration plate, and control the camera to collect the image of the structured light projected onto the measured surface of the calibration plate, so as to calculate the modeling parameters by executing steps 1 to 3, and complete the calibration of the structured light module; wherein the structured light module includes a structured light emitter and a camera. When the Zhang calibration method disclosed in the prior art is used for camera calibration, a calibration plate with a chessboard is used for camera calibration, and the detection error of the chessboard corner points is high, resulting in a large error in the internal parameter matrix solved later, and the rotation matrix in the external parameter no longer satisfies the constraints of the special orthogonal group, and the internal and external parameter matrices solved are inaccurate. Compared with the prior art, the calibration device disclosed in the present application has low requirements on the structure of the construction and adopts a simplified camera calibration model. It can be widely used in many scenarios such as rapid measurement of product dimensions and rapid guidance of robot trajectories in various fields such as automobile production and electronic manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a calibration method for a structured light module is disclosed in one embodiment of the present application.

[0015] Figure 2 It is a schematic diagram of the construction principle of the structured light ranging relationship disclosed in one embodiment of the present application.

[0016] Figure 3 This is a schematic diagram of the relative positions of a structured light emitter, a camera and a light bar disclosed in an embodiment of the present application. Implementation

[0017] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention. It can be understood by ordinary technicians in related fields that the goal of camera calibration is to find a suitable or approximate mathematical model, find the parameters of this model, and use a simple mathematical model to express the complex imaging process, and find the inverse process of imaging. After calibration, the camera can reconstruct the three-dimensional scene, that is, the perception of depth, which is a major branch of computer vision.

[0018] Considering that in actual applications, the structured light vision system uses a small-angle camera that meets the pinhole imaging principle, and there is a certain angle between the forward emission direction of the straight line laser and the optical axis of the camera, and in the optical path system, the line laser is mostly oblique relative to the surface of the incident object. In order to reduce the complexity of calibration and improve the calibration efficiency, this application aims to provide a calibration method that does not rely on camera intrinsic parameter calibration for a structured light module composed of a small-angle distortion-free camera that meets the pinhole model, and uses the triangulation method commonly used in single-point line laser ranging to simplify the calculation formula required for the calibration of the structured light module, and obtain the corresponding parameters to replace the traditional camera intrinsic parameters, so there is no need to calibrate the camera intrinsic parameters.

[0019] As an embodiment, a calibration method of a structured light module is disclosed, wherein the structured light module includes a structured light emitter and a camera. Figure 1 As shown, the calibration method includes the following steps: Step 1: Set the position of the calibration plate, structured light emitter and camera, including determining the relative position relationship between any two devices, and preparing the device for the calibration of the camera; wherein the structured light emitter can be, but is not limited to, a line laser emitter, an infrared modulated emitter, or other light source that supports emitting linear structured light. The relative position between the structured light emitter and the camera is fixed, so that the structured light module is fixed at one position to form a sensor device with a fixed detection angle; an initial distance is set between the line between the center of the structured light emitter and the optical center of the camera and the calibration plate, as the starting point for the displacement of the calibration plate relative to the structured light module, and the line between the center of the structured light emitter and the optical center of the camera is also set as a fixed baseline, which can form a fixed assembly angle with the displacement direction of the calibration plate.

[0020] In order to reduce distortion calculations, the camera included in the structured light module uses a small-angle distortion-free camera to meet the pinhole imaging principle and ensure the reliability of the camera's internal parameters; the image sensor / photosensitive element set inside the camera can collect the reflected light of the structured light through an oblique optical path; the photosensitive element is a semiconductor device that converts light images into electronic signals; the distance between the surface of the photosensitive element and the optical center of the camera is equal to the focal length; the mass of photosensitive small molecules on the surface of the photosensitive element is called a pixel. Generally, the structured light module adopts the principle of triangular reflection, and the surface of the photosensitive element can be regarded as a matrix, so a light strip corresponding to the structured light can be projected, and the displacement of all points on the light strip can be measured at the same time.

[0021] Step 2: Control the structured light emitter to emit structured light, project it onto the calibration plate and form a light strip, for example, Figure 3 The structured light emitted by the structured light emitter in the camera changes the light path through the lens and wave mirror of the camera, so that the structured light emitted by the structured light emitter forms a structured light plane. It can be understood that the emitted light is an effect of an approximate plane. The structured light emitted by the structured light emitter is "regarded as" a collection of multiple single-point structured lights. In the modeling process in the geometric space, it can be regarded as composed of multiple single-point structured lights; the structured light plane intersects with the surface of the object to be measured to form a Figure 3 The light strip shown, and the camera is controlled to collect the calibration plate image, including the original image of the light strip, wherein a single-point structured light can be regarded as emitted by a single-point structured light emitter. For example, a single-point line laser is used to emit a single-point line laser. A single-point structured light in this embodiment represents a single-point line laser. The single-point line laser emitted by the word line laser passes through the lens of the camera to form a laser plane. The emitted laser plane is composed of multiple single-point line lasers. The word line laser can be decomposed into multiple single-point line lasers. Similarly, this application The disclosed structured light emitter can be regarded as composed of multiple single-point structured light emitters, each of which emits a single-point structured light, and multiple single-point structured lights form a structured light plane, that is, the structured light emitter emits a structured light plane and intersects with the calibration plate to form a light strip. When using single-point laser triangulation ranging, that is, using the pinhole imaging model (pinhole model) to establish a mathematical model or range measurement, it is necessary to divide the structured light into multiple single-point structured lights according to the measurement angle or directly regard it as a single-point structured light, corresponding to the optical path of the single-point structured light.

[0022] The structured light module can actively or under the command of an external controller extract height calibration data, X-axis component or Y-axis component of the light strip point cloud, X-direction calibration data of the black and white area or white area in the calibration plate, etc. from the calibration plate image, which can be used to convert the row pixel position information of the light strip (or the center line of the light strip), including the light point pixel coordinates, wherein the light point pixel coordinates can be regarded as the projection coordinates of the position point on the intersection line of the structured light plane emitted by the structured light emitter and the calibration plate in the pixel coordinate system of the camera. In addition, the calibration plate is controlled to be displaced relative to the structured light module, and the calibration plate can be driven by a motor to displace relative to the camera, so that the camera obtains calibration plate images at different distances, and each frame of the calibration plate image preferably includes light strips. Generally, the calibration plate moves away from the camera or approaches the camera in one direction, and the camera successively collects at least two frames of calibration plate images, and the two frames of calibration plate images are images of the calibration plate at two positions with different distances relative to the structured light module at the same position, and both include light strips, that is, each frame of the calibration plate image includes imaging information of the light strips.

[0023] In some embodiments, before the calibration plate is displaced, a certain distance is left between the calibration plate and the camera, so that when the calibration plate is displaced relative to the structured light module, the camera can collect two-dimensional images of the calibration plate at multiple distances, establish a more comprehensive calibration plate image library, and obtain pixel position information at more distances, including the pixel coordinates of light spots corresponding to light bars at different distances, which are used to construct multiple sets of formulas based on the pinhole imaging principle. The set of equations can be combined to obtain parameters that can represent the camera's intrinsic parameters, thereby simplifying the camera calibration method.

[0024] Step 3: Based on the structured light ranging relationship constructed under the pinhole imaging principle and the calibration plate images at different distances, the modeling parameters required to construct the structured light ranging relationship are calculated. Specifically, it can be understood that the relationship between the pixel coordinates of the light spot and the test distance constructed under the pinhole imaging principle is combined with the calibration plate images at different distances to calculate the modeling parameters required to construct the structured light ranging relationship, and determine that the calibration of the structured light module is completed, which is actually the completion of the camera calibration.

[0025] The test distance is the distance between the calibration plate and the structured light module, specifically, the distance between the line between the optical center of the camera and the center of the structured light emitter and the calibration plate (the measured surface of the calibration plate), which is regarded as the ranging distance of the structured light module. If the structured light emitter is decomposed into multiple single-point structured light emitters, the test distance is configured as the distance between the line between the center of the single-point structured light emitter and the optical center of the camera and the calibration plate. At this time, the structured light emitted by the structured light emitter is a straight-line structured light, which can be decomposed into multiple single-point structured lights, each of which is a straight line of light emitted by a single-point structured light emitter.

[0026] For light bars, Figure 3 In the two-dimensional light strip image of the intersection of the structured light plane and the surface of the object to be measured, the light spot pixel coordinates can be extracted, including the X-axis component and the Y-axis component, which correspond to the U-axis component (expressed by the number of pixels) and the V-axis component (expressed by the number of pixels) of the pixel coordinate system, which are derived from the result of the projection of the one-line structured light or multiple single-point structured lights. In this embodiment, the light spot pixel coordinates can represent the pixel coordinates that are not affected by distortion.

[0027] In this embodiment, in the process of calculating the modeling parameters required to construct the structured light ranging relationship by combining the constructed structured light ranging relationship and the calibration plate images at different distances, a relationship between the light spot pixel coordinates and the test distance is constructed. The modeling parameters required for the relationship are unknown quantities but remain constant, while the light spot pixel coordinates extracted from the calibration plate images at different distances and the test distance (the distance of the calibration plate relative to the structured light module) are both known quantities. Therefore, the known quantities are substituted into the structured light ranging relationship, and a group of equations with the modeling parameters as unknown quantities are established. The modeling parameters required to construct the structured light ranging relationship are solved to establish a mathematical model for structured light module ranging.

[0028] After constructing the structured light ranging relationship, the modeling parameters required for constructing the structured light ranging relationship are obtained. The modeling parameters are necessary parameters for constructing the structured light ranging relationship, and can be the result of deformation of the camera intrinsic parameters according to the pinhole imaging principle (pinhole model, generally used in the single-point structured light reflection optical path), so as to replace or omit the calibration of the camera intrinsic parameters before the ranging operation.

[0029] In summary, the calibration method does not require additional target auxiliary calibration on the calibration plate. It only needs to project the light bar and be collected by the camera. Moreover, the parameters can be calibrated by constructing the structured light ranging relationship only by controlling the displacement of the calibration plate relative to the structured light module, thereby improving the efficiency of the calibration of the structured light module while ensuring the reliability of the calibration results of the structured light module. In addition, based on the structured light ranging relationship constructed in this application, the ranging modeling method between the structured light and the camera is simplified to only calculate and obtain the modeling parameters, so that the calibration method has less calculation amount and improves the calibration efficiency; due to the simplification of the calibration method, the required calculation efficiency is also improved in the actual application process. Specifically, the mathematical formula constructed by the triangle ranging is simplified, and there is no need to calibrate the internal parameters of the camera. The amount of calculation is less, and the calibration efficiency of the structured light module is improved; in the actual application process, after obtaining the modeling parameters, the modeling and ranging operations can be performed without calibrating the internal parameters of the camera, and the calculation efficiency is improved.

[0030] As an embodiment, step 1 specifically includes: adjusting the emission direction of the structured light emitter to form an angle with the optical axis direction of the camera, the emission direction of the structured light emitter and the optical axis direction of the camera are set at a fixed angle, and an oblique light path is formed when the calibration plate is used for calibration calculation; the line between the center of the structured light emitter and the center of the camera is set to be parallel to the measured surface of the calibration plate, which can be fixed by a special fixture. Note that it is not limited to the camera and the calibration plate to be parallel, nor is it limited to the intersection of the structured light plane emitted by the structured light emitter and the calibration plate to be horizontal. Among them, the angle between the emission direction of the structured light emitter and the optical axis direction of the camera is also fixed by the fixture, which is used to project structured light and capture images on the same calibration plate, regardless of whether the calibration plate is displaced; the relative position between the center of the structured light emitter and the center of the camera, and the angle between the emission direction of the structured light emitter and the optical axis direction of the camera are fed back to the controller of the structured light module, and the controller of the structured light module can fix the relative position and angle by controlling the fixture. Combined with Figure 2 and Figure 3 It can be seen that in step 1 of this embodiment, the controller of the structured light module sets the emission direction of the structured light emitter to intersect with the calibration plate, so that the structured light (generally a single light ray, such as a single-point structured light, which can refer to the optical path of a single-point line laser) emitted by the structured light emitter in step 2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera, and the camera collects the image formed by the reflected light, wherein the camera collects the image of the reflected light of the structured light in the photosensitive element of the camera, and uses the image of the light bar in the imaging plane of the camera to represent it, and the image of the light bar in the imaging plane of the camera is a part of the calibration plate image. When the structured light emitter adopts a line laser emitter, the emitted laser line is structured light, such as a single-point line laser, and a laser plane is composed of countless laser lines emitted by the same line laser emitter, corresponding to Figure 3 The structured light plane shown in the figure, the intersection of the structured light plane and the calibration plate forms a light strip, so that the light strip is a linear structured light strip, such as a projection spot of a straight line laser in the calibration plate; it can be extracted from the calibration plate image to perform a triangulation operation (constructing similar triangles for distance calculation), specifically, each pixel point on the light strip (the projection position of each single-point structured light in the pixel plane or the imaging position in the imaging plane) can perform a triangulation operation (using the aforementioned modeling parameters to construct a structured light ranging relationship for triangulation calculation), and obtain the distance information of each pixel position within the angle range covered by the light strip relative to the camera. In this way, the relative posture between the camera and the structured light emitter is kept unchanged, similar triangles are stably established for triangulation calculation, calibration is accelerated, and the accuracy and efficiency of the camera in collecting images including light strips are improved.

[0031] It should be noted that the above description is only for the convenience of explaining the relative positions of the components in the structured light module. When the relative positions remain unchanged, the structured light module and the calibration plate do not necessarily have to be placed on the horizontal plane, but can also be placed on the vertical plane; when executing step 2, driven by the motor, the calibration plate moves freely or rotates relative to the slide rail, and the placement posture is unconstrained.

[0032] On the basis of the above embodiments, when the surface of the photosensitive element of the camera (photosensitive surface) is preferably parallel to the measured surface of the calibration plate, the object plane (from the calibration plate) and the image plane (photosensitive element of the camera) determined by the pinhole imaging principle are kept parallel, so as to facilitate the establishment of similar triangles for triangular distance measurement calculation, thereby using the triangulation distance measurement method commonly used in single-point line laser ranging to speed up the calibration. At this time, it may be necessary to regard the structured light plane as being decomposed into multiple single-point structured lights (structured lights emitted by single-point structured light emitters). For example, the structured light (straight-line structured light) is divided into single-point structured lights in multiple directions according to a preset angle (just like a straight-line laser is divided into multiple single-point line lasers), and then the triangulation distance measurement method (pinhole imaging principle) is used for each single-point structured light to speed up the calibration, so that the triangulation distance measurement methods in different directions can calibrate the corresponding modeling parameters.

[0033] On the basis of the above embodiment, step 2 also includes: the controller of the structured light module can drive the calibration plate to slide on the slide rail through a motor; under the drive of the motor, the calibration plate slides a certain distance along the slide rail, and the camera collects the calibration plate image once, and the light bar is offset once on the calibration plate, which is an offset in the height direction of the calibration plate, and the offset distance can be detected by the structured light module in real time; the line between the center of the structured light emitter and the optical center of the camera is set as the baseline; the relative position between the structured light emitter and the camera is fixed, and the structured light module is installed at one end of the slide rail. Each time the calibration plate image is collected, the calibration plate is located at a different position on the slide rail. When the structured light emitter is turned on, the camera collects the image containing the light bar, and the camera detects the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane in real time. Specifically, when the calibration plate is set on the normal plane of the slide rail, the height direction of the calibration plate can be regarded as the vertical direction of the slide rail, or parallel to the normal plane of the slide rail, and can also be used as the longitudinal direction of the imaging plane or pixel plane of the camera; preferably, the calibration plate is slidably mounted on one end of the slide rail, and the structured light module is fixedly mounted on the other end of the slide rail. Figure 2 As shown, the calibration plate is installed on the right side of the photosensitive element and the calibration plate is supported to slide left and right on the slide rail. The calibration plate is installed at the right end of the slide rail, and the structured light module is installed at the left end of the slide rail. Specifically, the structured light emitter and the camera are both installed at the left end of the slide rail, and the relative position between the structured light emitter and the camera is fixed by the fixture. Figure 2The line between the center A of the structured light emitter and the optical center B of the camera is set as the baseline AB and is fixed, at least in the process of calibrating the parameters of the camera and the structured light; the slide rail does not have to be placed on a horizontal plane, but can also be placed on a vertical plane and can be rotated at will. Every time the calibration plate slides a certain distance along the slide rail, the light bar on the calibration plate is translated once along the vertical direction of the slide rail; combined with Figure 2It can be seen that the calibration plate starts to slide to the right from a starting position of the slide rail (set at a track position at a known distance from the camera). When the calibration plate slides to the position where the structured light emitted by the structured light emitter (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the measured surface of the calibration plate at point O, the reflected light at position point O is imaged to position E on the surface of the camera's photosensitive element through the optical center B of the camera's lens, so that the camera obtains the pixel coordinates at position E in real time, which are expressed as the pixel coordinates of the light spot projected in the pixel plane after the light bar is reflected from position point O; at the same time, the vertical distance CO from point O to the baseline AB is also determined, which is expressed as a test distance, which is also the distance between the calibration plate at the current position and the baseline AB, and can be expressed by the displacement of the calibration plate relative to the structured light module along the slide rail. At this time, the angle between the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) and the baseline AB is ɑ, and the baseline AB is the line connecting the optical center B of the camera and the center A of the structured light emitter. When the calibration plate is set on the normal plane of the slide rail, the baseline AB is parallel to the measured surface of the calibration plate (regarded as perpendicular to the slide rail); then, when the angle between the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) and the baseline AB is fixed at ɑ, the calibration plate is only driven to slide away from the camera until the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting When a single-point structured light is emitted and intersects the measured surface of the calibration plate at point O', the distance between point O' and point O is equal to y, and the reflected light at point O' is imaged to position F on the surface of the camera's photosensitive element through the optical center B of the camera lens, that is, in the process of the calibration plate deviating from the fixed position of the camera along the slide rail, the imaging position of the reflected light on the surface of the camera's photosensitive element is offset from position E to position F along the height direction, and the camera obtains the pixel coordinates at position F in real time, which are expressed as the pixel coordinates of the light spot projected in the pixel plane after the light bar is reflected from point O' of the calibration plate; at the same time, the vertical distance C'O' from point O' to the baseline AB is also determined, which is expressed as a test distance, which is also the distance between the calibration plate at the current position and the baseline AB, and can be expressed by the displacement of the calibration plate relative to the structured light module along the slide rail. On this basis, the distance CO and the pixel coordinates at position E are substituted into the structured light ranging relationship described in step 3, and the distance C'O' and the pixel coordinates at position F are substituted into the structured light ranging relationship described in step 3 to construct a binary equation group for solving the relevant modeling parameters to complete the calibration process.Therefore, the calibration plate calibrates the structured light module under the drive of the motor. There is no need to specially set up targets such as a checkerboard. The calibration can be performed by simply controlling the calibration plate to slide along the slide rail, which is easy to automate. This improves the efficiency of the structured light module calibration while ensuring the reliability of the internal reference calibration results.

[0034] In this embodiment, the calibration plate, the slide rail, and the fixture for fixing the sensor in the structured light module can form a calibration device for calibrating the structured light module under the drive of a motor, wherein the structured light module can collect two-dimensional images of the calibration plate at different distances, and the light strips in a certain frame of the two-dimensional image will be clearly imaged in the imaging plane of the camera and will not deviate from the imaging plane of the camera (the surface of the photosensitive element). When the structured light module changes, including at least one of the focal length of the camera, the relative position between the structured light emitter and the camera, the emission direction of the structured light emitter (whose change will affect whether a light strip can be projected on the calibration plate at a certain distance), and the optical axis direction of the camera, the calibration plate needs to be driven by a motor to slide to the corresponding position before the structured light emitted by the structured light emitter can be incident on the calibration plate and the reflected light can be imaged through the camera lens to the surface of the camera's photosensitive element. Only then can the camera capture the image containing the light strip and then calibrate the camera. Therefore, it is necessary to constrain the sliding distance of the calibration plate on the slide rail, such as constraining the calibration plate to slide only one step distance each time under the drive of the motor to meet the camera imaging requirements.

[0035] It should be noted that the size of the calibration plate and the length of the slide rail should be adaptively designed according to the relative position between the camera and the structured light emitter and the ranging range limited on the slide rail to ensure that the posture information within the effective measurement range of the structured light module is covered.

[0036] Preferably, when the camera is a distortion-free camera, when the surface of the camera's photosensitive element and the measured surface of the calibration plate form a certain oblique angle, the distance between the baseline and the calibration plate can be directly calculated according to the pinhole imaging principle. The surface of the camera's photosensitive element and the measured surface of the calibration plate can be regarded as parallel. In this way, the distance between the structured light module and the calibration plate can be directly calculated according to the pinhole imaging principle, which will not affect the calibration accuracy of the camera.

[0037] Based on the aforementioned embodiment, in step 2, during the sliding of the calibration plate along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures an image of the light strip, and the imaging position of the light strip in the imaging plane is offset by a preset height in the height direction, that is, the imaging position in the image captured before the current sliding is offset by a preset height in the height direction, and then the pixel coordinates of the light spot formed by the projection of the light strip in the pixel plane are offset once in the opposite direction of the height direction, expressed by the number of pixels, and detected by the camera in real time.

[0038] When the calibration plate approaches the structured light module, that is, when it approaches the structured light module in steps according to a preset fixed distance, corresponding to the calibration plate sliding along the slide rail from the upper limit of the range to the lower limit of the range with the preset fixed distance as the stepping distance, the imaging position of the light bar in the imaging plane is offset in the height direction with the preset height as the stepping distance, which can be detected by the camera in real time; the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset in the opposite direction of the height direction with the preset longitudinal offset as the stepping distance, because the changes in the pixel plane and the changes on the imaging surface are reversed, for example, the imaging position of the light bar in the imaging plane is offset upward once along the longitudinal axis, and the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset downward once along the longitudinal axis; the camera detects the pixel coordinates of the light spot in the pixel coordinate system or pixel plane, preferably starting from a pixel offset starting point in the pixel plane of the camera, and measured with the preset longitudinal offset as the stepping distance.

[0039] When the calibration plate is away from the structured light module, that is, when it moves away from the structured light module in steps according to a preset fixed distance, the calibration plate slides along the slide rail from the lower limit of the range to the upper limit of the range with a preset fixed distance as the stepping distance. The imaging position of the light bar in the imaging plane is offset along the height direction with a preset height as the stepping distance, which can be detected by the camera in real time. The pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset once in the opposite direction of the height direction with a preset longitudinal offset as the stepping distance. The reason is that the changes in the pixel plane and the changes on the imaging surface are reversed. For example, the imaging position of the light bar in the imaging plane is offset upward once along the longitudinal axis, and the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset downward once along the longitudinal axis. The camera detects the pixel coordinates of the light spot in the pixel coordinate system or pixel plane, preferably starting from a pixel offset starting point in the pixel plane of the camera, and measured with a preset longitudinal offset as the stepping distance.

[0040] The upper limit of the range of the slide rail is the position of the calibration plate on the slide rail that is farthest from the structured light module (also considered as the camera), and the distance relative to the structured light module or the baseline is obtained in advance. The lower limit of the range of the slide rail is the position of the calibration plate on the slide rail that is closest to the structured light module (also considered as the camera), and the distance relative to the structured light module or the baseline is obtained in advance.

[0041] On the slide rail, when the initial position of the calibration plate is placed at the upper limit or the lower limit of the range, when the calibration plate slides to a position on the slide rail, the offset formed by the imaging position of the light bar in the imaging plane relative to the reference position is marked as the light bar offset, that is, the displacement from the reference position in the imaging plane coordinate system. Since the calibration plate will slide from the initial position to a position, the sliding distance and the position reached by the calibration plate are obtained in real time to determine the distance between the baseline and the calibration plate; since the imaging position of the light bar in the imaging plane is at a preset height, Since the step distance is offset along the height direction, the imaging position of the light strip in the imaging plane is obtained in real time, specifically, it is determined by calculating the coordinate information of the imaging position of the light strip reflected from the calibration plate at the initial position in the imaging plane; since the pixel coordinates of the light spot formed by the projection of the light strip in the pixel plane are offset in the opposite direction of the height direction with the preset longitudinal offset as the step distance, the pixel coordinates of the light spot formed by the projection of the light strip in the pixel plane are obtained in real time, specifically, it is determined by calculating the coordinate information of the pixel position projected from the light strip reflected from the calibration plate at the initial position in the pixel plane. Wherein, when the relative position of the structured light emitter and the camera remains unchanged, when the calibration plate is displaced relative to the baseline, the light strip offset changes, and the pixel coordinates of the light spot change.

[0042] refer to Figure 2 It can be known that when the calibration plate slides until the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the calibration plate at position point O, and the intersection line of the structured light plane formed by the structured light emitter and the measured surface of the calibration plate passes through position point O, the offset of the imaging position E of the light bar in the imaging plane relative to the reference position D is the light bar offset DE, represented by x in the figure; wherein, during the sliding of the calibration plate on the slide rail, the line between the center of the structured light emitter and the center of the camera remains parallel to the calibration plate. The motor driving the calibration plate to slide is a stepper motor. At each position to which the calibration plate slides driven by the stepper motor, the distance the calibration plate slides each time is the stepping distance generated by the stepping motor control, and the stepping distance is a preset fixed distance. When the calibration plate steps the same distance along the slide rail, the distance of the calibration plate relative to the structured light module and its change can be measured and recorded. The offset of the light bar on the calibration plate is equal, and the light bar can be projected in the imaging plane of the camera. Then the light bar offset and its change are detected in real time, and at least the pixel coordinates of the light point can be obtained so as to construct multiple sets of the structured light ranging relationship in step 3 to solve the modeling parameters required to construct the structured light ranging relationship.

[0043] In some embodiments, on the slide rail, the initial position of the calibration plate is placed at the lower limit of the range, that is, the structured light module is fixedly mounted at one end of the slide rail, and the calibration plate is slidably mounted on the other end of the slide rail, and is driven by a stepper motor to step toward the upper limit of the range of the slide rail in sequence. Here, the constraint on the range measurement range defined on the slide rail can be determined based on whether the light bar is clearly imaged on the photosensitive element of the camera, at least to ensure that the image coordinate position (the coordinate position in the image coordinate system (i.e., the imaging plane coordinate system)) projected by the light bar does not deviate from the surface area of ​​the photosensitive element, such as the farthest position from the camera that the calibration plate is allowed to slide on the slide rail is the farthest position that supports the light bar projected on the calibration plate by the structured light emitted by the structured light emitter to focus on the camera. When the calibration plate is at the upper limit of the range of the slide rail, the imaging position of the light bar in the imaging plane is set as the reference position, wherein the upper limit of the range of the slide rail is the farthest position from the structured light module (which can also be regarded as the camera) that the calibration plate is allowed to slide on the slide rail. Alternatively, the reference position is the imaging position in the camera of the reflected light of the structured light emitted by the structured light emitter on the measured surface of the measured object farthest from the structured light module within the detectable area of ​​the structured light module, which is recorded as the extreme position of the reflected light imaged on the surface of the photosensitive element when the measured surface of the measured object is infinitely far from the baseline; wherein the measured object includes the calibration plate. In summary, the reference position is used to represent the offset reference point of the projection position of the light bar in the imaging plane coordinate system, and provides an imaging position offset starting point for the measured object or calibration plate at any distance, which is convenient for measuring and describing the displacement of the light bar in the imaging plane or in the photosensitive element from the reference position, thereby improving the calibration efficiency.

[0044] As an embodiment, in step 3, the method of constructing a structured light ranging relationship according to the pinhole imaging principle includes: First, the mathematical model between the light point pixel coordinates and the test distance to be established (i.e. the aforementioned structured light ranging relationship) is explained. For ease of understanding, the single-point laser triangulation ranging principle is explained first. From the perspective of a two-dimensional plane, generally the perspective of the normal plane of the measured surface of the calibration plate, the structured light emitted by the structured light emitter is Figure 2 In the figure, a single-point structured light is regarded as being emitted, which is equivalent to a single-point line laser optical path diagram. When the structured light emitted by the structured light emitter in step 2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, the reflected light is imaged into the photosensitive element of the camera through the lens at position B. B is the optical center of the camera and also the pinhole in the pinhole model.

[0045] like Figure 2As shown, in this embodiment, the line between the center A of the structured light emitter and the optical center B of the camera is set as the baseline, and the length of the baseline is set as the baseline distance AB; the baseline distance AB is predetermined in this embodiment, and is an assembly parameter of the structured light module and also an assembly parameter in the height direction.

[0046] In this embodiment, the distance between the baseline and the calibration plate is set as the test distance. For example, when the calibration plate slides to the position where the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the measured surface of the calibration plate at point O, the reflected light at position point O is imaged to position E on the surface of the camera's photosensitive element through the optical center B of the camera's lens, so that the camera obtains the pixel coordinates at position E in real time, which are expressed as the pixel coordinates of the light point projected in the pixel plane after the light bar is reflected from position point O. Figure 2 The distance OC represents the test distance.

[0047] The structured light ranging relationship constructed according to the pinhole imaging principle is: Px=k0 / d+k1, which is actually a relationship between the pixel coordinates of the light spot and the test distance, where d is the test distance and Px is the pixel coordinates of the light spot. Here, the analysis is based on the X-axis component (associated with the baseline distance AB), and the unit is pixel, which can also be sub-pixel level. A set of modeling parameters required to construct the structured light ranging relationship include k0 and k1. Specifically, when the calibration plate slides on the slide rail, the pixel coordinates Px of the light spot, the baseline distance AB, the focal length f of the camera and the test distance OC are all measured in advance, and the emission angle of the structured light emitter and the optical axis direction of the camera are predetermined. The reflected light path of the structured light is determined from the perspective of the two-dimensional plane, and the structured light ranging relationship is constructed using the pinhole imaging principle without converting the three-dimensional coordinate system to the two-dimensional coordinate system.

[0048] The size of a single pixel and the projection error are both unknown quantities, so k0 and k1 are both unknown quantities that need to be solved. Figure 2The triangular ranging relationship constructed by a pair of similar triangles, triangle ABO and triangle DEB, is simplified, especially when converting from the imaging plane coordinate system to the pixel coordinate system, reducing the type and number of parameters that need to be calculated, which is equivalent to reducing the number of camera intrinsic parameters that need to be calibrated, simplifying the camera calibration method. Therefore, it is only necessary to determine the two parameters k0 and k1 in the aforementioned structured light ranging relationship to complete the modeling of the structured light module, that is, to establish a model that uses structured light to measure the distance between the baseline and the object being measured, which is equivalent to completing the calibration of the camera, and has simplified the camera calibration method. Then, on the basis of simplifying the camera intrinsic calibration operation, the functional relationship between the test distance and the light point pixel coordinates d=k0 / (Px-k1) is determined, which is used to calculate the test distance when the light point pixel coordinates are known, as the ranging information of the structured light module.

[0049] Based on the above embodiment, in step 3, based on the structured light ranging relationship constructed under the pinhole imaging principle and the calibration plate images at different distances, the method for calculating the modeling parameters required to construct the structured light ranging relationship includes: Under the control of the controller of the aforementioned structured light module, the motor drives the calibration plate to slide on the slide rail. In the process of sliding along the slide rail from the upper limit of the range to the lower limit of the range, the calibration plate slides through at least two positions in sequence according to a preset fixed distance (the step distance driven by the motor). At each position, the camera captures a frame of the calibration plate image including the light strip and extracts the pixel coordinates of the light spot, including the pixel coordinates of the light spot corresponding to a pixel point in the pixel coordinate system; when the relative position of the structured light emitter and the camera remains unchanged, when the calibration plate is displaced relative to the baseline, the light strip offset changes, the pixel coordinates of the light spot change, and k0 and k1 remain unchanged, so that k0 and k1 replace the internal parameters of the camera. Therefore, step 3 specifically contains: The calibration plate slides along the slide rail to the first position, and the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px1, which is obtained by the camera in real time from the surface of the photosensitive element, and at the same time determines that the test distance is d1; then construct equation 1: Px1=k0 / d1+k1, where Px1 and d1 are known quantities, and k0 and k1 are unknown quantities.

[0050] The installation position of the structured light module on the slide rail remains unchanged. For the interior of the structured light module, the optical axis direction of the camera remains unchanged, the emission direction of the structured light emitter remains unchanged, and the distribution position of the baseline remains unchanged (the relative position between the camera and the structured light emitter remains unchanged); then, the calibration plate slides along the slide rail to the second position, and the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px2, and at the same time determines that the test distance is d2; then construct equation 2: Px2=k0 / d2+k1, where Px2 and d2 are known quantities, and k0 and k1 are unknown quantities.

[0051] Then, equation 1 and equation 2 are combined to solve the modeling parameters k0 and k1, and the calibration of the structured light module is determined to be completed. Among them, the distance between the second position and the baseline is not equal to the distance between the first position and the baseline, that is, d2 is not equal to d1; then the modeling parameters k0 and k1 satisfy the structured light ranging relationship corresponding to all calibration plate images collected by the calibration plate along the slide rail, and then expand to meet the ranging requirements of the structured light module for the measured object outside it, that is, the structured light ranging relationship can be used to measure the distance of the measured object, that is, as the modeling parameters required for the ranging model of the structured light module.

[0052] It should be noted that the unchanging modeling parameters calculated are the necessary parameters in the structured light ranging relationship followed by each calibration plate image and the corresponding light spot pixel coordinates during the sliding process of the calibration plate. Therefore, to construct this structured light ranging relationship, it is necessary to introduce the light spot pixel coordinates projected at multiple distance positions for calculation in order to calibrate the modeling parameters. When the measured object has a relative displacement with the baseline, x will change accordingly, and Px will then be displaced. Therefore, it is only necessary to determine the two parameters k0 and k1 in this structured light ranging relationship to complete the modeling of the distance measurement model of the structured light module.

[0053] In summary, the pixel coordinates of the light spots disclosed in this embodiment do not need to be converted into an internal parameter matrix according to the internal parameter calibration method disclosed in the prior art, but rely on a motor to drive the calibration plate to slide to different distances to establish a set of equations for the structured light ranging relationship, solve the modeling parameters, and simplify the calculation amount of the transformation relationship from the camera coordinate system to the pixel coordinate system; compared with the prior art, based on the aforementioned structured light ranging relationship, the types and quantities of camera internal parameters that need to be calculated can be reduced, thereby improving the calibration efficiency of the structured light module; when it is subsequently actually applied to ranging, the calculation efficiency is also improved.

[0054] In the above embodiment, based on the pinhole imaging principle, the ratio of the baseline distance to the test distance is equal to the ratio of the light strip offset to the focal length of the camera. Figure 2It can be seen that the position D on the surface of the camera's photosensitive element is used as the reference position, where the line connecting the reference position D and the camera's optical center B is parallel to the structured light emitted by the structured light emitter at position A (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light, which can be a single-point line laser). The angle ɑ formed by the structured light emitted by the structured light emitter and the baseline AB (also regarded as the baseline distance AB) also determines the reference position. Figure 2 It can be seen that when the calibration plate slides until the structured light emitted by the structured light emitter (from the perspective of the two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single point of structured light) intersects with the calibration plate at position point O, and when the intersection line of the structured light plane formed by the structured light emitter and the measured surface of the calibration plate passes through position point O, the offset formed by the imaging position E of the light bar in the imaging plane relative to the reference position D is the light bar offset DE, which is represented by x in the figure; the distance between the baseline AB and the calibration plate, that is, the test distance is represented by OC. Since the baseline AB is parallel to the calibration plate, the surface of the camera's photosensitive element is regarded as parallel to the baseline, so based on the pinhole imaging principle, the triangle ABO and the triangle DEB are similar, so there is: AB / OC=x / f. Therefore, x=(AB*f) / OC, where OC supports real-time measurement during the sliding of the calibration plate on the slide rail. Thus, a calculation model for the light bar offset is established through the baseline distance, focal length and test distance.

[0055] In the aforementioned embodiment, the light strip offset x is equal to the sum of the product of the light spot pixel coordinate Px and the single pixel size CellSize and the projection error Dev, that is, x=Px*CellSize+Dev. The premise for this relationship is that the imaging of the light strip in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light strip offset are both set in the imaging plane coordinate system, the light spot pixel coordinates are set in the pixel coordinate system, and the pixel coordinate system is the coordinate system of the pixel plane of the camera. The projection error Dev represents a light strip on the surface of the object to be measured. The difference between the light strip offset acquired by the camera in real time and the light strip offset formed by the actual projection in the imaging plane of the camera is the result obtained in advance. Figure 2In the equation, combined with x=(AB*f) / OC, the structured light ranging relationship that can be obtained is Px=[(AB*f / OC)-Dev] / CellSize; then, k0 is represented by (AB*f) / CellSize; at the same time, k0 is represented by -Dev / CellSize; at this point, the structured light ranging relationship can be simplified to Px=k0 / d+k1. In this embodiment, k0 and k1 are both modeling parameters to be determined. Therefore, the camera intrinsic parameters that originally need to be calibrated are simplified from Dev, CellSize, AB and f to k0 and k1, which not only simplifies the ranging relationship that requires coordinate system conversion under the pinhole imaging principle, but also reduces the parameters that need to be calibrated. The functional relationship between the pixel coordinates of the light spot in the photosensitive element and the test distance (the distance between the measured surface (target surface) of the calibration plate and the baseline) is simplified.

[0056] This application needs to obtain the structured light ranging relationship, so it is necessary to understand how the camera itself obtains the world coordinates in the three-dimensional space step by step from the pixel coordinates. Here, the following coordinate systems are involved: World coordinate system: A reference coordinate system is also selected in the environment to describe the position of the camera and the object being measured. This coordinate system is called the world coordinate system. The relationship between the camera coordinate system and the world coordinate system can be described by a rotation matrix and a translation vector. In this application, the coordinate system where the structured light module or structured light emitter is located can be used as the world coordinate system.

[0057] Camera coordinate system: The origin of the camera coordinate system is the optical center of the camera. The x-axis and y-axis are parallel to the X and Y axes of the image. The z-axis is the optical axis of the camera, which is perpendicular to the image plane. The spatial rectangular coordinate system formed by this is called the camera coordinate system, also known as the camera coordinate system. The camera coordinate system is a three-dimensional coordinate system. The intersection of the optical axis and the image plane is the origin of the image coordinate system. The rectangular coordinate system formed by the X and Y axes of the image is the image coordinate system. The image coordinate system is a two-dimensional coordinate system. If the world coordinates are converted to camera coordinates, there is a rigid body transformation. In fact, it corresponds to the camera extrinsic parameters. The camera coordinates are converted to image coordinates, which is a 3D->2D perspective projection process.

[0058] Imaging plane coordinate system (X, Y): The pixel coordinate system is not conducive to coordinate transformation, so it is necessary to establish an image coordinate system XOY (i.e., imaging plane coordinate system). The units of its coordinate axes are usually millimeters (mm), and the origin is the intersection of the camera optical axis and the image plane (called the principal point), i.e., the center point of the image. Therefore, the two coordinate systems are actually in a translation relationship, that is, they can be obtained by translation.

[0059] The pixel coordinate system (U, V) is a two-dimensional rectangular coordinate system that reflects the arrangement of pixels on a two-dimensional image plane. The U axis and the V axis are parallel to the two sides of the image plane. The unit of the coordinate axis in the pixel coordinate system is pixel (integer). The image coordinates are converted into pixel coordinates, and the image coordinate system is discretized. The origin of the pixel coordinate system is generally located in the upper left corner of the image. The offset from the optical center to the origin of the pixel coordinate system is recorded as the reference pixel coordinate, that is, the coordinate of the camera optical center in the pixel coordinate system. The size of each pixel in the horizontal and vertical directions is recorded as (mm / pixel). The pixel coordinates of the light bar can be at the sub-pixel level, that is, the pixel coordinate value can be expressed as a decimal to improve the marking accuracy of the pixel coordinates.

[0060] As an embodiment, after the light strip is imaged by the camera, it is represented by multiple columns of pixels in the pixel plane. Each column of pixels identifies the pixel position of each point cloud point constituting the light strip projected in the camera's photosensitive element. The coordinates of each column of pixels in the pixel coordinate system are represented as the light point pixel coordinates. In this embodiment, the camera can detect the light point pixel coordinates in each column formed by the projection of the light strip in the pixel plane in real time. Each light strip contained in the calibration plate image can be represented by multiple columns of pixels in the pixel coordinate system. The structured light emitted by the structured light emitter is composed of multiple single-point structured lights, and the structured light emitter actually emits Figure 3 Although the light projected onto the surface of the object to be measured or the surface to be measured on the calibration plate is a light strip, the number of columns of pixels corresponding to the light strip forming the calibration and the distribution of the columns of pixels projected by the same light strip on the pixel plane can be determined according to the camera resolution of the structured light module, the operating computing power of the controller of the structured light module, and the specifications of the internal and external parameters of the camera, and this embodiment does not impose any restrictions.

[0061] In this embodiment, each column of pixels forms a corresponding measurement angle relative to the optical axis of the camera, and there is a light point pixel coordinate in the pixel coordinates of a column of pixels projected corresponding to the light bar, and a light point pixel coordinate corresponds to a measurement angle. A measurement angle represents the azimuth formed by a position point on the measured surface of the calibration plate relative to the optical axis of the camera. The structured light emitted by the structured light emitter can be regarded as consisting of multiple single-point structured lights on the basis of the determined multiple measurement angles to form a structured light plane; within the structured light plane, a single-point structured light forms a corresponding measurement angle relative to the optical axis of the camera, and a single-point structured light is correspondingly projected in a column of the pixel plane (pixel coordinate system) of the camera to form a light point pixel coordinate, so that each column of pixels is located at a pixel position formed by the projection of a single-point structured light in the pixel plane after being reflected by the calibration plate. Since the structured light plane can be approximately regarded as a combination of countless single-point structured lights emitted from the structured light emitter, the structured light ranging relationship can be constructed for certain discrete pixel points in the pixel coordinate system according to step 3 to obtain corresponding modeling parameters at a finite number of measurement angles, and the distance information of the measured object at a finite number of measurement angles of the camera can be obtained, which can meet some simple contour point ranging application scenarios.

[0062] Preferably, since the structured light emitted by the structured light emitter constitutes a structured light plane and the light strip is located at the intersection of the structured light plane and the measured surface of the calibration plate, the projection position of each laser point constituting the light strip in the camera's photosensitive element can be connected into a horizontal projection line segment in the pixel coordinate system, thereby reflecting that the structured light is a straight-line structured light and the light strip is parallel to the slide rail, reducing the complexity of the posture conversion required for calibration and improving the efficiency of calibration.

[0063] On the basis of the above embodiment, since each column of pixel points identifies the pixel position of each point cloud point constituting the light strip projected in the photosensitive element of the camera, a column of pixel points forms a corresponding measurement angle relative to the optical axis of the camera, a single point structured light forms a corresponding measurement angle relative to the optical axis of the camera, and a single point structured light is correspondingly projected in a column of the pixel plane of the camera to form a light point pixel coordinate, so there is at least one light point pixel coordinate in each column of pixel points and it is located at a corresponding measurement angle relative to the optical axis of the camera. In step 3, a structured light ranging relationship is constructed for each column of pixel points and the corresponding modeling parameters are obtained. In step 3, the structured light ranging relationship constructed for each column of pixel points corresponds to a measurement angle, so that: the structured light emitter is equivalent to being decomposed into a plurality of single point structured light emitters, each single point structured light emitter emits a single point structured light, and the corresponding test distance at each measurement angle is updated to the distance between the line connecting the center of the single point structured light emitter and the optical center of the camera and the calibration plate, thereby realizing the execution of steps 1 to 3 for the ranging system composed of the single point structured light emitter and the camera at each measurement angle in sequence.

[0064] Based on the embodiments described in the aforementioned steps 1 to 3, when a camera that satisfies the pinhole imaging principle is used, each column of pixels on the pixel plane can be approximately regarded as a picture seen at the same vertical angle of the structured light plane. At this vertical angle, there is a single-point structured light emitter (similar to the function of a single-point laser) "extracted" from the straight-line structured light (the light strip), and the vertical angle is the aforementioned measurement angle. Therefore, by executing steps 1 to 3, it can be modeled with two modeling parameters (corresponding to constructing the structured light ranging relationship, and a structured light ranging relationship is constructed at a vertical angle). By analogy, if there are N columns of pixel points in the pixel plane where the calibration plate image is located, N light point pixel coordinates may be extracted therefrom; steps 1 to 3 are executed in turn for modeling the ranging system composed of the single-point structured light emitter and the camera at each measurement angle, that is, the structured light ranging relationship is constructed at each measurement angle to obtain the corresponding modeling parameters, avoiding the need to calibrate the camera internal parameters first; and the ranging modeling of the entire structured light module is also completed.

[0065] The present application also discloses a chip, which is used to execute the calibration method. As the execution subject of steps 1, 2 and 3 in the above-mentioned embodiment, the chip can not only drive the calibration plate on the slide rail to displace relative to the structured light module by controlling the motor, but also control the light strip formed by the structured light emitted by the structured light module on the surface of the measured object, and also control the image of the camera including the light strip, so that the coordinates of the pixel position projected on the pixel plane of the structured light module are detected by the camera in real time, then the chip is equivalent to a common controller of the structured light module and the motor. Further, the chip can also perform steps 1 to 3 to model the ranging system composed of the single-point structured light emitter and the camera at each measuring angle in sequence when the structured light emitter is equivalent to being decomposed into a plurality of single-point structured light emitters or the structured light emitted by the structured light emitter is regarded as being composed of a plurality of single-point structured lights, that is, the structured light ranging relationship is constructed at each measuring angle to obtain the corresponding modeling parameters, avoiding the need to calibrate the camera internal parameters first; and also completing the ranging modeling of the entire structured light module. Then, based on the structured light distance measurement relationship and the pre-calculated pixel coordinates of each light point, the test distance is converted, and the converted test distance is set as the distance between the measured object and the structured light module. In this way, the most effective depth data in all directions can be obtained without performing overly complex camera internal parameter calibration and coordinate system conversion calculations in advance.

[0066] The present application also discloses a calibration device, which is used to calibrate the structured light module. The calibration device includes a calibration plate, a slide rail and the chip disclosed in the above-mentioned embodiment, wherein the chip is used to control the calibration plate to slide along the slide rail, specifically, the calibration plate is driven by a motor to slide within a predetermined range, and the structured light emitter is controlled to emit structured light to the measured surface of the calibration plate, and the camera is controlled to collect the image of the structured light projected onto the measured surface of the calibration plate, which is an image including light strips; so as to calculate the modeling parameters by executing steps 1 to 3; wherein the structured light module includes a structured light emitter and a camera, specifically, the structured light emitter can be decomposed into a plurality of single-point structured light emitters or the structured light emitted by the structured light emitter can be composed of a plurality of single-point structured lights.

[0067] The calibration device disclosed in the present application obtains the modeling parameters required to construct the structured light ranging relationship after constructing the structured light ranging relationship. The modeling parameters are necessary parameters for constructing the structured light ranging relationship, and can be the result of the deformation of the camera internal parameters according to the pinhole imaging principle (pinhole model, generally used in the single-point structured light reflection light path), which can represent the camera internal parameters in the traditional sense, so as to replace or omit the calibration of the camera internal parameters before the ranging operation. There is no need to calibrate the camera's internal parameters, the amount of calculation is less, and the calibration efficiency of the calibration device for the structured light module is improved; in the actual application process, after obtaining the modeling parameters, the modeling and ranging operations can be performed without calibrating the camera's internal parameters, thereby improving the calibration efficiency of the calibration device.

[0068] Compared with the prior art, the calibration device disclosed in the present application has low requirements on the structure of the construction and adopts a simplified camera calibration model. It can be widely used in many scenarios such as rapid measurement of product dimensions and rapid guidance of robot trajectories in various fields such as automobile production and electronic manufacturing.

[0069] Preferably, the calibration device further comprises a fixture, which is used to fix the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate, and is also used to assemble the emission direction of the structured light emitter and the optical axis direction of the camera to be set at an angle; the distance between the center of the structured light emitter and the center of the camera is fixed. The emission direction of the structured light emitter intersects with the calibration plate, so that the structured light emitted by the structured light emitter is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera, wherein the camera collects the image of the reflected light of the structured light in the photosensitive element of the camera, and uses the image of the light bar in the imaging plane of the camera to represent it; the image of the light bar in the imaging plane of the camera is part of the image of the calibration plate; it is worth noting that the camera is configured to be free of distortion. Therefore, this embodiment reduces distortion calculation, and supports the use of the pinhole imaging principle to accelerate the establishment of the structured light ranging relationship, thereby accelerating calibration.

[0070] The relative position between the center of the structured light emitter and the center of the camera, and the angle between the emission direction of the structured light emitter and the optical axis direction of the camera are fed back to the chip or structured light module. The chip can fix the relative position and angle by controlling the fixture, thereby maintaining the relative posture between the camera and the structured light emitter unchanged.

[0071] It should be noted that the above description is only for the convenience of explaining the relative positions of the components in the structured light module. When the relative positions remain unchanged, the structured light module and the calibration plate do not necessarily have to be placed on the horizontal plane, but can also be placed on the vertical plane; when executing step 2, driven by the motor, the calibration plate moves freely or rotates relative to the slide rail, and the placement posture is unconstrained.

[0072] On the basis of the above embodiments, when the surface of the photosensitive element of the camera (photosensitive surface) is preferably parallel to the measured surface of the calibration plate, the object plane (from the calibration plate) and the image plane (photosensitive element of the camera) determined by the pinhole imaging principle are kept parallel, so as to facilitate the establishment of similar triangles for triangular distance measurement calculation, thereby utilizing the triangulation distance measurement method commonly used in single-point line laser distance measurement to speed up the calibration. At this time, it may be necessary to decompose the structured light plane into multiple single-point structured lights (structured lights emitted by single-point structured light emitters). For example, the structured light (straight-line structured light) is divided into single-point structured lights in multiple directions according to a preset angle (just like a straight-line laser is regarded as divided into multiple single-point line lasers), and then the triangulation distance measurement method (pinhole imaging principle) is used for each single-point structured light to speed up the calibration, so that the triangulation distance measurement methods in different directions can calibrate the corresponding modeling parameters.

[0073] Preferably, the calibration device further comprises a stepper motor, the calibration plate is mounted at one end of the slide rail, and the structured light module is mounted at the other end of the slide rail, then the stepper motor can drive the calibration plate to move away from the baseline of the structured light module (the line between the center of the structured light emitter and the center of the camera) at a certain step, or to approach the baseline of the structured light module (the line between the center of the structured light emitter and the center of the camera) at a certain step. Specifically, the chip controls the stepper motor to drive the calibration plate to slide on the slide rail; each time the calibration plate slides a fixed distance along the slide rail, the camera captures the calibration plate image once, the imaging position of the light bar in the imaging plane is offset by a preset fixed height along the height direction, and the corresponding projected light point pixel coordinates are offset by a fixed longitudinal offset in the opposite direction of the height direction in the pixel plane. Therefore, the calibration model with the structured light emitter as the main perspective obtains the light point pixel coordinates and their offsets required to construct the structured light ranging relationship on the same light bar, the offset of the imaging position of the light bar in the imaging plane, and the distance the calibration plate slides along the slide rail.

[0074] Therefore, in this embodiment, a motor, a calibration plate, a slide rail, and a clamp for fixing the sensor in the structured light module constitute a calibration device for calibrating the structured light module under the drive of the motor, wherein the structured light module can collect two-dimensional images of the calibration plate at different distances, and the light strips in a certain frame of the two-dimensional image will be clearly imaged in the imaging plane of the camera and will not deviate from the imaging plane of the camera (the surface of the photosensitive element). When the structured light module changes, including at least one of the focal length of the camera, the relative position between the structured light emitter and the camera, the emission direction of the structured light emitter (its change will affect whether a light strip can be projected on the calibration plate at a certain distance), and the optical axis direction of the camera, the calibration plate needs to be driven by a motor to slide to the corresponding position before the structured light emitted by the structured light emitter can be incident on the calibration plate and the reflected light can be imaged through the lens of the camera to the surface of the camera's photosensitive element. Only then can the camera capture the image containing the light strip, and then the camera can be calibrated. Therefore, it is necessary to constrain the sliding distance of the calibration plate on the slide rail, such as constraining the calibration plate to slide only one step distance each time under the drive of the motor to meet the camera imaging requirements. Preferably, in the case where the camera is a distortion-free camera, when the surface of the camera's photosensitive element is at a certain oblique angle to the measured surface of the calibration plate, the distance between the baseline and the calibration plate can be directly calculated according to the pinhole imaging principle, which will not affect the calibration accuracy of the camera.

[0075] In summary, the calibration plate calibrates the structured light module under the drive of the motor. There is no need to specially set up targets such as a checkerboard. The calibration can be performed by simply controlling the calibration plate to slide along the slide rail, which is easy to automate. This improves the efficiency of the structured light module calibration while ensuring the accuracy of the calibration results of the structured light module.

[0076] Preferably, the calibration plate is arranged on the normal plane of the slide rail; the row pixel position in the pixel plane is parallel to the placement plane of the slide rail so that the line connecting a row of pixel points corresponding to the light strip is parallel to the placement plane of the slide rail, and the column pixel position in the pixel plane is regarded as perpendicular to the slide rail, wherein the slide rail can be placed on a horizontal plane; the structured light emitted by the structured light emitter is a straight-line structured light. Therefore, in the process of the calibration plate sliding along the slide rail, the intersection line between the structured light emitted by the structured light emitter and the calibration plate (the light strip) can be parallel to the horizontal plane, and the optical axis direction of the camera can be parallel to the slide rail. This realizes calibration for the longitudinal axis direction of the pixel coordinate system or the Y-axis direction of the imaging plane coordinate system while reducing coordinate system conversion or angle conversion, thereby improving the efficiency of calibration.

[0077] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calibrating a structured light module, characterized in that: The structured light module includes a structured light emitter and a camera; the calibration method includes the following steps: Step 1: Set the position of the calibration plate, structured light emitter and camera; Step 2: Control the structured light emitter to emit structured light, project it onto the calibration plate and form a light strip, and control the camera to collect the image of the calibration plate; and control the displacement of the calibration plate relative to the structured light module so that the camera can obtain images of the calibration plate at different distances; Step 3: Based on the structured light ranging relationship constructed under the pinhole imaging principle and the calibration plate images at different distances, the modeling parameters required to construct the structured light ranging relationship are calculated.

2. The calibration method according to claim 1, characterized in that: The step 1 specifically includes: Adjust the emission direction of the structured light emitter to form an angle with the optical axis direction of the camera, and set the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate; set the emission direction of the structured light emitter to intersect the calibration plate, so that the structured light emitted by the structured light emitter in step 2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera; The camera collects the imaging of the reflected light of the structured light in the photosensitive element of the camera, which is represented by the imaging of the light strip in the imaging plane of the camera; the imaging of the light strip in the imaging plane of the camera is a part of the calibration plate image.

3. The calibration method according to claim 2, characterized in that: The step 2 comprises: The calibration plate is driven to slide on the slide rail; each time the calibration plate slides a certain distance along the slide rail, the camera collects an image of the calibration plate, the light bar is offset once on the calibration plate, the camera detects in real time the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane, and the structured light module detects in real time the distance between the baseline and the calibration plate; Wherein, the line between the center of the structured light emitter and the optical center of the camera is set as the baseline; The relative position between the structured light emitter and the camera is fixed, and the structured light module is installed at one end of the slide rail.

4. The calibration method according to claim 3, characterized in that: In step 2, during the sliding of the calibration plate along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures an image of the light strip, and the imaging position of the light strip in the imaging plane is offset by a preset height in the height direction, and the corresponding projected light spot pixel coordinates are offset once in the opposite direction of the height direction in the pixel plane; When the calibration plate slides to a position on the slide rail, an offset formed by the imaging position of the light strip in the imaging plane relative to the reference position is marked as the light strip offset.

5. The calibration method according to claim 4, characterized in that: When the calibration plate slides along the slide rail to the upper limit of the range of the slide rail, the imaging position of the light strip in the imaging plane is set as the reference position, wherein the upper limit of the range of the slide rail is the position farthest from the structured light module to which the calibration plate is allowed to slide on the slide rail; Alternatively, the reference position is the imaging position in the camera of the reflected light of the structured light emitted by the structured light emitter on the measured surface of the measured object farthest from the structured light module within the detectable area of ​​the structured light module; wherein the measured object includes the calibration plate.

6. The calibration method according to claim 5, characterized in that: In step 3, the method of constructing a structured light ranging relationship according to the pinhole imaging principle includes: Set the length of the baseline to the baseline distance, which is predetermined; Set the distance between the baseline and the calibration plate as the test distance; The constructed structured light ranging relationship is: Px=k0 / d+k1, where d is the test distance and Px is the pixel coordinate of the light point; The modeling parameters required to construct the structured light ranging relationship include k0 and k1.

7. The calibration method according to claim 6, characterized in that: In step 3, based on the structured light ranging relationship constructed under the pinhole imaging principle and the calibration plate images at different distances, a method for calculating the modeling parameters required to construct the structured light ranging relationship includes: During the process of the calibration plate sliding along the slide rail from the upper limit of the measuring range to the lower limit of the measuring range, or during the process of the calibration plate sliding along the slide rail from the lower limit of the measuring range to the upper limit of the measuring range, there exists: The calibration plate slides along the slide rail to the first position, and the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as Px1, and at the same time determines the test distance as d1; then constructs equation 1: Px1=k0 / d1+k1; The calibration plate slides along the slide rail to the second position, and the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as Px2, and at the same time determines the test distance as d2; then constructs equation 2: Px2=k0 / d2+k1; Then, equation 1 and equation 2 are combined to solve the modeling parameters k0 and k1, and the calibration of the structured light module is completed.

8. Wherein, the distance between the second position and the baseline is not equal to the distance between the first position and the baseline.

9. The calibration method according to claim 7, characterized in that: Based on the pinhole imaging principle, the ratio of the baseline distance to the test distance is equal to the ratio of the light strip offset to the focal length of the camera.

10. The calibration method according to claim 8, characterized in that: The light strip offset is equal to the sum of the product of the light spot pixel coordinates and the single pixel size and the projection error; The single pixel size and the projection error are both predetermined; The imaging of the light strip in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light strip offset are both set in the imaging plane coordinate system; The pixel coordinates of the light spot are set in the pixel coordinate system, which is the coordinate system of the pixel plane of the camera.

11. The calibration method according to claim 9, characterized in that: After the light strip is imaged by the camera, it is represented by multiple columns of pixels in the pixel plane. Each column of pixels indicates the pixel position of each point cloud point that constitutes the light strip projected in the camera's photosensitive element. Each column of pixels forms a corresponding measurement angle relative to the optical axis of the camera.

12. The calibration method according to claim 10, characterized in that: There is at least one pixel coordinate of the light point in a column of pixels; a structured light ranging relationship is constructed for each column of pixels in step 3 and corresponding modeling parameters are obtained, and the structured light ranging relationship constructed for each column of pixels in step 3 corresponds to a measurement angle.

13. A chip, characterized in that: The chip is used to execute the calibration method described in any one of claims 1 to 11.

14. A calibration device, characterized in that: The calibration device is used to calibrate the structured light module described in any one of claims 1 to 11, and the calibration device includes a calibration plate, a slide rail and the chip described in claim 12, wherein the chip is used to control the calibration plate to slide along the slide rail, and control the structured light emitter to emit structured light to the measured surface of the calibration plate, and at the same time control the camera to collect the image of the structured light projected onto the measured surface of the calibration plate, so as to calculate the modeling parameters by executing steps 1 to 3, and complete the calibration of the structured light module; wherein the structured light module includes a structured light emitter and a camera.

15. The calibration device according to claim 13, characterized in that: The calibration device also includes a fixture, which is used to fix the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate, and is also used to assemble the emission direction of the structured light emitter and the optical axis direction of the camera to form an angle; the distance between the center of the structured light emitter and the center of the camera is fixed; The emission direction of the structured light emitter intersects with the calibration plate, so that the structured light emitted by the structured light emitter is incident on the calibration plate at a certain angle to the normal line of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera; The camera collects the image of the reflected light of the structured light in the photosensitive element of the camera, which is represented by the image of the light strip in the imaging plane of the camera; the image of the light strip in the imaging plane of the camera is a part of the image of the calibration plate; The camera is configured to avoid distortion.

16. The calibration device according to claim 14, characterized in that: The calibration device also includes a stepper motor, a calibration plate is mounted on one end of the slide rail, and a structured light module is mounted on the other end of the slide rail; The chip controls the stepper motor to drive the calibration plate to slide on the slide rail; every time the calibration plate slides a fixed distance along the slide rail, the camera captures an image of the calibration plate, and the imaging position of the light bar in the imaging plane is offset by a preset fixed height along the height direction, and the corresponding projected light spot pixel coordinates are offset once in the opposite direction of the height direction in the pixel plane.

Citation Information

Cited By

  • Method for rapidly detecting thickness of double-sided tape

    CN122170783A

  • A rapid method for detecting the thickness of double-sided tape

    CN122170783B