Structural light module modeling parameter calibration method, chip and calibration device

Abstract

The application discloses a modeling parameter calibration method of a structured light module, a chip and a calibration device, and comprises the following steps: step 1, a structured light transmitter is controlled to emit structured light, which is projected onto a calibration board to form a light strip; the calibration board is displaced relative to the structured light module, a plurality of calibration board images are obtained, and the light strip center coordinates of each calibration column are extracted from the same calibration board image, and the corresponding test distance is determined; step 2, in the obtained plurality of calibration board images, the light strip center coordinates of the same calibration column and the test distance corresponding to the same calibration column are nonlinearly fitted to obtain the modeling parameters corresponding to the same calibration column; step 3, the light strip center coordinates of each calibration column in each calibration board image obtained and the test distance corresponding thereto are repeatedly executed in step 2 to sequentially obtain the modeling parameters corresponding to all calibration columns. The efficiency of calibration of the structured light module is improved.

Description

Technical Field

[0001] This application relates to the technical field of camera calibration, and in particular to a method, chip, and calibration device for calibrating modeling parameters of a structured light module. Background Technology

[0002] Structured light vision systems are measurement systems combining line lasers and monocular cameras. For example, a line laser sensor can be composed of a camera and a line laser. Before using a structured light vision system, camera calibration and laser plane calibration are required to enable 3D measurement using imaging principles and laser plane parameters. This calibration method is applicable to most cameras. The line laser projects laser light onto the surface of the object being measured. The laser light is diffusely reflected and enters the camera's receiving lens, where it forms an image on the camera's photosensitive element. The position of the image changes with the distance to the target. By calculating the relative position of the image on the camera's photosensitive element, the relative height of the object being measured can be obtained. However, this method requires specially designed calibration targets to simultaneously calibrate the camera's intrinsic and extrinsic parameters, increasing the complexity of the calibration calculation process, the difficulty of fitting, and resulting in relatively low camera calibration efficiency. Summary of the Invention

[0003] This application discloses a method, chip, and calibration device for modeling parameters of a structured light module. The specific technical solution is as follows: A method for calibrating modeling parameters of a structured light module, which includes a structured light emitter and a camera, is disclosed. The method comprises the following steps: Step 1: Controlling the structured light emitter to emit structured light, which is projected onto a calibration plate to form light stripes. Simultaneously, controlling the calibration plate to shift relative to the structured light module, the camera acquires images of the calibration plate at different test distances, obtaining multiple frames of calibration plate images. The center coordinates of the light stripes in each calibration column are extracted from the same frame of the calibration plate image, and the test distance corresponding to the same calibration column in the same frame of the calibration plate image is determined. Each frame of the calibration plate image includes light stripes. Step 2: In the multiple frames of calibration plate images obtained in Step 1, nonlinear analysis is performed on the center coordinates of the light stripes in the same calibration column and the test distance corresponding to the same calibration column. Step 3: By repeatedly executing Step 2, the center coordinates of the light stripes in each calibration column of each frame of calibration board image obtained in Step 1 and their corresponding test distances are traversed, and the modeling parameters corresponding to all calibration columns are obtained in sequence, and the calibration of the modeling parameters of the structured light module is completed. The number of times Step 2 is executed is equal to the number of calibration columns in a frame of calibration board image, and the number of calibration columns in each frame of calibration board image is equal. The modeling parameters corresponding to the same calibration column are the coefficients required to construct the structured light ranging relationship under the same calibration column. The structured light ranging relationship is the relationship between the center coordinates of the light stripes in the same calibration column and their corresponding test distances, which is constructed based on the pinhole imaging principle.

[0004] In summary, steps 1 to 3 of this embodiment involve controlling the displacement of the calibration plate relative to the structured light module and using the coordinates of the light stripe centers of different calibration columns in the calibration plate images acquired at different distances and their corresponding test distances for nonlinear fitting to complete parameter calibration. The fitted modeling parameters can replace the traditional camera intrinsic parameters in one direction or one test angle relative to the camera's optical axis, thereby improving the efficiency of structured light module calibration while ensuring the accuracy of modeling parameter calibration, i.e., ensuring the accuracy of the structured light module's calibration parameters. Furthermore, based on the modeling parameters corresponding to each calibration column fitted in this application and the structured light ranging relationship that can be constructed, the ranging modeling method between structured light and the camera, or the camera calibration method, is simplified to simply fitting and calculating the modeling parameters corresponding to each calibration column. This replaces or eliminates the need for camera intrinsic parameter calibration before the ranging operation, reducing the computational load of camera calibration and improving the calibration efficiency of the structured light module. Consequently, after obtaining the modeling parameters, modeling and ranging operations can be performed without calibrating the camera's intrinsic parameters, thus improving computational efficiency.

[0005] A chip for executing the modeling parameter calibration method, serving as the execution entity for steps 1, 2, and 3 in the foregoing embodiments.

[0006] A calibration device includes a calibration plate, a slide rail, and a chip. The calibration device is used to calibrate the structured light module. The chip controls the calibration plate to slide along the slide rail and controls the structured light emitter to emit structured light onto the surface under test of the calibration plate. Simultaneously, it controls a camera to acquire images of the structured light projected onto the surface under test of the calibration plate. This allows for the acquisition of modeling parameters corresponding to each calibration column in each frame of the calibration plate image by executing steps 1 to 3, and confirms the completion of the calibration of the modeling parameters of the structured light module. Attached Figure Description

[0007] Figure 1 This is a flowchart of a method for calibrating modeling parameters of a structured light module, as disclosed in one embodiment of this application.

[0008] Figure 2 This is a schematic diagram of the optical path principle of a single-point line laser disclosed in one embodiment of this application.

[0009] Figure 3 This is a schematic diagram showing the relative positions of a structured light emitter, a camera, and the object being measured, as disclosed in one embodiment of this application. Implementation

[0010] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. To further illustrate the embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. Those skilled in the art will understand that the goal of camera calibration is to find a suitable or approximate mathematical model, calculate the parameters of this model, and realize the expression of a complex imaging process using a simple mathematical model.

[0011] Considering practical applications, structured light vision systems typically employ small-angle cameras to meet the pinhole imaging principle. Furthermore, the forward emission direction of the line laser and the camera's optical axis are at a certain angle, and the line laser in the optical path system is often obliquely incident on the surface of the object being measured. To reduce calibration complexity and improve calibration efficiency, this application aims to provide a rapid calibration method for structured light modules composed of small-angle, distortion-free cameras that meet the pinhole model, independent of camera intrinsic parameter calibration. This method utilizes light stripe image data from calibration plates at different distances to fit the parameters required for the ranging function of the structured light module, thus simplifying the calibration steps for both camera intrinsic and extrinsic parameters.

[0012] As one embodiment, a method for calibrating modeling parameters of a structured light module is disclosed. The execution entity of the modeling parameter calibration method can be a controller built into the structured light module or an external control device. The structured light module includes a structured light emitter and a camera. Figure 1 As shown, the modeling parameter calibration method includes the following steps: Step 1: Control the structured light emitter to emit structured light, project it onto the calibration plate, and form a light stripe, for example, Figure 3 The structured light emitted by the structured light emitter in the image is considered to be composed of multiple single-point line lasers with continuous angles, in sequence as follows: Figure 3The emitted light rays 301, 302, and 303, etc., are laser rays emitted from the light source of the structured light emitter. The light path is altered by the camera lens and a wave mirror, causing the structured light emitted by the structured light emitter to form a structured light surface. This can be understood as the emitted light having an approximately planar effect. The structured light surface intersects with the surface of the object being measured to form light stripes as shown in the diagram. The camera is controlled to acquire images of the calibration board, including the original image of the light stripes. A single-point line laser can be considered as emitted by a single-point line laser emitter. For example, one single-point line laser emits one single-point line laser, and multiple single-point line lasers emit multiple single-point line lasers, thus forming a single-line laser. A laser plane can be emitted, and the emitted laser plane can be regarded as being composed of multiple single-point line lasers. Thus, a single-line laser can be equivalently decomposed into multiple single-point line lasers. Similarly, the structured light emitter disclosed in this application can be regarded as being composed of multiple single-point line laser emitters. Each single-point line laser emitter emits one single-point line laser, and multiple single-point line lasers form a structured light surface. That is, the structured light emitter emits a structured light surface and intersects with the calibration plate to form a light stripe. When using single-point laser triangulation for ranging, that is, when using the pinhole imaging principle (pinhole model) to establish a mathematical model or for ranging, it is necessary to decompose the structured light into multiple single-point line lasers to form the corresponding single-point line laser light path.

[0013] Furthermore, in step 1, the calibration board is also controlled to move relative to the structured light module. This can be achieved by a motor driving the calibration board to move relative to the camera, allowing the camera to obtain images of the calibration board at different distances. Each frame of the calibration board image includes light bars. Generally, the calibration board moves away from or closer to the camera in one direction.

[0014] The camera acquires calibration board images at different test distances, obtaining multiple frames of calibration board images. Each frame of calibration board image is an image of the calibration board at a different distance relative to the structured light module at the same position, and all of them include light bars, that is, each frame of calibration board image includes the imaging information of light bars. Furthermore, for each frame of the calibration board image acquired, the number of calibration columns set in the pixel plane of the camera is equal for each frame of the calibration board image. For each frame or multiple frames of calibration board images acquired by the camera, the center coordinates of the light stripes of each calibration column are extracted from the same frame of the calibration board image. At the same time, the test distance corresponding to the same calibration column in the same frame of the calibration board image is determined. This test distance can be understood as the distance between the light stripe and the structured light module at the angle or direction corresponding to the calibration column. In step 1, the structured light module can actively or under the command of an external controller extract height calibration data, X-axis or Y-axis components of the light stripe point cloud, X-direction calibration data of the black and white alternating areas or white areas in the calibration board from the calibration board image. This data can be used to calculate the row pixel position information of the light stripe (or the center line of the light stripe) or extract the centroid position of each column of the light stripe, including the center coordinates of the light stripe, to provide input information for subsequent fitting relationships and calculation of modeling parameters.

[0015] In some embodiments, a certain gap is left between the calibration plate and the structured light module before the calibration plate is displaced. This allows the camera to acquire two-dimensional images of the calibration plate at multiple distances when the calibration plate is displaced relative to the structured light module. This enables the establishment of a more comprehensive calibration plate image library and the acquisition of pixel position information at more distances, including the center coordinates of the light stripes at different distances. This allows for the fitting of the formula required for the pinhole imaging principle, obtaining the corresponding optimal fitting parameters, which can replace the camera's intrinsic parameters and simplify the camera calibration method.

[0016] Step 2: In the multi-frame calibration board images obtained in Step 1, nonlinear fitting is performed on the center coordinates of the light stripe in the same calibration column and the test distance corresponding to the same calibration column to obtain the modeling parameters corresponding to the same calibration column and reduce fitting error. The same calibration column is a column with the same order in each frame of calibration board images, which represents the projection position information of the light stripe in the pixel position. Specifically, it is the pixel coordinates projected by the light stripe in an angle or direction in each frame of calibration board images. Then, the coefficients required to construct the structured light ranging relationship under the pinhole imaging principle are obtained by nonlinear fitting of the test distance corresponding to the same calibration column in each frame of calibration board images. This minimizes the fitting error generated in the same calibration column in each frame of calibration board images, ensuring fitting accuracy, which is equivalent to ensuring the accuracy of the camera's intrinsic parameters.

[0017] In this embodiment, the modeling parameters corresponding to the same calibration column are the coefficients required to construct the structured light ranging relationship under the same calibration column. The structured light ranging relationship is the relationship between the center coordinates of the light stripe in the same calibration column and its corresponding test distance, constructed based on the pinhole imaging principle. Constructing / defining the structured light ranging relationship under the pinhole imaging principle can be specifically understood as the relationship between the center coordinates of the light stripe and its corresponding test distance, constructed based on the pinhole imaging principle. By combining data from calibration board images at different distances, the coefficients required to construct the structured light ranging relationship under the same calibration column can be fitted. In this embodiment, the structured light ranging relationship can be used as the corresponding function to be fitted in nonlinear fitting to fit the modeling parameters, including at least two fitting coefficients. Therefore, compared to the existing line laser-camera modeling ranging scenario, this embodiment converts the mathematical formula constructed by triangular ranging into the relationship between the center coordinates of the light stripe and its corresponding test distance, without requiring additional correction of the camera's intrinsic and extrinsic parameters, resulting in less calibration computation.

[0018] In this embodiment, during the nonlinear fitting process of the center coordinates of the light stripe in the same calibration column and the test distance corresponding to the same calibration column, the optimal relationship between the center coordinates of the light stripe and the test distance will be fitted and the modeling parameters will be obtained. The center coordinates of the light stripe and the test distance (the distance between the calibration board and the structured light module) extracted from the calibration board images at different distances are known quantities that can be obtained in real time from each frame of calibration board images. Therefore, using these known quantities, the coefficients required to construct the structured light ranging relationship can be fitted and calculated from the image data of multiple frames of calibration board images, and a mathematical fitting model can be established for the ranging of the structured light module.

[0019] It should be noted that the test distance refers to the distance between the calibration board and the structured light module. Specifically, it is the distance between the line connecting the optical center of the camera and the center of the structured light emitter and the calibration board (the surface of the calibration board being measured), which is considered the ranging distance of the structured light module. The structured light emitted by the structured light emitter is a linear structured light beam, which forms a beam when projected onto the surface of the object being measured. Figure 3 The structured light beam shown is a linear beam. This linear beam can be approximately decomposed into multiple single-point line lasers, which can be multiple single-point line lasers with continuous angles. Each single-point line laser can be considered a straight line beam. If the structured light emitter is decomposed into multiple single-point line laser emitters, the test distance is configured as the distance between the line connecting the center of the single-point line laser emitter and the optical center of the camera and the calibration plate. For the light stripe, Figure 3In the two-dimensional light stripe image of the intersection line between the structured light surface and the surface of the object under test, the center coordinates of the light stripe can be extracted, including the X-axis component and the Y-axis component, which correspond to the U-axis component (represented by the number of pixels) and the V-axis component (represented by the number of pixels) of the pixel coordinate system. These coordinates originate from the result of the linear structured light or multiple single-point line laser projections. In this embodiment, the center coordinates of the light stripe can represent pixel coordinates unaffected by distortion.

[0020] Therefore, by performing nonlinear fitting on the mapping relationship between the center coordinates of the light stripes and their test distances in the same calibration column of multiple calibration board images in step 2, the aforementioned modeling parameters can be calculated as the optimal parameters for constructing the mapping relationship between the center coordinates of the light stripes and their test distances in the same calibration column, i.e., the coefficients required to construct the structured light ranging relationship. The fitted modeling parameters can replace the traditional camera intrinsic parameters in one direction or one test angle relative to the optical axis of the camera, so as to replace or eliminate the calibration of the camera intrinsic parameters before the ranging operation.

[0021] Step 3: By repeatedly executing Step 2, the center coordinates of the light stripes in each calibration column of each frame of calibration board image obtained in Step 1 and their corresponding test distances are traversed. This process sequentially obtains the modeling parameters corresponding to all calibration columns and confirms the completion of the calibration of the structured light module's modeling parameters. The number of executions in Step 2 is equal to the number of calibration columns in one frame of calibration board image, and the number of calibration columns in each frame of calibration board image is equal. Thus, by repeatedly executing Step 2, Step 3 can sequentially traverse all frames of calibration board image to obtain each calibration column within the same frame and determine the corresponding test distances on the calibration columns. It can also complete the traversal of each calibration column (or all pixel columns) within the same frame of calibration board image to extract the center coordinates of the light stripes in each calibration column. To obtain the modeling parameters corresponding to the same calibration column, it is necessary to first traverse the center coordinates of the light stripes in the same calibration column of all frames of calibration board image, and then traverse the center coordinates of the light stripes in the next calibration column of all frames of calibration board image. Based on this traversal order, the modeling parameters corresponding to all calibration columns are obtained sequentially through nonlinear fitting in step 2, and the calibration of the modeling parameters of the structured light module is completed, including the calibration of the modeling parameters on all test angles or all calibration columns, thus confirming the completion of the calibration of the structured light module.

[0022] In summary, steps 1 to 3 of this embodiment involve controlling the calibration board to move relative to the structured light module, and using the coordinates of the light stripe centers of different calibration columns in the calibration board images acquired at different distances and their corresponding test distances to perform nonlinear fitting to complete the parameter calibration. The fitted modeling parameters can replace the traditional camera intrinsic parameters in one direction or one test angle relative to the optical axis of the camera, thereby improving the efficiency of the structured light module calibration while ensuring the accuracy of the calibration parameters.

[0023] Furthermore, based on the modeling parameters corresponding to each calibration column fitted in this application and the structured light ranging relationship that can be constructed, the ranging modeling method between structured light and camera or the camera calibration method is simplified to simply fitting and calculating the modeling parameters corresponding to each calibration column. This replaces or eliminates the calibration of camera intrinsic parameters before ranging operations, reducing the amount of camera calibration calculation and improving the calibration efficiency of structured light modules. Consequently, after obtaining the modeling parameters, modeling and ranging operations can be performed without calibrating the camera intrinsic parameters, thus improving computational efficiency.

[0024] In one embodiment, after the light stripe is imaged by the camera, it is represented by multiple columns of pixels (i.e., multiple calibration columns of pixels) in the pixel plane. Each column of pixels identifies the pixel position projected onto the camera's photosensitive element by each point cloud point that makes up the light stripe. Each column of pixels contains the center coordinates of the light stripe in the pixel coordinate system. Therefore, in this embodiment, the camera can detect the center coordinates of the light stripe in each column formed by the projection of the light stripe onto the pixel plane in real time. Each light stripe contained within the calibration plate image can be represented by multiple columns of pixels (i.e., multiple calibration columns of pixels) in the pixel coordinate system. Although the surface to be measured projected onto the object being measured or the surface to be measured on the calibration plate is... Figure 3 The linear light rays shown are within the angle range formed by the single-point line laser 303 and the single-point line laser 301 emitted from the structured light emitter. The corresponding light paths are all within the camera's field of view. However, the number of calibration columns formed by the light stripes and the distribution of calibration columns projected onto the pixel plane by the same light stripe can be determined according to the camera resolution of the structured light module, the computing power of the structured light module's controller, and the camera's intrinsic and extrinsic parameter specifications. This embodiment does not impose any restrictions.

[0025] Understandably, the structured light emitted by the structured light emitter uses a camera to alter the optical path, creating a structured light surface. This surface can be considered equivalent to multiple single-point laser lines decomposed according to the test angle. The structured light emitter itself has only one light source; the optical path is altered using lenses and wave mirrors to create an approximately planar effect, equivalent to a structured light surface. Figure 3The structured light surface is shown; for example, when the structured light is a line laser, the line laser can be regarded as a collection of multiple single-point line lasers, rather than being physically composed of multiple single-point line lasers. Within the structured light surface, a single-point line laser forms a corresponding test angle relative to the optical axis of the camera, and a single-point line laser forms the center coordinates of a light stripe within a calibration column of the camera's pixel plane; the light stripe is the intersection line between the structured light surface and the measured surface of the calibration plate. To form the light stripe, the calibration plate can also be the object being measured, and the measured surface of the calibration plate is the measured surface of the object being measured.

[0026] In this embodiment, the light spots of the light stripe at each test angle are projected onto the pixel position of the camera's photosensitive element as the pixel position in the corresponding calibration column of the pixel plane, so that the light stripe is represented by pixels in multiple calibration columns in the pixel plane; each calibration column corresponds to a test angle, that is, the pixels in each calibration column also correspond to the light spots of the light stripe at the corresponding test angle. Each column of pixels forms a corresponding test angle relative to the camera's optical axis. Within the pixel coordinates of a column of projected pixels, there exists a center coordinate of the light stripe. In other words, within the pixel coordinates of a calibration column, there exists a center coordinate of the light stripe. Each center coordinate of the light stripe corresponds to a test angle, representing the azimuth angle formed by a point on the test surface of the calibration plate relative to the camera's optical axis. The structured light emitted by the structured light emitter changes its optical path after passing through the camera, forming a structured light surface. Based on the determination of multiple test angles, it can be considered as composed of multiple single-point line lasers. Within the structured light surface, a single-point line laser forms a corresponding test angle relative to the camera's optical axis. The projection of a single-point line laser onto a calibration column within the camera's pixel plane (pixel coordinate system) forms the center coordinate of the light stripe, ensuring that each pixel in the calibration column is located at the pixel position formed by the projection of a single-point line laser onto the pixel plane after reflection from the calibration plate. Since the structured light surface can be approximated as a combination of countless single-point line lasers emitted from the structured light emitter, the center coordinates of the light strips in the same calibration column and the corresponding test distances in the same calibration column can be nonlinearly fitted according to step 2 to obtain the corresponding modeling parameters at a finite number of test angles, thereby obtaining the distance information of the measured object at a finite number of test angles of the camera, which can meet some simple contour point ranging application scenarios.

[0027] Preferably, since the structured light emitted by the structured light emitter can be regarded as forming a structured light surface, and the light strip is located at the intersection of the structured light surface and the measured surface of the calibration plate, the projection position of each laser point that makes up the light strip in the photosensitive element of the camera can be connected into a horizontal projection line segment in the pixel coordinate system; thus, it reflects that the structured light is a straight structured light and the light strip is parallel to the slide rail, which reduces the pose conversion complexity required for calibration and improves the efficiency of calibration.

[0028] Based on the above embodiments, since each calibration column of pixels identifies the pixel position projected onto the camera's photosensitive element by each light spot that makes up the light stripe, a column of pixels (i.e., pixels in a calibration column) forms a corresponding test angle relative to the camera's optical axis, a single-point laser line forms a corresponding test angle relative to the camera's optical axis, and a single-point laser line is projected into a column of the camera's pixel plane to form the center coordinates of the light stripe, each column of pixels has at least one of the light stripe center coordinates and is located at a corresponding test angle relative to the camera's optical axis. Therefore, the center coordinates of the light stripe in each calibration column (derived from the corresponding column pixels) are obtained in step 2 through nonlinear fitting. These parameters are the coefficients required to construct the structured light ranging relationship under the same calibration column, so that the structured light ranging relationship under the same calibration column forms the fitting function corresponding to the nonlinear fitting in step 2. Therefore, the structured light ranging relationship under each calibration column corresponds to a test angle, so that the structured light emitter is equivalent to being decomposed into multiple single-point line laser emitters. Each single-point line laser emitter emits a single-point line laser. The test distance corresponding to each test angle is updated to the distance between the line connecting the center of the single-point line laser emitter and the optical center of the camera and the calibration plate. Therefore, steps 1 to 3 are executed sequentially on the ranging system composed of the single-point line laser emitter and the camera at each test angle.

[0029] It should be noted that the modeling parameters corresponding to the same calibration column are the coefficients required to construct the structured light ranging relationship under the same calibration column. The structured light ranging relationship is the relationship between the center coordinates of the light stripe of the same calibration column and its corresponding test distance, which is constructed based on the pinhole imaging principle.

[0030] Based on the embodiments described in steps 1 to 3 above, when using a camera that satisfies the pinhole imaging principle, each column of pixels on the pixel plane can be approximated as an image seen from the same vertical angle of the structured light surface. At this vertical angle, there exists a single-point line laser emitter (similar to the function of a single-point laser) "extracted" from the linear structured light (the light strip). This vertical angle is the aforementioned test angle. Thus, by executing steps 1 to 3, it can be modeled using two modeling parameters (corresponding to the construction of the structured light ranging relationship, and one structured light ranging relationship is constructed for each vertical angle). Similarly, if there are N columns of pixels in the pixel plane where the calibration board image is located, the center coordinates of N light stripes can be extracted from them. Steps 1 to 3 are executed sequentially to model the ranging system composed of the single-point line laser emitter and the camera at each test angle, that is, the structured light ranging relationship is constructed at each test angle to obtain the corresponding modeling parameters, avoiding the need to calibrate the camera intrinsic parameters first; this also completes the ranging modeling of the entire structured light module.

[0031] As one embodiment, before performing step 1, the emission direction of the structured light emitter is configured to form an angle with the optical axis of the camera, and the line connecting the center of the structured light emitter and the optical center of the camera is set to be parallel to the measured surface of the calibration plate. The structured light emitted by the structured light emitter is arranged to intersect with the measured surface of the calibration plate, and the intersection line forms the light stripe in the measured surface of the calibration plate, so that the structured light emitted by the structured light emitter is incident on the calibration plate at a certain angle with the normal of the measured surface of the calibration plate, and the reflected light is imaged into the camera's photosensitive element through the camera's lens. The positions of the calibration plate, structured light emitter, and camera are set, including determining the relative positional relationship between any two devices, to prepare the device for camera calibration. The camera collects the image of the reflected light of the structured light in the camera's photosensitive element, and uses the image of the light stripe in the camera's imaging plane to represent it; the image of the light stripe in the camera's imaging plane is a part of the calibration plate image, so that the calibration plate image forms a two-dimensional structured light stripe image. To reduce distortion calculations, the camera included in the structured light module is a small-angle, distortion-free camera, satisfying the pinhole imaging principle and ensuring the reliability of the camera's intrinsic parameters. The camera's internal image sensor / photosensitive element can collect reflected light from the structured light through an oblique optical path. A photosensitive element is a semiconductor device that converts a light image into an electronic signal. The distance between the surface of the photosensitive element and the optical center of the camera is equal to the focal length. The mass of the photosensitive molecules on the surface of the photosensitive element is called a pixel. Generally, the structured light module uses the triangular reflection principle; the surface of the photosensitive element can be considered as a matrix, thus allowing the projection of a light stripe corresponding to the structured light. Simultaneously, the displacement of all light spots on the light stripe in the calibration plate, as well as the offset of the light stripe's center coordinates in the pixel plane, can be measured.

[0032] It should be noted that the structured light emitter can be, but is not limited to, a line laser emitter or other light source that supports the emission of linear structured light. The relative position between the structured light emitter and the camera is fixed, so that the structured light module is fixedly set in one position, forming a sensor device with a fixed detection angle. An initial distance is set between the line connecting the center of the structured light emitter and the optical center of the camera and the calibration plate, serving as the starting point for the displacement of the calibration plate relative to the structured light module. The line connecting the center of the structured light emitter and the optical center of the camera is also set as a fixed baseline, which can form a fixed assembly angle with the displacement direction of the calibration plate.

[0033] As one embodiment, before performing step 1, the emission direction of the structured light emitter is adjusted to form an angle with the optical axis of the camera. This fixed angle creates an oblique optical path during calibration calculations using a calibration plate. The line connecting the center of the structured light emitter and the center of the camera is set parallel to the measured surface of the calibration plate. This can be fixed by a special fixture. Note that it is not required that the camera and calibration plate be parallel, nor is it required that the intersection of the structured light emitted by the structured light emitter and the calibration plate be horizontal. The angle between the emission direction of the structured light emitter and the optical axis of the camera is fixed by this fixture, used for projecting structured light onto the same calibration plate and acquiring images, regardless of whether the calibration plate is displaced. The relative position between the center of the structured light emitter and the center of the camera, and the angle between the emission direction of the structured light emitter and the optical axis of the camera, are fed back to the controller of the structured light module. The controller of the structured light module can fix these relative positions and angles by controlling the fixture.

[0034] Combination Figure 2 and Figure 3 It can be seen that the controller of the structured light module sets the emission direction of the structured light emitter to intersect with the calibration board, so that the structured light emitter... Figure 2 The structured light emitted (usually a single ray, such as a single-point line laser, refer to the optical path of a single-point line laser) is incident on the calibration plate at a certain angle to the normal of the surface being measured. The reflected light is imaged into the camera's sensor through the camera lens, and the camera then captures the image formed by the reflected light. The image captured by the camera of the reflected structured light in the camera's sensor is represented by the image of the light stripe in the camera's imaging plane. The image of the light stripe in the camera's imaging plane is a part of the calibration plate image. When a line laser emitter is used as the structured light emitter, the emitted laser line is structured light, such as a line laser. It can be considered as a laser plane composed of countless laser lines emitted by the same line laser emitter, corresponding to... Figure 3 The structured light surface shown has a laser plane intersecting with the calibration plate to form a light stripe, making the light stripe a line structured light stripe, such as the projection spot of a line laser within the calibration plate. This light stripe can be extracted from the calibration plate image for triangulation (constructing similar triangles to calculate distances). Specifically, each pixel on the light stripe (the projection position of each single-point line laser in the pixel plane or the imaging position in the imaging plane) can be used for triangulation (using the aforementioned modeling parameters to construct a structured light ranging relationship for triangulation calculation), obtaining the distance information of each pixel position relative to the camera within the angle range covered by the light stripe. This maintains the relative pose between the camera and the structured light emitter, stably establishing similar triangles for triangulation distance calculation.

[0035] It should be noted that the above description is only for the convenience of illustrating the relative positions of the components in the structured light module. When the relative positions remain unchanged, the structured light module and the calibration plate do not necessarily have to be placed on a horizontal plane, but can both be placed on a vertical plane. When performing step 1, the calibration plate moves freely or rotates relative to the slide rail under the drive of the motor, and its placement posture is unrestrained.

[0036] Based on the above embodiments, when the surface of the camera's photosensitive element (photosensitive surface) is preferably parallel to the measured surface of the calibration plate, the object plane (from the calibration plate) and the image plane (the camera's photosensitive element) determined by the pinhole imaging principle remain parallel, which facilitates the establishment of similar triangles for triangulation distance calculation. Thus, the triangulation method commonly used in single-point line laser ranging, combined with nonlinear fitting methods (such as the least squares method), is used to accelerate the calibration of modeling parameters. At this time, it may be necessary to decompose the structured light surface into multiple single-point line lasers for processing. For example, the structured light (a straight line of light) is divided into single-point line lasers in multiple directions according to a preset angle. Then, for each single-point line laser, the triangulation method (pinhole imaging principle) and nonlinear fitting are used to accelerate the calibration of the pixel coordinates of the same calibration column, so that the corresponding modeling parameters are calibrated in different directions.

[0037] As one embodiment, in step 1, the method for controlling the displacement of the calibration plate relative to the structured light module includes: the controller of the structured light module can drive the calibration plate to slide on the slide rail via a motor; under the drive of the motor, the camera captures an image of the calibration plate every time the calibration plate slides a certain distance along the slide rail, and the light stripe shifts once on the calibration plate, causing the calibration plate to shift along the height direction. The distance of the shift can be detected in real time by the structured light module; the line connecting the center of the structured light emitter and the optical center of the camera is set as the baseline; the relative position between the structured light emitter and the camera is fixed, the structured light module is fixedly installed at one end of the slide rail, and the initial position of the calibration plate is set at the other end of the slide rail, wherein the calibration plate is slidably set on the slide rail. Each time an image of the calibration board is acquired, the calibration board is located at different positions on the slide rail. When the structured light emitter is turned on, the camera acquires an image containing the light stripe. The camera detects in real time the center coordinates of the light stripe formed by the projection of the light stripe in the pixel plane, extracts the center coordinates of the light stripe formed in the pixel plane from the image of the calibration board, and records the test distance at the corresponding test angle for use in the nonlinear fitting of step 2.

[0038] Specifically, when the calibration plate is set on the normal plane of the slide rail, the height direction of the calibration plate can be considered as the vertical direction of the slide rail, or parallel to the normal plane of the slide rail, and can also be considered as the longitudinal direction of the camera's imaging plane or pixel plane; preferably, the calibration plate is slidably mounted on one end of the slide rail, and the structured light module is fixedly mounted on the other end of the slide rail. Figure 2 As shown, the calibration plate is mounted on the right side of the photosensitive element and supports left and right sliding on the slide rail. The calibration plate is mounted on the right end of the slide rail, and the structured light module is mounted on the left end of the slide rail. Specifically, both the structured light emitter and the camera are mounted on the left end of the slide rail, and the relative positions between the structured light emitter and the camera are fixed by the fixture. Figure 2 The line connecting the center A of the structured light emitter and the optical center B of the camera is set as baseline AB and is fixed, at least during the parameter calibration of the camera and structured light. The slide rail does not necessarily have to be placed on a horizontal plane; it can also be on a vertical plane and can be rotated arbitrarily. For every certain distance the calibration plate slides along the slide rail, the light strip translates once on the calibration plate along the vertical direction of the slide rail. Figure 2It can be seen that the calibration plate starts sliding to the right from a starting position on the slide rail (set at a known distance from the camera). When the calibration plate slides to the point O where the structured light emitted by the structured light emitter (viewed from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point line laser) intersects with the measured surface of the calibration plate, the reflected light at point O is imaged by the optical center B of the camera lens onto the position E of the camera's photosensitive element surface, so that the camera obtains the pixel coordinates at position E in real time, which is represented as the center coordinates of the light strip projected in the pixel plane after the light strip is reflected by point O; at the same time, the vertical distance CO from point O to the baseline AB is also determined, which is represented as a test distance, and is also the distance between the calibration plate at the current position and the baseline AB, which can be represented by the displacement of the calibration plate relative to the structured light module along the slide rail. At this point, the structured light emitted by the structured light emitter (viewed from a two-dimensional plane perspective, generally from the normal plane of the calibration plate's measured surface, considered as emitting a single-point line laser) forms an angle α with the baseline AB. The baseline AB is the line connecting the optical center B of the camera and the center A of the structured light emitter. With the calibration plate set on the normal plane of the slide rail, the baseline AB is parallel to the measured surface of the calibration plate (considered perpendicular to the slide rail). Then, with the angle between the structured light emitted by the structured light emitter (viewed from a two-dimensional plane perspective, generally from the normal plane of the calibration plate's measured surface, considered as emitting a single-point line laser) and the baseline AB fixed at α, the calibration plate is only driven to slide away from the camera until the structured light emitted by the structured light emitter (viewed from a two-dimensional plane perspective, generally from the normal plane of the calibration plate's measured surface, considered as emitting a single-point line laser) is... When a single-point laser beam intersects the test surface of the calibration plate at point O', the distance between point O' and point O is equal to y. The reflected light from point O' is imaged onto the surface of the camera's photosensitive element at position F after passing through the optical center B of the camera lens. That is, as the calibration plate moves away from the fixed camera along the slide rail, the image position of the reflected light on the camera's photosensitive element shifts upwards along the longitudinal axis from position E to position F. The camera obtains the pixel coordinates at position F in real time, representing the center coordinates of the light stripe projected onto the pixel plane after reflection from point O' on the calibration plate. Simultaneously, the vertical distance C'O' from point O' to the baseline AB is determined, representing a test distance, which is also the distance between the calibration plate at the current position and the baseline AB. This distance can be represented by the displacement of the calibration plate relative to the structured light module along the slide rail. Based on this, the distance C'O and the pixel coordinates at position E can be used as a vector group for the nonlinear fitting in step 2, and the distance C'O' and the pixel coordinates at position F can also be used as a vector group for the nonlinear fitting in step 2 to fit relevant modeling parameters.Therefore, the calibration board calibrates the structured light module under the drive of the motor, without the need for special targets such as checkerboard patterns. Calibration can be performed simply by controlling the calibration board to slide along the slide rail, which can be easily automated. Thus, while ensuring the reliability of the calibration results of the structured light module, that is, while ensuring the accuracy of the calibration parameters of the structured light module, the efficiency of the calibration of the structured light module is improved.

[0039] In this embodiment, the calibration plate, slide rail, and fixture for fixing the sensor in the structured light module can form a calibration device for calibrating the structured light module under the drive of a motor. The structured light module can acquire two-dimensional images of the calibration plate at different distances. There will be light stripes in a certain frame of two-dimensional image that are clearly imaged in the imaging plane of the camera and will not deviate from the imaging plane of the camera (the surface of the photosensitive element). When changes occur in the structured light module, including changes in the camera's focal length, the relative position between the structured light emitter and the camera, the emission direction of the structured light emitter (which affects whether a light stripe can be projected onto the calibration plate at a certain distance), and the optical axis direction of the camera, the calibration plate needs to slide to the corresponding position under the drive of the motor before the structured light emitted by the structured light emitter can be incident on the calibration plate and the reflected light can be imaged onto the surface of the camera's photosensitive element through the camera's lens. Only then can the camera acquire an image containing the light stripe, and only then can the camera be calibrated. Therefore, it is necessary to constrain the sliding distance of the calibration plate on the slide rail, for example, constraining the calibration plate to slide only one step distance at a time under the drive of the motor to meet the camera's imaging requirements.

[0040] It should be noted that the size of the calibration plate and the length of the slide rail should be adaptively designed according to the relative position between the camera and the structured light transmitter and the distance measurement range limited on the slide rail, so as to ensure that the pose information is covered within the effective measurement range of the structured light module.

[0041] Preferably, when the camera is a distortion-free camera, and the surface of the camera's photosensitive element is at a certain angle to the surface of the calibration plate being measured, the distance between the baseline and the calibration plate can be directly calculated according to the pinhole imaging principle. The surface of the camera's photosensitive element and the surface of the calibration plate being measured can be considered parallel, so the distance between the structured light module and the calibration plate can be directly calculated according to the pinhole imaging principle without affecting the calibration accuracy of the camera.

[0042] As one embodiment, in step 1, during the process of the calibration plate sliding along the slide rail from the upper limit of the range to the lower limit of the range, or during the process of the calibration plate sliding along the slide rail from the lower limit of the range to the upper limit of the range, the camera acquires an image of the calibration plate every time the calibration plate slides a preset fixed distance. The imaging position of the light stripe in the imaging plane is offset by a preset height along the height direction. Specifically, the imaging position of the light stripe at a corresponding test angle is offset by a preset height along the height direction, that is, relative to the imaging position in the image acquired before the current slide, it is offset by a preset height along the height direction. The center coordinates of the projected light stripe are offset once in the pixel plane along the opposite direction of the height direction. This can be represented by the number of pixels, or it can be at the sub-pixel level. The offset on the pixel does not necessarily have to be an integer, it can also be a decimal, and it is detected by the camera in real time.

[0043] refer to Figure 2 It can be seen that when the calibration plate slides to the point where the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point line laser) intersects with the calibration plate at position point O, it is also when the intersection line of the structured light surface formed by the structured light emitted by the structured light emitter and the measured surface of the calibration plate passes through position point O. The offset of the imaging position E of the light strip in the imaging plane relative to the reference position D is the light spot offset DE, which is represented by x in the figure. It is the light spot offset corresponding to a calibration column or a test angle. During the process of the calibration plate sliding on the slide rail, the line connecting the center of the structured light emitter and the center of the camera remains parallel to the calibration plate. The motor driving the calibration board to slide is a stepper motor. At each position the stepper motor drives the calibration board to slide, the distance the calibration board slides each time is the step distance generated by the stepper motor control, and the step distance is a preset fixed distance. When the calibration board steps the same distance along the slide rail, the distance between the calibration board and the structured light module, and its change, are measurable and recordable. The offset of the light stripe on the calibration board is equal and always within the camera's field of view, which is also equivalent to the detectable area of ​​the structured light module. The offset and change of the light spot are detected in real time, and the coordinates of its center are obtained. This allows for the acquisition of multiple sets of vectors for nonlinear fitting in step 2. Each set of vectors represents the center coordinates of the light stripe in a calibration column of a frame of the calibration board image and its corresponding test distance. Multiple sets of vectors represent the center coordinates of the light stripe in the same calibration column of all frames of the calibration board image and their corresponding test distances, thereby improving the accuracy of fitting the modeling parameters corresponding to the same calibration column.

[0044] Specifically, when the calibration plate approaches the structured light module, that is, when it approaches the structured light module in a step-by-step manner according to a preset fixed distance, the calibration plate slides along the slide rail from the upper limit of the range to the lower limit of the range at a preset fixed step distance. During this process, the imaging position of the light stripe in the imaging plane shifts along the height direction at a preset height step distance, which can be detected by the camera in real time. The center coordinates of the light stripe projected in the pixel plane shift in the opposite direction of the height direction at a preset vertical offset step distance. This is because the changes in the pixel plane and the changes on the imaging plane are reversed. For example, if the imaging position of the light stripe in the imaging plane shifts upward along the vertical axis once, then the center coordinates of the light stripe projected in the pixel plane shift downward along the vertical axis once. The camera detects the center coordinates of the light stripe in the pixel coordinate system or pixel plane, preferably by measuring them at a preset vertical offset step distance, starting from a pixel offset starting point in the camera's pixel plane.

[0045] Specifically, when the calibration plate moves away from the structured light module, that is, when it moves away from the structured light module in a step-by-step manner according to a preset fixed distance, the corresponding process is that during the sliding process of the calibration plate along the slide rail from the lower limit of the range to the upper limit of the range with a preset fixed distance as the step distance, the imaging position of the light stripe in the imaging plane shifts along the height direction with a preset height as the step distance, which can be detected by the camera in real time; the center coordinates of the light stripe projected in the pixel plane shift in the opposite direction of the height direction with a preset vertical offset as the step distance, because the changes in the pixel plane and the changes on the imaging plane are reversed; for example, if the imaging position of the light stripe in the imaging plane shifts upward along the vertical axis once, then the center coordinates of the light stripe projected in the pixel plane shift downward along the vertical axis once; the camera detects the center coordinates of the light stripe in the pixel coordinate system or pixel plane, preferably starting from a pixel offset starting point in the pixel plane of the camera and measuring it with a preset vertical offset as the step distance.

[0046] In some embodiments, the initial position of the calibration plate on the slide rail is set at the lower limit of the range, i.e., the structured light module is fixedly mounted on one end of the slide rail, and the calibration plate is slidably mounted on the other end of the slide rail, and moves sequentially towards the upper limit of the slide rail's range under the drive of a stepper motor. The constraint on the ranging range defined on the slide rail can be determined based on whether the light stripe is clearly imaged on the camera's photosensitive element, at least ensuring that the image coordinate position (coordinate position in the image coordinate system (i.e., the imaging plane coordinate system) projected by the light stripe does not deviate from the surface area of ​​the photosensitive element. For example, the furthest position the calibration plate is allowed to slide on the slide rail from the camera is the furthest position where the light stripe projected on the calibration plate by the structured light emitted by the structured light emitter is focused and imaged in the camera. When the calibration plate is at the upper limit of the slide rail's range, the imaging position of the light stripe in the imaging plane is set as the reference position, where the upper limit of the slide rail is the furthest position the calibration plate is allowed to slide on the slide rail from the structured light module (which can also be considered as the camera).

[0047] In other embodiments, the reference position is the imaging position in the camera of the reflected light from the structured light emitted by the structured light emitter on the surface of the object being measured, which is furthest from the structured light module (or the camera). This position is denoted as the limiting position where the reflected light is imaged on the surface of the photosensitive element when the surface of the object being measured is infinitely far from the baseline. The angle α between the structured light emitted by the structured light emitter and the baseline AB affects the setting of the reference position. Preferably, corresponding to the same calibration column, the single-point line laser emitted by the structured light emitter is parallel to the line connecting the reference position and the optical center of the camera. When the center A of the structured light emitter remains unchanged and the optical center B of the camera remains unchanged, the angle α increases, i.e., ray AO' rotates counterclockwise around A, and the reference position D shifts downward along the longitudinal axis. When the center A of the structured light emitter remains unchanged and the optical center B of the camera remains unchanged, the angle α decreases, i.e., ray AO' rotates clockwise around A, and the reference position D shifts upward along the longitudinal axis. The object being measured includes the calibration plate. In summary, the reference position is used as an offset reference point to represent the projected position of the light stripe in the imaging plane coordinate system. It provides an imaging position offset starting point for the measured object or calibration plate at any distance, which facilitates the measurement and description of the displacement of the light stripe from the reference position in the imaging plane or photosensitive element, and improves calibration efficiency.

[0048] As one embodiment, the upper limit of the slide rail's range is the furthest position the calibration plate is allowed to slide on the slide rail from the structured light module (which can also be considered the camera). The distance between this position and the structured light module or baseline is predetermined, and the position falls within the detectable region of the structured light module. The lower limit of the slide rail's range is the closest position the calibration plate is allowed to slide on the slide rail from the structured light module (which can also be considered the camera). The distance between this position and the baseline is predetermined, and the position falls within the detectable region of the structured light module. The calibration plate slides along the slide rail between the upper and lower limits of its range. Structured light emitted by the structured light emitter is incident on the measured surface of the calibration plate. The reflected light travels along a test angle back to the light path formed by the camera's lens, falling within the detectable region of the structured light module. With the initial position of the calibration plate on the slide rail at the upper or lower limit of the range, when the calibration plate slides to a position on the slide rail, the offset of the imaging position of the light stripe in the imaging plane relative to the reference position is marked as the light spot offset. Specifically, it is the displacement of a pixel in a calibration column from the reference position in the imaging plane coordinate system. Since the calibration plate slides from the initial position to a position, the distance it slides and the position it reaches are obtained in real time, determining the distance between the baseline and the calibration plate; because the imaging position of the light stripe in the imaging plane... The calibration plate is offset along the height direction with a preset height as the step distance, so the imaging position of the light stripe in the imaging plane is obtained in real time. Specifically, it is calculated and determined from the coordinate information of the imaging position of the light stripe reflected from the light stripe at the initial position of the calibration plate. Since the center coordinates of the light stripe projected into the pixel plane are offset in the opposite direction of the height direction with a preset vertical offset as the step distance, the center coordinates of the light stripe projected into the pixel plane are obtained in real time. Specifically, it is calculated and determined from the coordinate information of the pixel position projected from the light stripe reflected from the light stripe at the initial position of the calibration plate. When the relative position of the structured light emitter and the camera remains unchanged, the light spot offset changes, and the center coordinates of the light stripe change when the calibration plate is displaced relative to the baseline. Thus, by constraining the sliding range of the calibration plate on the slide rail by the upper and lower limits of the slide rail's range, the test distance is limited, thereby creating a limitation on the offset range of the imaging position of the light stripe in the imaging plane on the calibration plate, to obtain the center coordinates of the light stripe and the corresponding test distance within a reasonable range that supports nonlinear fitting.

[0049] As one embodiment, under the same calibration series, the method for constructing a structured light ranging relationship based on the pinhole imaging principle includes: firstly, explaining the mathematical model between the coordinates of the light stripe center and the test distance (i.e., the aforementioned structured light ranging relationship), which also serves as the function to be fitted in nonlinear fitting. For ease of understanding, the explanation begins with the principle of single-point laser triangulation. From a two-dimensional plane perspective, generally the perspective of the normal plane of the measured surface of the calibration plate, the structured light emitted by the structured light emitter... Figure 2 The structured light emitted in step 1 is equivalent to a single-point line laser. When the structured light emitted by the structured light emitter in step 1 is incident on the calibration plate at a certain angle with the normal of the measured surface of the calibration plate, the reflected light is imaged into the photosensitive element of the camera through the lens at position B. B is the optical center of the camera and also the pinhole in the pinhole model.

[0050] like Figure 2 As shown, in this embodiment, the line connecting the center A of the structured light emitter and the optical center B of the camera is set as the baseline, and the length of the baseline is set as the baseline distance AB. The baseline distance AB is predetermined in this embodiment and is an assembly parameter of the structured light module, as well as an assembly parameter in the height direction. In some embodiments, when the distance between the structured light module and the calibration plate is represented by the horizontal distance between the line connecting the center of the structured light emitter and the optical center of the camera and the measured surface of the calibration plate, it can represent the displacement of the calibration plate relative to the structured light module along the slide rail when the slide rail is placed horizontally, or it can represent the horizontal component of the displacement of the calibration plate relative to the structured light module along the slide rail when the slide rail is tilted to the horizontal plane.

[0051] In this embodiment, the distance between the baseline and the calibration plate is set as the test distance. For example, when the calibration plate slides to the point where the structured light emitted by the structured light emitter (viewed from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point line laser) intersects with the measured surface of the calibration plate at point O, the reflected light at point O is imaged onto the position E of the camera's photosensitive element surface through the optical center B of the camera lens. This allows the camera to obtain the pixel coordinates at position E in real time, which are represented as the center coordinates of the light stripe projected onto the pixel plane after the light stripe is reflected from point O. Figure 2 The distance OC represents the test distance.

[0052] The structured light ranging equation constructed based on the pinhole imaging principle is: Px = k0 / d + k1. This is essentially a relationship between the center coordinates of the light stripe and the test distance under the same calibration column or test angle. Here, d is the test distance, and Px is the center coordinates of the light stripe, analyzed using the X-axis component (related to the baseline distance AB), with units in pixels, representing the number of pixels. The coefficients required to construct this structured light ranging equation are k0 and k1, both used as modeling parameters. Specifically, during the sliding process of the calibration plate on the slide rail, the center coordinates of the light stripe Px, the baseline distance AB, the camera's focal length f, and the test distance OC are all pre-measured or pre-calculated. Furthermore, the emission angle of the structured light emitter and the optical axis direction of the camera are pre-determined. The reflected light path of the structured light is determined from the perspective of a two-dimensional plane. The structured light ranging equation is constructed using the pinhole imaging principle without the need for a three-dimensional coordinate system to two-dimensional coordinate system conversion.

[0053] Under the same calibration column or the same test angle, based on the pinhole imaging principle, the ratio of baseline distance to test distance is equal to the ratio of light spot offset to camera focal length. Specifically, in conjunction with... Figure 2 It can be seen that position D on the surface of the camera's photosensitive element is used as a reference position. The line connecting reference position D and the camera's optical center B is parallel to the structured light emitted by the structured light emitter at position A (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is considered as emitting a single-point line laser). The structured light emitted by the structured light emitter is parallel to the baseline AB (also considered as the baseline distance AB). Reference Figure 2 It can be seen that when the calibration plate slides until the structured light emitted by the structured light emitter (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point line laser) intersects with the calibration plate at position point O, which is also when the intersection line of the structured light surface formed by the structured light emitter and the measured surface of the calibration plate passes through position point O, the offset of the imaging position E of the light stripe in the imaging plane relative to the reference position D is the light spot offset DE, represented by x in the figure; the distance between the baseline AB and the calibration plate, i.e., the test distance, is represented by OC. Since the baseline AB is parallel to the calibration plate, the surface of the camera's photosensitive element is regarded as parallel to the baseline. Therefore, based on the pinhole imaging principle, triangle ABO and triangle DEB are similar, so there exists: AB / OC=x / f. Therefore, x=(AB*f) / OC, where OC can be measured in real time during the sliding process of the calibration plate on the slide rail. Thus, a calculation model for the light spot offset is established through the baseline distance, focal length, and test distance.

[0054] The light spot offset x is equal to the sum of the product of the light stripe center coordinates Px and the single pixel size CellSize, and the projection error Dev, i.e., x = Px * CellSize + Dev. This relationship arises because the light stripe's image on the camera's imaging plane is set in the imaging plane coordinate system. The reference position and the light spot offset are also set in the imaging plane coordinate system, which can be understood as the image coordinate system of the camera coordinate system. The light stripe center coordinates are set in the pixel coordinate system, which is the coordinate system of the camera's pixel plane. The projection error Dev represents the difference between the real-time light spot offset acquired by the camera and the actual light spot offset projected onto the camera's imaging plane for a light stripe on the surface of the object being measured; this is a result obtained through prior experiments. Figure 2 In the above, combining x=(AB*f) / OC, the structured light ranging relationship can be obtained as Px=[(AB*f / OC)-Dev] / CellSize; then, k0 is represented by (AB*f) / CellSize; simultaneously, k0 is represented by -Dev / CellSize; thus, the structured light ranging relationship can be simplified to Px=k0 / d+k1. In this embodiment, k0 and k1 are modeling parameters to be determined. Therefore, the camera intrinsic parameters that originally needed to be calibrated are simplified from Dev, CellSize, AB, and f to k0 and k1, which not only simplifies the ranging relationship that requires coordinate system transformation under the pinhole imaging principle, but also reduces the parameters that need to be calibrated. The functional relationship between the center coordinates of the light stripe in the photosensitive element and the test distance (the distance between the measured surface (target surface) of the calibration plate and the baseline) is simplified.

[0055] In summary, both the individual pixel size and projection error are unknowns. Therefore, k0 and k1 are both unknowns in the structured light ranging equation and need to be fitted. Figure 2The trigonometric ranging equation constructed from a pair of similar triangles, ABO and DEB, is simplified, especially when transforming from the imaging plane coordinate system to the pixel coordinate system. Reducing the types and number of parameters to be calculated is equivalent to reducing the number of camera intrinsic parameters to be calibrated, thus simplifying the camera calibration method. Furthermore, when the calibration plate and the baseline undergo relative displacement, x will change, leading to a displacement of Px. Therefore, only the two parameters k0 and k1 in this structured light ranging equation need to be determined to complete the ranging model of the structured light module. Thus, in this embodiment, only the two parameters k0 and k1 in the aforementioned structured light ranging equation need to be determined to complete the modeling of the structured light module or fit the corresponding functional relationship, that is, to establish a model for measuring the distance between the baseline and the object being measured using structured light, which is also equivalent to completing the camera calibration, simplifying the camera calibration method. Then, based on the simplified camera intrinsic parameter calibration operation, the functional relationship between the test distance and the center coordinates of the light stripe is determined as d=k0 / (Px-k1), which is used to calculate the test distance when the center coordinates of the light stripe are known, and serves as the ranging information of the structured light module.

[0056] This application requires obtaining the structured light ranging formula, which necessitates understanding how the camera itself derives its world coordinates in 3D space step-by-step from pixel coordinates. The following coordinate systems are involved: World coordinate system: A reference coordinate system is also selected in the environment to describe the positions of the camera and the measured object. This coordinate system is called the world coordinate system. The relationship between the camera coordinate system and the world coordinate system can be described by rotation matrices and translation vectors. In this application, the coordinate system where the structured light module or structured light emitter is located can be used as the world coordinate system.

[0057] Camera coordinate system: The origin of the camera coordinate system is the camera's optical center. The x-axis and y-axis are parallel to the X and Y axes of the image, and the z-axis is the camera's optical axis, which is perpendicular to the image plane. This spatial rectangular coordinate system is called the camera coordinate system, also known as the camera camera coordinate system. The camera coordinate system is a three-dimensional coordinate system. The intersection of the optical axis and the image plane is the origin of the image coordinate system. The rectangular coordinate system formed by the optical axis and the image's X and Y axes is the image coordinate system, which is a two-dimensional coordinate system. Converting world coordinates to camera coordinates involves a rigid body transformation. This corresponds to the camera's extrinsic parameters. The transformation from camera coordinates to image coordinates is a 3D to 2D perspective projection process.

[0058] The imaging plane coordinate system (X, Y) is the image coordinate system XOY in the aforementioned camera coordinate system. The unit of its coordinate axes is usually millimeters (mm). The origin is the intersection of the camera optical axis and the image plane (called the principal point), which is the center point of the image. The X-axis and Y-axis are parallel to the U-axis and V-axis, respectively.

[0059] The pixel coordinate system (U, V) is a two-dimensional Cartesian coordinate system that reflects the arrangement of pixels on a two-dimensional image plane. The U-axis and V-axis are parallel to the two sides of the image plane, respectively. The unit of the coordinate axes in the pixel coordinate system is pixels (integers). Transforming image coordinates into pixel coordinates involves discretizing the image coordinate system. The origin of the pixel coordinate system is generally located at the upper left corner of the image. The offset of the optical center from the origin of the pixel coordinate system is defined as the reference pixel coordinate, which is the coordinate of the camera's optical center in the pixel coordinate system. The dimensions of each pixel in the horizontal and vertical directions are denoted as (mm / pixel).

[0060] As one embodiment, under the control of the controller of the aforementioned structured light module, the motor drives the calibration plate to slide on the slide rail. During the sliding process along the slide rail from the upper limit to the lower limit of the range, the calibration plate slides sequentially through multiple positions at least according to a preset fixed distance (the step distance driven by the motor), preferably eight positions, all located between the upper and lower limits of the slide rail's range. At each position, the camera acquires a frame of the calibration plate image and extracts the center coordinates of the light stripes in the corresponding calibration column, including the center coordinates of the light stripe corresponding to a pixel in the pixel coordinate system. When the relative position of the structured light emitter and the camera remains unchanged, and the calibration plate shifts relative to the baseline, the light spot offset changes, and the center coordinates of the light stripe change. Therefore, in step 2, the method for nonlinearly fitting the center coordinates of the light stripes in the same calibration column and the test distance corresponding to the same calibration column includes: Step 21: Iterate through the i-th calibration column of the M-frame calibration board images acquired in Step 1; generally, start from the first column of pixels in the first frame of the calibration board image acquired by the camera at the initial position of the slider. Then execute Step 22.

[0061] Step 22: From the currently traversed calibration board image, extract the row pixel position coordinates Pxj of the light bar center in the i-th calibration column, and record them as the coordinates of the i-th column light bar center in the currently traversed calibration board image. The row pixel position coordinates of the light bar center in the i-th calibration column can be understood as the coordinate position of the light bar centroid in the i-th calibration column in the pixel plane, which can be extracted in real time or calculated by relevant algorithms; at the same time, determine the test distance dj corresponding to the i-th calibration column in the same calibration board image, which can be measured from the slide rail; j represents the sort number of the currently traversed calibration board image, so that the currently traversed calibration board image is marked as the j-th calibration board image; j is a positive integer, j is greater than 0, j is less than or equal to M, M is also a positive integer, preferably 8 or a larger value. Then execute step 23.

[0062] Step 23: Configure j=j+1, so that the data of the (j+1)th frame calibration board image is updated to the data of the jth frame calibration board image. Then repeat step 22 until the i-th calibration column of the M frames calibration board images has been traversed. The coordinates of the center coordinates of the i-th column light stripe in each frame calibration board image are formed into a coordinate set {Px1, Px2, ..., PxM}. At the same time, the test distances corresponding to the i-th calibration column in each frame calibration board image are formed into a distance set {d1, d2, ..., dM} or a distance set {1 / d1, 1 / d2, ..., 1 / dM} to fit the aforementioned relation Px=k0 / d+k1. Then execute step 24.

[0063] In step 23, k0 and k1 are both unknowns, while Px and d are both knowns. Px includes Px1, Px2, ..., PxM; d includes d1, d2, ..., dM. In step 21, j increments from the value 1. Therefore, configuring j = j + 1 in step 23 is equivalent to traversing the next frame of the calibration board image in step 21.

[0064] Preferably, the center coordinates of any two light stripes in {Px1, Px2, ..., PxM} are not equal, and the test distances of any two light stripes in {d1, d2, ..., dM} are not equal.

[0065] Step 24: Configure the coordinate set and distance set from Step 23 into two sets of data on two coordinate axes (i.e., two sets of input data for the function to be fitted), perform nonlinear fitting, and obtain the modeling parameters k0 and k1 corresponding to the i-th calibration column, thus completing the calibration of the modeling parameters corresponding to the i-th calibration column; and determine the structured light ranging relationship corresponding to the i-th calibration column. Whenever the structured light ranging relationship corresponding to the i-th calibration column is fitted, it is determined that the relationship between the light stripe center coordinate Px and the test distance d is found, so it is not necessary to calibrate the camera intrinsic parameters in advance, or it can be understood as simplifying the camera calibration process; and record the modeling parameters k0 and k1 corresponding to the i-th calibration column as a modeling parameter pair of the i-th calibration column, i.e., the modeling parameters corresponding to the i-th calibration column. At this time, in the aforementioned relationship Px=k0 / d+k1, k0 and k1 become known quantities, while Px and d become unknown quantities. Then execute Step 3.

[0066] The modeling parameters k0 and k1 corresponding to the same calibration column finally fitted in step 24 satisfy the structured light ranging relationship corresponding to the same calibration column collected by the calibration plate moving along the slide rail. This is further extended to satisfy the ranging requirements of the structured light module for the measured object at the corresponding test angle, that is, the fitting coefficient required by the ranging model of the structured light module at the corresponding test angle.

[0067] It should be noted that the modeling parameters fitted by the modeling parameter calibration method are necessary parameters in the structured light ranging relationship that the same calibration column and the corresponding light stripe center coordinates of each frame of calibration board image follow during the sliding process of the calibration board. Therefore, to construct this structured light ranging relationship on the same calibration column of each frame of calibration board image, it is necessary to introduce the light stripe center coordinates projected at multiple distance positions for fitting in order to calibrate the corresponding modeling parameters of each frame of calibration board image in the same calibration column.

[0068] In summary, this embodiment does not require conversion to an intrinsic parameter matrix according to the disclosed intrinsic parameter calibration method in the prior art. It does not require obtaining the actual spatial coordinates through light plane parameters and camera intrinsic parameters as in traditional methods. Instead, it relies on a motor-driven calibration plate to slide to different distances to collect the center coordinates of the light stripes in the same calibration column and their corresponding test distances. It then fits two modeling parameters corresponding to the same calibration column. Furthermore, these are two modeling parameters fitted from multiple frames of calibration plate images in the same calibration column, rather than multiple optical parameters corresponding to each frame of calibration plate images in the same calibration column. Compared with existing technologies, this reduces the types and number of camera intrinsic parameters that need to be calculated, thereby simplifying the computational workload of the transformation relationship from the camera coordinate system to the pixel coordinate system and improving the calibration efficiency of the structured light module. In subsequent practical applications for ranging, the computational efficiency is also improved.

[0069] Based on the above embodiments, after each calibration of the modeling parameters corresponding to a calibration column is completed by executing step 2, step 3 is executed once. In step 3, i is configured to i+1, so that the pixels of the (i+1)th calibration column in the j-th frame calibration board image are updated to the pixels of the ith calibration column in the j-th frame calibration board image. Then, steps 21 to 24 are repeated until all N calibration columns in each frame calibration board image acquired in step 1 have been traversed, and it is determined that after all calibration columns are fitted nonlinearly, N pairs of modeling parameters are accumulated, thus confirming that the calibration of the modeling parameters of the structured light module is complete. Here, i in step 21 starts from the value 1 and increments. Therefore, configuring i=i+1 in step 3 is equivalent to traversing the next calibration column of the same frame calibration board image in step 21. Specifically, in the M-frame calibration board images, by repeatedly executing steps 21 to 24, N structured light ranging relationships are cumulatively fitted; the N modeling parameter pairs correspond to the N calibration columns in each frame of the calibration board image. The N calibration columns are the N columns calibrated in the pixel plane of the same frame of the calibration board image, and each column corresponds to a test angle and a test distance; i represents the column number of the currently traversed calibration column in a frame of the calibration board image, i is a positive integer, i is greater than 0, and i is less than or equal to N. Therefore, there are N columns of pixels or N calibration columns of light stripe center coordinates in the camera image space. Simple modeling can be performed sequentially at the N test angles using the methods described in steps 2 and 3 above, cumulatively fitting N structured light ranging relationships and modeling parameter pairs, that is, cumulatively fitting 2*N modeling parameters, thus completing the camera calibration. Therefore, this embodiment repeats steps 21 to 24 to complete the calibration of the corresponding modeling parameters column by column in each frame of the calibration board image, thereby achieving the calibration of the structured light module from multiple test perspectives.

[0070] Preferably, the nonlinear fitting adopts the least squares method; specifically, the test distance corresponding to the center coordinates of the i-th column of the calibration plate image in the j-th frame and the i-th calibration column of the calibration plate image in the j-th frame supports the least squares method for fitting, fitting the mapping relationship corresponding to the i-th calibration column, and thus achieving the optimal coefficients by fitting the least squares method as the aforementioned modeling parameters, so as to minimize the fitting error.

[0071] This application also discloses a chip used to execute the modeling parameter calibration method. As the execution entity for steps 1, 2, and 3 in the aforementioned embodiments, this chip can control a motor to drive the calibration plate relative to the structured light module on a slide rail, control the structured light emitted by the structured light module to form a light stripe on the surface of the object being measured, and control the camera to include the image of the light stripe. This allows the camera to detect in real-time the coordinates of the pixel position projected onto the pixel plane of the structured light module. Thus, this chip acts as a common controller for both the structured light module and the motor. Furthermore, when the structured light emitted by the structured light emitter is considered to consist of multiple single-point line lasers, this chip can sequentially perform steps 1 to 3 to model each calibration column of the ranging system composed of the single-point line laser emitter and the camera at each test angle. That is, at each test angle, the modeling parameters on the corresponding calibration column are obtained through nonlinear fitting, avoiding the need to calibrate the camera's intrinsic parameters first.

[0072] In this embodiment, during the nonlinear fitting process of the center coordinates of the light stripe in the same calibration column and the test distance corresponding to the same calibration column, the optimal relationship between the center coordinates of the light stripe and the test distance will be fitted and the modeling parameters will be obtained. The center coordinates of the light stripe and the test distance (the distance between the calibration board and the structured light module) extracted from the calibration board images at different distances are known quantities that can be obtained in real time from each frame of calibration board images. Therefore, using these known quantities, the coefficients required to construct the structured light ranging relationship can be fitted and calculated from the image data of multiple frames of calibration board images, and a mathematical fitting model can be established for the ranging of the structured light module.

[0073] Therefore, by performing nonlinear fitting on the mapping relationship between the center coordinates of the light stripes and their test distances in the same calibration column of multiple calibration board images, the chip can calculate the aforementioned modeling parameters as the optimal parameters for constructing the mapping relationship between the center coordinates of the light stripes and their test distances in the same calibration column, i.e., the coefficients required to construct the structured light ranging relationship. The fitted modeling parameters can replace the traditional camera intrinsic parameters in one direction or one test angle relative to the optical axis of the camera, so as to replace or eliminate the calibration of the camera intrinsic parameters before the ranging operation.

[0074] In summary, the aforementioned chip, by executing steps 1 to 3, controls the displacement of the calibration board relative to the structured light module and uses the coordinates of the light stripe centers in different calibration columns in the calibration board images acquired at different distances, along with their corresponding test distances, to perform nonlinear fitting to complete parameter calibration. The fitted modeling parameters can replace the traditional camera intrinsic parameters in one direction or one test angle relative to the camera's optical axis, thereby improving the efficiency of structured light module calibration while ensuring the accuracy of the calibration parameters. Furthermore, the modeling parameters corresponding to each calibration column fitted by the chip and the structured light ranging relationship it can construct simplify the ranging modeling method between structured light and the camera, or the camera calibration method, to simply fitting and calculating the modeling parameters corresponding to each calibration column. This replaces or eliminates the need for camera intrinsic parameter calibration before the ranging operation, reducing the computational load of camera calibration and improving the calibration efficiency of the structured light module. Consequently, after obtaining the modeling parameters, modeling and ranging operations can be performed without calibrating the camera's intrinsic parameters, further improving computational efficiency.

[0075] This application also discloses a calibration device, which includes a calibration plate, a slide rail, and the chip disclosed in the foregoing embodiments. The calibration device is used to calibrate the structured light module. In the calibration device, the chip is used to control the calibration plate to slide along the slide rail. Specifically, the calibration plate is driven by a motor to slide within a predetermined range. The chip also controls the structured light emitter to emit structured light onto the surface to be measured on the calibration plate. At the same time, the chip controls a camera to capture images of the structured light projected onto the surface to be measured on the calibration plate. At different distances from the structured light module, a frame of image including light stripes is captured, namely the aforementioned calibration plate image. This allows the modeling parameters corresponding to each calibration column in each frame of the calibration plate image to be obtained by executing steps 1 to 3, and the calibration of the modeling parameters of the structured light module is completed.

[0076] The structured light module includes a structured light emitter and a camera. Specifically, the structured light emitted by the structured light emitter can be composed of multiple single-point line lasers. Each single-point line laser corresponds to a calibration column and also corresponds to a test angle formed relative to the optical axis of the camera.

[0077] The modeling parameters corresponding to the same calibration column fitted by the calibration device disclosed in this application are the coefficients required to construct the structured light ranging relationship under the same calibration column. The structured light ranging relationship is the relationship between the center coordinates of the light stripe in the same calibration column and its corresponding test distance, constructed based on the pinhole imaging principle. Constructing / defining the structured light ranging relationship under the pinhole imaging principle can be specifically understood as the relationship between the center coordinates of the light stripe and its corresponding test distance, constructed based on the pinhole imaging principle. Combining the data from calibration board images at different distances, the coefficients required to construct this structured light ranging relationship can be calculated. In this embodiment, the structured light ranging relationship can be used as the corresponding function to be fitted in nonlinear fitting to fit the modeling parameters, including at least two fitting coefficients. Therefore, compared to the existing line laser-camera modeling ranging scenario, this embodiment converts the mathematical formula constructed by triangular ranging into the relationship between the center coordinates of the light stripe and its corresponding test distance, without requiring additional correction of the camera's intrinsic and extrinsic parameters, resulting in less calibration computation.

[0078] The existing Zhang's calibration method for camera calibration uses a calibration board arranged like a chessboard. This method suffers from high detection errors at the corner points of the chessboard, leading to significant errors in the subsequently solved intrinsic parameter matrix. Furthermore, the rotation matrix in the extrinsic parameters no longer satisfies the constraints of a special orthogonal group, resulting in inaccurate intrinsic and extrinsic parameter matrices. In contrast, the calibration device disclosed in this application has lower structural requirements and uses a simplified camera calibration model. It can be widely applied in various fields such as automotive production and electronics manufacturing for rapid product dimension measurement and rapid robot trajectory guidance.

[0079] Preferably, the calibration device further includes a clamp for fixing the line connecting the center of the structured light emitter and the center of the camera parallel to the measured surface of the calibration plate, and for assembling the emission direction of the structured light emitter at an angle to the optical axis of the camera; the distance between the center of the structured light emitter and the center of the camera is fixed. The emission direction of the structured light emitter intersects the calibration plate, so that the structured light emitted by the structured light emitter enters the calibration plate at a certain angle with the normal of the measured surface of the calibration plate. The reflected light is imaged into the camera's photosensitive element through the camera's lens. The camera acquires the image of the reflected structured light in the camera's photosensitive element, represented by the image of the light stripe in the camera's imaging plane; the image of the light stripe in the camera's imaging plane is part of the image of the calibration plate; it is worth noting that the camera is configured to be distortion-free. Thus, this embodiment reduces distortion calculations and supports the use of the pinhole imaging principle to accelerate the establishment of the structured light ranging formula, thereby speeding up calibration.

[0080] The relative position between the center of the structured light emitter and the center of the camera, and the angle between the emission direction of the structured light emitter and the optical axis direction of the camera are fed back to the chip or structured light module. The chip can fix the relative position and angle by controlling the fixture, thereby keeping the relative pose between the camera and the structured light emitter unchanged.

[0081] It should be noted that the above description is only for the convenience of illustrating the relative positions of the components in the structured light module. Under the condition that the relative positions remain unchanged, the structured light module and the calibration plate do not necessarily have to be placed on a horizontal plane, and can also be placed on a vertical plane. Driven by the motor, the calibration plate can move freely or rotate relative to the slide rail, and its placement posture is unrestrained.

[0082] When the surface of the camera's photosensitive element (photosensitive surface) is preferably parallel to the measured surface of the calibration plate, the object surface (from the calibration plate) and the image surface (the camera's photosensitive element) can remain parallel, which facilitates the establishment of similar triangles using the pinhole imaging principle for triangular distance measurement calculation, thereby using the triangulation method commonly used in single-point line laser ranging to speed up calibration.

[0083] The plane containing the calibration board image captured by the camera can be considered as being decomposed into columns. Theoretically, the columns can be fitted with parameters independently, which is equivalent to decomposing the structured light surface into multiple single-point line lasers (structured light emitted by single-point line laser emitters). For example, the structured light (a line of structured light) can be divided into single-point line lasers in multiple directions according to a preset angle (just like a line of lasers is divided into multiple single-point line lasers). Then, triangulation (the principle of pinhole imaging) is used for each single-point line laser to speed up the calibration, so that the triangulation method on different directions, different test angles, or different calibration columns can calibrate the corresponding modeling parameters.

[0084] Preferably, the calibration device further includes a stepper motor. The calibration plate is mounted at one end of the slide rail, and the structured light module is mounted at the other end. The stepper motor can drive the calibration plate to move away from the baseline of the structured light module (the line connecting the center of the structured light emitter and the center of the camera) by a certain number of steps, or to move closer to the baseline of the structured light module (the line connecting the center of the structured light emitter and the center of the camera) by a certain number of steps. Specifically, the chip controls the stepper motor to drive the calibration plate to slide on the slide rail. Every time the calibration plate slides a fixed distance along the slide rail, the camera captures an image of the calibration plate. The imaging position of the light stripe in the imaging plane is offset by a preset height along the height direction, and the corresponding projected center coordinates of the light stripe are offset by a vertical offset in the pixel plane along the height direction. Therefore, the calibration model with the structured light emitter as the primary viewpoint obtains the center coordinates and offset of the light stripe required to construct the structured light ranging formula, the offset of the imaging position of the light stripe in the imaging plane, and the distance the calibration plate has slid along the slide rail, all on the same light stripe.

[0085] Therefore, in this embodiment, the motor, calibration plate, slide rail, and fixture for fixing the sensor in the structured light module constitute a calibration device for calibrating the structured light module under the drive of the motor. The structured light module can acquire two-dimensional images of the calibration plate at different distances. There will be light stripes in a certain frame of two-dimensional image that are clearly imaged in the imaging plane of the camera and will not deviate from the imaging plane of the camera (the surface of the photosensitive element). When changes occur in the structured light module, including changes in the camera's focal length, the relative position between the structured light emitter and the camera, the emission direction of the structured light emitter (which affects whether a light stripe can be projected onto the calibration plate at a certain distance), and the camera's optical axis direction, the calibration plate needs to slide to the corresponding position under the drive of a motor before the structured light emitted by the structured light emitter can be incident on the calibration plate and the reflected light can be imaged onto the surface of the camera's photosensitive element through the camera's lens. Only then can the camera acquire an image containing the light stripe, and only then can camera calibration be performed. Therefore, it is necessary to constrain the sliding distance of the calibration plate on the slide rail, for example, constraining the calibration plate to slide only one step distance at a time under the drive of the motor to meet the camera imaging requirements. Preferably, when the camera is a distortion-free camera, when the surface of the camera's photosensitive element is at a certain angle to the measured surface of the calibration plate, the distance between the baseline and the calibration plate can be directly calculated according to the pinhole imaging principle without affecting the camera's calibration accuracy.

[0086] In summary, the calibration board, driven by a motor, calibrates the structured light module without the need for specially set up targets such as checkerboard patterns. Calibration can be performed simply by controlling the calibration board to slide along the slide rail, which can be easily automated. Thus, the efficiency of structured light module calibration is improved while ensuring the reliability of the calibration results.

[0087] Preferably, the calibration plate is disposed on the normal plane of the slide rail; the row pixel positions in the pixel plane are parallel to the placement plane of the slide rail so that the line connecting the row of pixels corresponding to the light strip is parallel to the placement plane of the slide rail, and the column pixel positions in the pixel plane are considered to be perpendicular to the slide rail, wherein the slide rail can be placed on a horizontal plane; the structured light emitted by the structured light emitter is a linear structured light. Therefore, as the calibration plate slides along the slide rail, the intersection line between the structured light emitted by the structured light emitter and the calibration plate (the light strip) can be parallel to the horizontal plane, and the optical axis direction of the camera can be parallel to the slide rail. This enables calibration in the height direction disclosed in the aforementioned embodiments with reduced coordinate system conversion or angle conversion, improving the convenience of calibration.

[0088] Preferably, the structured light emitted by the structured light emitter is incident perpendicularly on the measured surface of the calibration plate, and the intersection line of the structured light and the calibration plate is parallel to the horizontal line. Specifically, when the slide rail is placed on a horizontal plane and the calibration plate is located on the normal plane of the slide rail (the normal plane in the direction parallel to the slide rail), under the fixed assembly of the fixture, after the structured light surface emitted by the structured light emitter is incident perpendicularly on the measured surface of the calibration plate, the intersection line (light stripe) of the structured light surface and the measured surface of the calibration plate is parallel to the horizontal plane, and the row pixel positions in the pixel plane are parallel to the placement plane of the slide rail. This reduces the conversion of coordinate system angles and accelerates the construction of the structured light ranging relationship using the pinhole imaging principle or the use of the least squares method for fitting.

[0089] Preferably, the Z-axis of the camera coordinate system is perpendicular to the measured surface of the calibration plate, and the intersection of the structured light and the calibration plate is parallel to the horizontal line. Specifically, when the slide rail is placed on a horizontal plane and the calibration plate is located on the normal plane of the slide rail (the normal plane in the direction parallel to the slide rail), under the fixed assembly of the fixture, the Z-axis of the camera coordinate system is perpendicular to the measured surface of the calibration plate (if the imaging plane of the camera is set on the normal plane of the slide rail (the normal plane in the direction parallel to the slide rail), then the Z-axis of the camera coordinate system is regarded as the optical axis), the intersection of the structured light surface and the measured surface of the calibration plate (light stripe) is parallel to the horizontal plane, and the row pixel position in the pixel plane is parallel to the placement plane of the slide rail. This reduces the conversion of coordinate system angles and speeds up the construction of the structured light ranging formula using the pinhole imaging principle or the use of the least squares method for fitting.

[0090] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for calibrating modeling parameters of a structured light module, characterized in that, The structured light module includes a structured light emitter and a camera; the modeling parameter calibration method includes the following steps: Step 1: Control the structured light emitter to emit structured light, which is projected onto the calibration board to form light stripes; simultaneously, control the calibration board to shift relative to the structured light module, and the camera to acquire images of the calibration board at different test distances, obtaining multiple frames of calibration board images. The center coordinates of the light stripes in each calibration column are extracted from the same frame of the calibration board image, and the test distance corresponding to the same calibration column in the same frame of the calibration board image is determined; each frame of the calibration board image includes light stripes. Step 2: In the multi-frame calibration board images obtained in Step 1, perform nonlinear fitting on the center coordinates of the light stripes in the same calibration column and the test distance corresponding to the same calibration column to obtain the modeling parameters corresponding to the same calibration column; Step 3: Repeat Step 2 to traverse the center coordinates of the light stripes in each calibration column of each frame of calibration board image obtained in Step 1 and their corresponding test distances, obtain the modeling parameters corresponding to all calibration columns in sequence, and determine that the calibration of the modeling parameters of the structured light module is complete. Among them, the modeling parameters corresponding to the same calibration column are the coefficients required to construct the structured light ranging relationship under the same calibration column. The structured light ranging relationship is the relationship between the center coordinates of the light stripe of the same calibration column and its corresponding test distance, which is constructed based on the pinhole imaging principle. The constructed structured light ranging relationship is: Px=k0 / d+k1, where d is the test distance and Px is the center coordinate of the light stripe. The coefficients required to construct the structured light ranging relationship under the same calibration column are k0 and k1, respectively.

2. The modeling parameter calibration method according to claim 1, characterized in that, The light spots of the light stripe at each test angle are projected onto the pixel positions of the camera's photosensitive element, which respectively correspond to the pixel positions in the corresponding calibration columns of the pixel plane, so that the light stripe is represented by pixels in multiple calibration columns in the pixel plane; each calibration column corresponds to a test angle.

3. The modeling parameter calibration method according to claim 2, characterized in that, Before performing step 1, configure the emission direction of the structured light emitter to be at an angle to the optical axis of the camera, and set the line connecting the center of the structured light emitter and the optical center of the camera to be parallel to the measured surface of the calibration plate; set the structured light emitted by the structured light emitter to intersect the measured surface of the calibration plate, so that the structured light emitted by the structured light emitter enters the calibration plate at a certain angle with the normal of the measured surface of the calibration plate, and the reflected light passes through the lens of the camera and is imaged into the photosensitive element of the camera; The camera captures the image of the reflected light from the structured light in the camera's photosensitive element, and the image of the light stripe in the camera's imaging plane is represented by the image of the light stripe. The image of the light stripe in the camera's imaging plane is a part of the calibration board image, so that the calibration board image forms a two-dimensional structured light stripe image.

4. The modeling parameter calibration method according to claim 3, characterized in that, In step 1, the method for controlling the displacement of the calibration plate relative to the structured light module includes driving the calibration plate to slide on the slide rail, wherein the structured light module is fixedly installed at one end of the slide rail, and the initial position of the calibration plate is set at the other end of the slide rail. As the calibration plate slides a certain distance along the slide rail, the camera captures an image of the calibration plate, extracts the center coordinates of the light stripe formed in the pixel plane from the calibration plate image, and records the test distance at the corresponding test angle.

5. The modeling parameter calibration method according to claim 4, characterized in that, In step 1, as the calibration plate slides along the slide rail from the upper limit of the range to the lower limit of the range, or as the calibration plate slides along the slide rail from the lower limit of the range to the upper limit of the range, the camera captures an image of the calibration plate every time the calibration plate slides a preset fixed distance. The imaging position of the light stripe in the imaging plane is offset by a preset height along the height direction, and the corresponding center pixel coordinates of the projected light stripe are offset once in the pixel plane along the opposite direction of the height direction. The offset of the imaging position of the light stripe in the imaging plane relative to the reference position is marked as the light spot offset.

6. The modeling parameter calibration method according to claim 5, characterized in that, The reference position is the imaging position in the camera of the reflected light from the structured light emitted by the structured light emitter on the test surface of the object farthest from the structured light module within the detectable area of ​​the structured light module. The object being tested includes the calibration plate.

7. The modeling parameter calibration method according to claim 6, characterized in that, The upper limit of the range of the slide rail is the furthest position that the calibration plate is allowed to slide on the slide rail from the structured light module, which is within the detectable area of ​​the structured light module; The lower limit of the range of the slide rail is the position that the calibration plate is allowed to slide to on the slide rail to be closest to the structured light module, which is within the detectable area of ​​the structured light module.

8. The modeling parameter calibration method according to claim 7, characterized in that, Methods for constructing structured light ranging relationships under the same calibration column include: Set the line connecting the center of the structured light emitter and the optical center of the camera as the baseline, and set the length of the baseline as the baseline distance; Set the distance between the baseline and the calibration board as the test distance; Under the same calibration column or the same test angle, the ratio of the baseline distance to the test distance is equal to the ratio of the light spot offset to the camera's focal length; the light spot offset is equal to the sum of the product of the light stripe center coordinates and the size of a single pixel and the projection error; the imaging of the light stripe in the camera's imaging plane is set in the imaging plane coordinate system, and the reference position and the light spot offset are both set in the imaging plane coordinate system; the light stripe center coordinates are set in the pixel coordinate system, which is the coordinate system of the camera's pixel plane.

9. The modeling parameter calibration method according to claim 8, characterized in that, In step 2, the method for nonlinearly fitting the center coordinates of the light stripes in the same calibration column and the corresponding test distances in the same calibration column includes: Step 21: Iterate through the i-th calibration column of the M frames of calibration board images acquired in Step 1; Step 22: From the currently traversed calibration board image, extract the row pixel position coordinates Pxj of the light bar center in the i-th calibration column, and record it as the coordinates of the light bar center in the i-th column of the currently traversed calibration board image. At the same time, determine the test distance dj corresponding to the i-th calibration column in the same calibration board image; j represents the sort number of the currently traversed calibration board image, so that the currently traversed calibration board image is marked as the j-th calibration board image; j is a positive integer, j is greater than 0, j is less than or equal to M; Step 23: Configure j=j+1, so that the data of the calibration board image of frame j+1 is updated to the data of the calibration board image of frame j. Then repeat step 22 until the i-th calibration column of M calibration board images has been traversed. The coordinates of the center of the i-th column of light stripe in each frame calibration board image are formed into a coordinate set {Px1, Px2, ..., PxM}. At the same time, the test distances corresponding to the i-th calibration column in each frame calibration board image are formed into a distance set {d1, d2, ..., dM} or a distance set {1 / d1, 1 / d2, ..., 1 / dM}. Step 24: Configure the coordinate set and distance set from Step 23 into data sets on two coordinate axes, perform nonlinear fitting to obtain the modeling parameters k0 and k1 corresponding to the i-th calibration column, and determine the structured light ranging relationship corresponding to the i-th calibration column; and record the modeling parameters k0 and k1 corresponding to the i-th calibration column as a modeling parameter pair of the i-th calibration column; then execute Step 3.

10. The modeling parameter calibration method according to claim 9, characterized in that, In step 3, configure i=i+1, so that the pixel of the (i+1)th calibration column in the j-th frame calibration board image is updated to the pixel of the ith calibration column in the j-th frame calibration board image. Then repeat steps 21 to 24 until all N calibration columns in each frame calibration board image acquired in step 1 have been traversed. After all calibration columns have been fitted by nonlinearity, N pairs of modeling parameters are accumulated, and the calibration of the modeling parameters of the structured light module is completed. In the M-frame calibration board image, by repeatedly executing steps 21 to 24, N structured light ranging relationships are cumulatively fitted; The N modeling parameter pairs correspond to the N calibration columns in each frame of the calibration board image. The N calibration columns are the N columns of the same frame of the calibration board image calibrated in the pixel plane of the camera. Each column corresponds to a test angle and a test distance. i represents the column number of the calibration column being traversed in a frame of the calibration board image. i is a positive integer, greater than 0, and less than or equal to N.

11. The modeling parameter calibration method according to claim 10, characterized in that, Nonlinear fitting employs the least squares method to minimize the fitting error.

12. A chip, characterized in that, The chip is used to perform the modeling parameter calibration method according to any one of claims 1 to 11.

13. A calibration device, characterized in that, The calibration device includes a calibration plate, a slide rail, and the chip described in claim 12. The calibration device is used to calibrate the structured light module described in any one of claims 1 to 11. The chip is used to control the calibration plate to slide along the slide rail and to control the structured light emitter to emit structured light onto the surface under test of the calibration plate. At the same time, the chip controls the camera to acquire images of the structured light projected onto the surface under test of the calibration plate. This is to obtain the modeling parameters corresponding to each calibration column in each frame of the calibration plate image by executing steps 1 to 3, and to determine that the calibration of the modeling parameters of the structured light module is complete. The structured light module includes a structured light emitter and a camera.

14. The calibration device according to claim 13, characterized in that, The calibration device also includes a clamp, which is used to fix the line connecting the center of the structured light emitter and the optical center of the camera to be parallel to the surface under test of the calibration plate, and is also used to assemble the emission direction of the structured light emitter and the optical axis direction of the camera to form an angle; the distance between the center of the structured light emitter and the optical center of the camera is fixed. The emission direction of the structured light emitter intersects with the calibration plate, so that the structured light emitted by the structured light emitter enters the calibration plate at a certain angle with the normal of the measured surface of the calibration plate, and the reflected light is imaged into the camera's photosensitive element through the camera's lens. The camera was configured to not produce distortion.

15. The calibration device according to claim 13, characterized in that, The calibration device also includes a stepper motor, a calibration plate is slidably mounted on one end of the slide rail, and a structured light module is fixedly mounted on the other end of the slide rail; The chip controls the stepper motor to drive the calibration board to slide on the slide rail; every time the calibration board slides a fixed distance along the slide rail, the camera captures an image of the calibration board, and the center coordinates of the corresponding projected light stripe are offset by a preset vertical offset amount in the pixel plane along the height direction.

16. The calibration device according to claim 13, characterized in that, The surface to be measured on the calibration plate is set on the normal plane of the slide rail, and the row pixel positions in the pixel plane are parallel to the placement plane of the slide rail. The structured light emitted by the structured light emitter is a linear structured light.

17. The calibration device according to claim 13, characterized in that, The structured light emitted by the structured light emitter is incident perpendicularly on the measured surface of the calibration plate, and the intersection of the structured light and the calibration plate is parallel to the horizontal line; or, the Z-axis of the camera coordinate system is perpendicular to the measured surface of the calibration plate, and the intersection of the structured light and the calibration plate is parallel to the horizontal line.

Images