Ranging method based on structured light module
By constructing a structured light ranging formula based on the pinhole imaging principle and simplifying camera calibration, the complex camera calibration problem in existing technologies is solved, and ranging efficiency and accuracy are improved. This method is applicable to rapid measurement and robot navigation in multiple fields.
Patent Information
- Application Number
- CN202310605924.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
AI Technical Summary
Existing camera calibration methods for structured light vision systems are complex and inefficient, affecting ranging costs and the implementation of ranging operations, and making it difficult to assist in robot navigation and positioning.
By constructing a structured light ranging equation based on the pinhole imaging principle, the modeling parameters are solved using a calibration device and converted into a ranging function, simplifying the camera calibration process, avoiding the calibration of camera intrinsic parameters, and achieving rapid ranging.
It improves the calibration efficiency and ranging accuracy of structured light modules, reduces ranging costs, and is suitable for rapid measurement of product dimensions and robot trajectory guidance in fields such as automobile production and electronics manufacturing, thus assisting in robot navigation and positioning.
Smart Images

Figure CN119024344A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of structured light ranging and camera calibration, and in particular to a ranging method based on a structured light module. Background Art
[0002] The structured light vision system is a measurement system that combines a line laser and a monocular camera. For example, a line laser sensor is composed of a camera and a line laser. Before using the structured light vision system, camera calibration and laser plane calibration are required to achieve three-dimensional measurement through imaging principles and laser plane parameters. This calibration method is applicable to most cameras. The line laser projects a line laser onto the surface of the object to be measured and forms a line laser plane. The line laser is reflected into the receiving lens of the camera and forms an image on the photosensitive element of the camera. The position of the image changes with the distance of the target. Since the line laser plane can be approximately regarded as a combination of countless single-point lasers emitted from the laser light source, the technicians in the relevant field will obtain the three-dimensional space coordinates of the parameters of the light plane where some discrete laser points are located and the camera internal parameters. In this process, the calibration calculation of the internal parameter matrix is complex, and the overall calibration efficiency is relatively low, which affects the implementation of the distance measurement operation of the structured light vision system, resulting in an increase in the distance measurement cost, and an increase in the difficulty of promoting the structured light vision system, which is difficult to assist in robot navigation and positioning. Summary of the invention
[0003] This application discloses a distance measurement method based on a structured light module, and the technical solutions involved are as follows: A distance measurement method based on a structured light module, the structured light module comprises a structured light emitter and a camera; the distance measurement method comprises: step A, solving a structured light distance measurement relationship constructed by the structured light module under the pinhole imaging principle by a calibration device to obtain modeling parameters; then converting the structured light distance measurement relationship into a distance measurement function with the modeling parameters as known quantities; step B, controlling the structured light emitter to emit structured light to the surface of the measured object and form light strips, and controlling the camera to collect an image of the structured light projected onto the surface of the measured object, wherein the image of the surface of the measured object includes an image of the light strips; step C, calculating the light strip center pixel coordinates from the image collected by the structured light module by using a light strip center extraction algorithm; and step D, inputting the calculated light strip center pixel coordinates into the distance measurement function to convert the distance between the measured object and the structured light module.
[0004] Combining steps A to D, the present technical solution further simplifies the distance measurement function based on the simplification of the structured light ranging relationship by calculating the modeling parameters. Thus, through the simplified function, the effective depth data can be obtained faster without the camera internal parameter calibration, and the efficiency of measuring the distance between the measured object and the structured light module can be improved, thereby facilitating the implementation of robot navigation and positioning.
[0005] Furthermore, after the light strip is imaged by the camera, it is represented by multiple columns of pixels in the camera's pixel plane. Each column of pixels identifies the pixel position of each point that composes the light strip in the camera's photosensitive element. Each column of pixels forms a corresponding measurement angle relative to the camera's optical axis, which is used to represent the orientation characteristics of each point of the light strip projected on the surface of the object being measured relative to the camera's optical center. Among them, the structured light emitted by the structured light emitter can be regarded as composed of multiple single-point structured lights to form a structured light plane. In the structured light plane, a single-point structured light forms a corresponding measurement angle relative to the camera's optical axis, and a single-point structured light forms a column of light point pixel coordinates in the camera's pixel plane. By obtaining corresponding modeling parameters at a finite number of measurement angles, and obtaining the distance information of the object being measured at a finite number of measurement angles of the camera, some simple contour point distance measurement application scenarios can be met.
[0006] Furthermore, a column of pixels corresponds to a column of light point pixel coordinates or a light point pixel coordinate; each column of pixels constructs a structured light ranging relationship in step A and obtains corresponding modeling parameters, and the structured light ranging relationship constructed for each column of pixels in step A corresponds to a measurement angle, so that: the structured light emitter is decomposed into a plurality of single-point structured light emitters, each single-point structured light emitter emits a single-point structured light, and the emission direction of each single-point structured light emitter corresponds to a measurement angle. Thus, the technical solution can perform step A on the structured light module composed of the single-point structured light emitter and the camera at each measurement angle in turn to model, that is, the structured light ranging relationship is constructed at each measurement angle to obtain the corresponding modeling parameters, avoiding the need to calibrate the camera internal parameters first, and also completing the ranging modeling of the entire structured light module.
[0007] Furthermore, whenever step C calculates the light strip center pixel coordinates at a measurement angle, step D converts the distance between the measured object and the structured light module at the corresponding measurement angle, representing the distance information at the position of the corresponding measurement angle; wherein, there is at least one light strip center pixel coordinate in a column of light spot pixel coordinates, so that when step D uses the distance measurement function for conversion, the light spot pixel coordinates are configured to be represented by the light strip center pixel coordinates. Thus, step D is used to obtain the distances between the position points in each direction covered by the light strip in the measured object and the structured light module.
[0008] Furthermore, in step C, the light stripe center extraction algorithm is used to extract the light stripe center line in the image captured by the structured light module, and multiple light stripe center pixel coordinates are obtained to form a center line point set; in the image including the light stripe, a column of pixel points corresponds to at least one light stripe center pixel coordinate; wherein the light stripe center extraction algorithm includes but is not limited to the grayscale centroid extraction algorithm. Under the premise of ensuring the accuracy of the added internal parameters, the light stripe center pixel coordinates can be extracted relatively accurately, and the ranging operation can be accelerated using the ranging function.
[0009] Further, the step A includes: step A1, setting the position of the calibration plate, the structured light emitter and the camera; wherein the calibration device includes the calibration plate; step A2, controlling the structured light emitter to emit structured light, projecting it onto the calibration plate and forming a light strip, and controlling the camera to collect the calibration plate image at the same time; and controlling the calibration plate to displace relative to the structured light module so that the camera obtains the calibration plate image at different distances; step A3, constructing a structured light ranging relationship based on the pinhole imaging principle, and combining the calibration plate image at different distances to calculate the modeling parameters required to construct the structured light ranging relationship, wherein each column of light point pixel coordinates corresponding to the structured light respectively constructs a corresponding structured light ranging relationship, and each column of light point pixel coordinates corresponds to a set of modeling parameters. Compared with the prior art, the technical solution simplifies the mathematical formula constructed by the triangle ranging, and does not require additional correction of the camera's internal parameters, the amount of calculation is less, and the calibration efficiency of the structured light module is improved; in actual application, after obtaining the modeling parameters, the modeling and ranging operations can be performed without calibrating the camera's internal parameters, and the calculation efficiency is improved. Step A disclosed in this technical solution does not have high requirements on the structure to be built, and the camera calibration model used is simplified, which reduces the cost of ranging. It can be widely used in many scenarios such as rapid measurement of product dimensions and rapid guidance of robot trajectories in various fields such as automobile production and electronic manufacturing, and assists in the implementation of robot navigation and positioning.
[0010] Furthermore, the step A1 specifically includes: adjusting the emission direction of the structured light emitter to be set at an angle with the optical axis direction of the camera, setting the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate; setting the emission direction of the structured light emitter to intersect with the calibration plate, and the structured light plane formed by the structured light emitter intersecting with the measured surface of the calibration plate to form the light strip; wherein the camera collects the imaging of the reflected light of the structured light in the photosensitive element of the camera, which is regarded as a combination of various imaging positions formed in the imaging plane after the multiple single-point structured lights constituting the structured light are reflected by the calibration plate. In this way, on the basis of keeping the relative posture between the camera and the structured light emitter unchanged, the distance information of each position of the projected structured light in the calibration plate can be calculated and restored by using the pinhole imaging principle, especially when the structured light emitter can be decomposed into multiple single-point structured light emitters, the distance measurement information at multiple angles can be obtained.
[0011] Furthermore, step A2 includes: driving the calibration plate to slide on the slide rail; each time the calibration plate slides a certain distance along the slide rail, the camera collects the calibration plate image once, the light bar shifts once on the calibration plate, and the camera detects the pixel coordinates of the light spot formed by the light bar in the pixel plane in real time; wherein, the calibration device also includes a slide rail and a clamp; the relative position between the structured light emitter and the camera is fixed by the clamp. Therefore, the calibration plate calibrates the structured light module under the drive of the motor, and there is no need to specially set up targets such as checkerboards. The calibration can be performed by only controlling the calibration plate to slide along the slide rail by the motor, which is easy to realize automation; thereby, the efficiency of the structured light module calibration is improved while ensuring the reliability of the structured light module calibration results.
[0012] Further, in step A2, during the sliding of the calibration plate along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures an image of the light bar, the imaging position of the light bar in the imaging plane is offset once along the height direction, and the pixel coordinates of the light spot at the corresponding measurement angle are offset by a preset longitudinal offset in the opposite direction of the height direction in the pixel plane; wherein, when the calibration plate slides to a position on the slide rail, the offset formed by the imaging position of the light bar in the imaging plane toward each measurement angle relative to the reference position at the corresponding measurement angle is marked as the light bar offset at the corresponding measurement angle; wherein the reference position at the corresponding measurement angle is set as the reference position, which is, within the detectable area of the structured light module, the imaging position in the camera of the reflected light of the structured light emitted by the structured light emitter on the measured surface of the measured object farthest from the structured light module; the measured object includes the calibration plate. In summary, at each measurement angle, the reference position is used to represent the offset reference point of the projection position of the light strip in the imaging plane coordinate system, providing an imaging position offset starting point for the measured object or calibration plate at any distance, which is convenient for measuring and describing the displacement of the light strip in the imaging plane or in the photosensitive element from the reference position.
[0013] Furthermore, in the step A3, the method for constructing a structured light ranging relationship based on the pinhole imaging principle includes: setting a line between the center of the structured light emitter and the optical center of the camera as a baseline, and setting the length of the baseline as a baseline distance; setting the distance between the baseline and the calibration plate at the corresponding measurement angle as a test distance d at the corresponding measurement angle; the structured light ranging relationship at the corresponding measurement angle is: Px=k0 / d+k1, forming a relationship between the pixel coordinates of the light point at the orientation represented by a column of pixel points and the test distance; wherein, k0 is a parameter to be calibrated in step A3; k1 is a parameter to be calibrated in step A3; Px is the pixel coordinates of the light point at the corresponding measurement angle; the modeling parameters required to construct the structured light ranging relationship at the corresponding measurement angle include k0 and k1; at the same measurement angle, the ratio of the baseline distance to the test distance is equal to the ratio of the light bar offset to the focal length of the camera. At the same measurement angle, the sum of the product of the light spot pixel coordinates and the single pixel size and the projection error is equal to the light bar offset; the single pixel size and the projection error are both predetermined; the imaging of the light bar in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light bar offset are both set in the imaging plane coordinate system; the light spot pixel coordinates are set in the pixel coordinate system, and the pixel coordinate system is the coordinate system of the pixel plane of the camera. By implementing this technical solution, the types and number of parameters that need to be calculated can be reduced, which is equivalent to reducing the number of camera intrinsic parameters that need to be calibrated, simplifying the camera calibration method. Therefore, it is only necessary to determine the two parameters k0 and k1 in the aforementioned structured light ranging relationship to complete the modeling of the structured light module, that is, to establish a model that uses structured light to measure the distance between the baseline and the object being measured, which is equivalent to completing the calibration of the camera, and the camera calibration method has been simplified. Then, on the basis of simplifying the camera intrinsic calibration operation, the functional relationship d=k0 / (Px-k1) between the test distance and the light spot pixel coordinates is determined, which is used to calculate the test distance when the light spot pixel coordinates are known as the ranging information of the structured light module.
[0014] Further, in step A3, a structured light ranging relationship is constructed based on the pinhole imaging principle, and combined with the calibration plate images at different distances, the modeling parameters required for constructing the structured light ranging relationship are calculated, including: in the process of the calibration plate sliding along the slide rail, at the same measurement angle, there is: the calibration plate slides along the slide rail to the first position, then the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px1, and at the same time determines that the test distance is d1; then construct equation 1: Px1=k0 / d1+k1; the calibration plate slides along the slide rail When the light strip moves to the second position, the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px2, and the test distance is determined to be d2; then the equation 2 is constructed: Px2=k0 / d2+k1; then the equations 1 and 2 are combined to solve the modeling parameters k0 and k1 required to construct the structured light ranging relationship at the corresponding measurement angle; wherein the distance between the second position and the baseline is not equal to the distance between the first position and the baseline; wherein, when the calibration plate is displaced relative to the baseline, at the same measurement angle, k0 and k1 remain unchanged. In summary, the pixel coordinates of the light spots disclosed in the technical solution do not need to be converted into an internal parameter matrix according to the internal parameter calibration method disclosed in the prior art, but rely on a motor to drive the calibration plate to slide to different distances to establish a set of equations for the structured light ranging relationship, solve the modeling parameters, and perform camera calibration, thereby simplifying the amount of calculation of the transformation relationship from the camera coordinate system to the pixel coordinate system at each measurement angle; since the mathematical model corresponding to the structured light ranging relationship is simplified compared to the traditional laser triangle ranging model, the types and quantities of the camera internal parameters that need to be calculated are reduced, thereby improving the calibration efficiency of the line structure module; when it is subsequently actually applied to ranging, the execution of subsequent ranging operations is accelerated.
[0015] Further, in step A3, the method of converting the structured light ranging relationship into the ranging function with the modeling parameters as known quantities includes: determining that the structured light ranging relationship at the corresponding measurement angle is Px=k0 / d+k1, setting the modeling parameters required for the structured light ranging relationship at the corresponding measurement angle to known quantities, and setting the light spot pixel coordinates Px at the corresponding measurement angle and the test distance d at the corresponding measurement angle to unknown quantities; then converting Px=k0 / d+k1 into d=k0 / (Px-k1), and recording the converted d=k0 / (Px-k1) as the ranging function, which is used to represent the relationship between the test distance and the light spot pixel coordinates, wherein k0 and k1 are both converted into known quantities, so that when the light bar center pixel coordinates calculated in step C are configured as the light spot pixel coordinates Px, the test distance is converted by the ranging function, and the test distance is represented in step D as the distance between the object to be measured and the structured light module at the corresponding measurement angle. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a flow chart of a distance measurement method based on a structured light module disclosed in one embodiment of the present invention.
[0017] Figure 2 It is a schematic diagram of the optical path of a single-point structured light disclosed in an embodiment of the present invention.
[0018] Figure 3 It is a schematic diagram of the ranging principle of a structured light emitter and a camera disclosed in an embodiment of the present invention.
[0019] Figure 4 It is a specific flow chart of step A disclosed in one embodiment of the present invention. Implementation
[0020] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.
[0021] As an example, Figure 1 As shown, a distance measurement method based on a structured light module is provided, comprising the following steps: Step A: Use a calibration device to solve the structured light ranging relationship constructed by the structured light module under the pinhole imaging principle to obtain modeling parameters; then use the modeling parameters as known quantities to convert the structured light ranging relationship into a ranging function as the ranging model of the structured light module. In step A, the structured light ranging relationship is generally determined and constructed based on the pinhole imaging model, for example, before the calibration device calibrates the structured light module. Among them, the structured light module includes a structured light emitter and a camera. In the specific implementation process, there is a motor, a calibration plate, a slide rail, and a fixture for fixing the sensor in the structured light module to form the calibration device, which is used to calibrate the structured light module under the drive of the motor. Therefore, the structured light module relies on the calibration device to simplify the mathematical relationship between the pixel coordinates of the light point constructed under the pinhole imaging principle (the point projected by the structured light emitter onto the calibration plate or the surface of the object to be measured, the projection point of a single-point structured light) and the test distance (the distance of the object to be measured that needs to be measured). After calculating the modeling parameters required to construct the structured light ranging relationship, it can be determined that the calibration of the camera is completed, thereby improving the calibration efficiency. The structured light ranging relationship will be converted into a ranging function with the modeling parameters as known quantities, which can be regarded as a ranging mathematical model for the structured light module at a measuring angle (a direction relative to the optical axis of the camera) in the placement plane where the slide rail is located, thereby achieving the acceleration of ranging efficiency by relying on the ranging function on the basis of improving the calibration efficiency.
[0022] Step B, control the structured light emitter to emit structured light to the surface of the object to be measured and form light strips, and control the camera to collect the image of the structured light projected onto the surface of the object to be measured, and the image of the surface of the object to be measured includes the image of the light strips; in step B, the structured light module is not set on the slide rail, and there is no need to collect the image of the calibration plate, so the camera is not calibrated by the calibration device, then the camera that collects the image including the light strips is regarded as a calibrated camera, that is, the structured light module has been calibrated in step A.
[0023] Figure 3 The structured light emitted by the structured light emitter in the image processing apparatus (considered to be composed of multiple single-point line lasers with continuous angles) changes the optical path through the lens and wave mirror of the camera, so that the structured light emitted by the structured light emitter forms a structured light surface, which can be understood as the effect of the emitted light being an approximate plane; the structured light surface intersects with the surface of the object to be measured to form the light strip shown in the figure, and controls the camera to collect the image of the calibration plate, including the original image of the light strip, wherein the structured light surface is regarded as divided into multiple single-point structured lights according to the test angle, where the test angle can be recorded as the step angle, and a single-point structured light can be regarded as emitted by a single-point structured light emitter. For example, a single-point line laser is used to emit a single-point line laser, and a single-point structured light is represented as a single-point line laser. Then A straight-line laser can emit a laser plane, and the laser plane it emits is composed of multiple single-point line lasers. Then the straight-line laser can be equivalent to being decomposed into multiple single-point line lasers; similarly, the structured light emitter disclosed in the present application can be regarded as being composed of multiple single-point structured light emitters, each single-point structured light emitter emits a single-point structured light, and multiple single-point structured lights form a structured light plane, that is, the structured light emitter emits light to form a structured light plane and intersects with the surface of the object to be measured to form a light strip. When using single-point laser triangulation for ranging, that is, using a pinhole imaging model (pinhole model) to establish a mathematical model or measure ranging, it is necessary to decompose the structured light into multiple single-point structured lights or directly regard it as a single-point structured light, corresponding to the optical path of the single-point structured light.
[0024] The structured light module can actively or under the command of an external controller extract height calibration data, the X-axis component or Y-axis component of the light strip point cloud, the X-direction calibration data of the black and white area or the white area in the calibration plate, etc. from the image of the object to be measured, and can be used to convert the row pixel position information of the light strip (or the center line of the light strip), including the light point pixel coordinates, wherein the light point pixel coordinates can be regarded as the projection coordinates of the position point on the intersection line of the structured light plane emitted by the structured light emitter and the surface of the object to be measured in the pixel coordinate system of the camera, and support the extraction of the light strip center extraction algorithm disclosed by the prior art, and configure it as the light strip center pixel coordinates, which are also the coordinates in the pixel coordinate system of the camera.
[0025] Step C, using the light stripe center extraction algorithm to calculate the light stripe center pixel coordinates from the image captured by the structured light module, where the light stripe center pixel coordinates are used to indicate that the structured light emitted by the structured light module is reflected by the surface of the object being measured and projected onto the pixel position in the pixel plane of the structured light module, and is located at the center position of each column of pixels in the ROI (region of interest) or at the candidate center point.
[0026] Step D, inputting the calculated light strip center pixel coordinates into the ranging function, converting the distance between the measured object and the structured light module, specifically converting the test distance, and setting the converted test distance as the distance between the measured object and the structured light module, wherein the basis for using the ranging function for conversion is that the aforementioned modeling parameters are known parameters and will not change due to changes in distance, and the light strip center pixel coordinates are a type of light spot pixel coordinates or the light spot pixel coordinates at the center position in the same column of pixels.
[0027] Combining steps A to D, this embodiment further simplifies the distance measurement function based on the structured light distance measurement relationship by calculating modeling parameters, so as to accelerate the acquisition of effective depth data, that is, the distance between the measured object and the structured light module, including the distance information at each measurement angle in the same structured light plane, through the simplified function without the camera internal parameter calibration. This further helps the implementation of robot navigation and positioning.
[0028] As an embodiment, after the light strip is imaged by the camera, it is represented by a plurality of corresponding columns of pixels in the pixel plane of the camera, and each column of pixels identifies the pixel position formed in the photosensitive element of the camera by each point cloud point constituting the light strip; each column of pixels forms a corresponding measurement angle relative to the optical axis of the camera, which is used to represent the orientation characteristics of each point projected by the light strip on the surface of the object to be measured relative to the optical center of the camera; in this embodiment, the camera can detect the pixel coordinates of the light points in each column formed by the projection of the light strip in the pixel plane in real time, and each light strip contained in the same frame image captured by the camera can be represented by a plurality of columns of pixels in the pixel coordinate system. Although a light strip is projected onto the surface of the object to be measured, the number of columns of pixels corresponding to the light strip and the distribution of the columns of pixels projected by the same light strip on the pixel plane can be determined according to the camera resolution of the structured light module, the operating computing power of the controller of the structured light module, and the camera internal and external parameter specification requirements, and this embodiment does not impose any restrictions.
[0029] It can be understood that the structured light emitted by the structured light emitter changes the optical path through the camera so that the structured light forms a structured light surface. The structured light surface can be equivalent to being decomposed into multiple single-point structured lights according to the test angle, which can be a single-point line laser; the light source of the structured light emitter has only one point, and the optical path is changed by the lens and the wave mirror, so that the emitted structured light forms an effect of an approximate plane, which is equivalent to a structured light surface, such as Figure 3 The structured light surface shown; for example, when the structured light is a line laser, the line laser can be regarded as a collection of multiple single-point line lasers, rather than being composed of multiple single-point line lasers in a physical sense. In the structured light surface, a single-point line laser forms a corresponding test angle relative to the optical axis of the camera, and a single-point line laser forms the center coordinate of the light bar in a calibration column of the pixel plane of the camera; the light bar is the intersection of the structured light surface and the measured surface of the calibration plate. In order to form the light bar, the calibration plate here can also be the measured object, and the measured surface of the calibration plate is the measured surface of the measured object.
[0030] In this embodiment, each column of pixel points forms a corresponding measurement angle relative to the optical axis of the camera, and there is a light point pixel coordinate in the pixel coordinates of a column of pixel points projected corresponding to the light bar, and a light point pixel coordinate corresponds to a measurement angle. A measurement angle represents the azimuth formed by a position point on the measured surface relative to the optical axis of the camera. The structured light emitted by the structured light emitter is composed of multiple single-point structured lights to form a structured light plane; within the structured light plane, a single-point structured light forms a corresponding measurement angle relative to the optical axis of the camera, that is, the measurement angle formed by a single-point structured light relative to the optical axis of the camera is set in the structured light plane; a single-point structured light is correspondingly projected in a column of the pixel plane (pixel coordinate system) of the camera to form a light point pixel coordinate, so that each column of pixel points is located at a pixel position formed by the projection of a single-point structured light in the pixel plane after being reflected by the calibration plate. Since the structured light plane can be approximately regarded as a combination of countless single-point structured lights emitted from the structured light emitter, the structured light ranging relationship can be constructed for certain discrete pixel points in the pixel coordinate system according to step A, so that the corresponding modeling parameters can be obtained at a finite number of measurement angles, and the distance information of the measured object at a finite number of measurement angles of the camera can be obtained, which can meet some simple contour point ranging application scenarios.
[0031] Preferably, since the structured light emitted by the structured light emitter forms a structured light plane after passing through the lens of the camera, the light strip is located at the intersection of the structured light plane and the measured surface of the calibration plate, so the projection position of each laser point constituting the light strip in the photosensitive element of the camera can be connected into a horizontal projection line segment in the pixel coordinate system; thereby reflecting that the structured light is a straight-line structured light and the light strip is parallel to a coordinate axis direction of the world coordinate system, reducing the complexity of the posture conversion required for calibration and improving the efficiency of calibration.
[0032] Since each column of pixels identifies the pixel position of each point cloud point that makes up the light strip projected into the camera's photosensitive element, a column of pixels forms a corresponding measurement angle relative to the camera's optical axis, a single-point structured light (which can be understood as a single-point line laser) forms a corresponding measurement angle relative to the camera's optical axis, and a single-point structured light is projected in a column on the camera's pixel plane to form a light point pixel coordinate. Therefore, a column of pixels corresponds to a column of light point pixel coordinates or a light point pixel coordinate, and is located at a corresponding measurement angle relative to the camera's optical axis. In step A, a structured light ranging relationship is constructed for each column of pixels and corresponding modeling parameters are obtained. The structured light ranging relationship constructed for each column of pixels in step A corresponds to a measurement angle, so that: the structured light emitter can be regarded as being decomposed into a plurality of single-point structured light emitters, each of which emits a single-point structured light, and the emission direction of each single-point structured light emitter corresponds to a measurement angle. In step A, the test distance corresponding to each measurement angle can be configured as the distance between the line connecting the center of the single-point structured light emitter and the optical center of the camera and the measured surface of the measured object. Therefore, step A is performed on the ranging system composed of the single-point structured light emitter and the camera at each measurement angle in turn. It can be understood that when using a camera that satisfies the pinhole imaging principle, each column of pixels on the pixel plane can be approximately regarded as a picture seen at the same vertical angle of the structured light plane. At this vertical angle, there is a single-point structured light emitter (similar to the function of a single-point laser) "extracted" from the straight-line structured light (the light strip). The vertical angle is the aforementioned measurement angle, and thus it can be modeled with two modeling parameters by executing step A (corresponding to constructing the structured light ranging relationship, and a structured light ranging relationship is constructed at a vertical angle). By analogy, if there are N columns of pixels in the pixel plane where the image captured by the camera is located, N light point pixel coordinates may be extracted therefrom; step A is executed in turn for modeling the structured light module composed of the single-point structured light emitter and the camera at each measurement angle, that is, the structured light ranging relationship is constructed at each measurement angle to obtain the corresponding modeling parameters, thereby avoiding the calibration of the camera internal parameters first, and completing the ranging modeling of the entire structured light module.
[0033] On the basis of the above embodiment, whenever step C calculates the light bar center pixel coordinates at a measurement angle, step D converts the distance between the measured object and the structured light module at the corresponding measurement angle, representing the distance information at the position of the corresponding measurement angle; wherein, there is at least one light bar center pixel coordinate in a column of light spot pixel coordinates, so that when step D uses the distance measurement function for conversion, the light spot pixel coordinates are configured to be represented by the light bar center pixel coordinates. If there are N columns of light spot pixel coordinates in the pixel plane where the image captured by the camera is located; the single-point structured light and the camera at each measurement angle are modeled in turn, and N distance measurement functions are constructed, correspondingly converting N groups of distance information between the measured object and the structured light module, thereby achieving the distance between the position points in each direction covered by the light bar in the measured object and the structured light module through step D.
[0034] As an embodiment, in step C, the light stripe center extraction algorithm is used to extract the center line of the light stripe in the image captured by the structured light module to obtain multiple light stripe center pixel coordinates and form a center line point set; in the image including the light stripe, a column of pixel points corresponds to at least one light stripe center pixel coordinate; wherein the light stripe center extraction algorithm includes but is not limited to the grayscale centroid extraction algorithm, and of course also includes many mature solutions such as the Steger algorithm based on the Hessian matrix, which can relatively accurately extract the light stripe center pixel coordinates while ensuring the accuracy of the added internal parameters, and use the ranging function to speed up the ranging operation.
[0035] Preferably, the light strip center extraction algorithm is used to extract the center line of the light strip (which can be understood as the light strip) in the image captured by the structured light module, specifically including: using maximum pooling to downsample the captured image including the light strip by 16 times; performing line scanning on the downsampled image to obtain the ROI (region of interest); performing image processing on the ROI area, and calculating the Jacobian matrix in the Y-axis direction of the imaging plane coordinate system (image coordinate system); using pre-set high and low thresholds to judge the Jacobian matrix and select candidate center points; performing weighted averaging on all candidate center points in the Y-axis direction, with the weight being the grayscale value, to obtain the final center point, and all final center points constitute the initial center line point set, which can be a row of pixel positions occupying the pixel plane. In summary, the accuracy and completeness of the structured light center point extraction are improved, while ensuring the point extraction efficiency.
[0036] In some embodiments, the structured light module can be installed on the body of the robot. After the structured light module is installed, the angle between the center line of the structured light emitted by the structured light emitter (the single-point structured light located at the angular bisector of the emission viewing angle) and the baseline is equivalent to the complementary angle of the angle between the structured light plane and the working surface of the robot, preferably 15 degrees; the baseline refers to the line between the center of the structured light emitter and the optical center of the camera when the structured light emitter and the camera are located at the same installation height, and the installation height is the baseline distance. The emission angle of the structured light emitter is related to the detection distance that the robot where the structured light module is located needs to meet, the body width of the robot, and the mechanical distance between the structured light emitter and the camera.
[0037] When the detection distance that the robot where the structured light module is located needs to meet, the body width of the robot, and the mechanical distance between the structured light emitter and the camera are determined, the emission angle of the structured light emitter can be directly obtained through the trigonometric function relationship, that is, the emission angle is a fixed value.
[0038] Of course, if a specific emission angle is required, it can be achieved by adjusting the detection distance that the robot where the structured light module is located needs to meet and the mechanical distance between the structured light emitter and the camera. In some application scenarios, when the detection distance that the robot needs to meet and the robot's body width are determined, the emission angle of the structured light emitter can be changed within a certain angle range by adjusting the mechanical distance between the structured light emitter and the camera.
[0039] As an example, Figure 4 As shown, the step A specifically includes the following steps: Step A1, setting the position of the calibration plate, structured light emitter and camera, including determining the relative position relationship between any two devices, and making device preparations for the calibration of the camera; wherein, the calibration device includes a calibration plate; the emission direction of the structured light emitter is set at a fixed angle with the optical axis direction of the camera, and an oblique light path is formed when the calibration plate is used for calibration calculation; the structured light emitter can be, but is not limited to, a structured light source that supports emitting linear structured light, such as a line laser emitter. The relative position between the structured light emitter and the camera is fixed, so that the structured light module is fixedly set at a position to form a sensor device with a fixed detection angle; an initial distance is set between the line between the center of the structured light emitter and the optical center of the camera and the calibration plate, as the starting point position of the displacement of the calibration plate relative to the structured light module, and the line between the center of the structured light emitter and the optical center of the camera is also set as a fixed baseline, which can form a fixed assembly angle with the displacement direction of the calibration plate.
[0040] In order to reduce distortion calculations, the camera included in the structured light module uses a small-angle distortion-free camera to meet the pinhole imaging principle and ensure the reliability of the camera's internal parameters; the image sensor / photosensitive element set inside the camera can collect the reflected light of the structured light through an oblique optical path; the photosensitive element is a semiconductor device that converts light images into electronic signals; the distance between the surface of the photosensitive element and the optical center of the camera is equal to the focal length; the mass of photosensitive small molecules on the surface of the photosensitive element is called a pixel. Generally, the structured light module adopts the principle of triangular reflection, and the surface of the photosensitive element can be regarded as a matrix, so a light strip corresponding to the structured light can be projected, and the displacement of all points on the light strip can be measured at the same time.
[0041] Step A2: Control the structured light emitter to emit structured light, project it onto the calibration plate and form a light strip, for example, Figure 3 The structured light emitter in the figure can be regarded as a cluster of single-point structured lights (multiple single-point structured lights) emitted, which form a structured light plane after passing through the camera lens. The structured light plane intersects with the surface of the object to be measured to form the light strip shown in the figure, and controls the camera to collect the calibration plate image, including the original image of the light strip.
[0042] A single-point structured light can be regarded as emitted by a single-point structured light emitter. For example, a single-point line laser is used to emit a single-point line laser, and multiple single-point line lasers emit multiple single-point line lasers. Then a line laser can emit a laser plane, and the laser plane emitted by it is composed of multiple single-point line lasers. Then the line laser can be decomposed into multiple single-point line lasers; similarly, the structured light emitter disclosed in the present application can be regarded as composed of multiple single-point structured light emitters, each single-point structured light emitter emits a single-point structured light, and multiple single-point structured lights form a structured light plane, that is, the structured light emitter emits a structured light plane and intersects with the calibration plate to form a light strip. When using single-point laser triangulation ranging, that is, when using the pinhole imaging model (pinhole model) to establish a mathematical model, it is necessary to decompose the structured light into multiple single-point structured lights or directly regard it as a single-point structured light, corresponding to the formation of Figure 2 The optical path of a single-point structured light.
[0043] The structured light module can actively or under the command of an external controller extract height calibration data, the X-axis component or Y-axis component of the light strip point cloud, the X-direction calibration data of the black and white area or the white area in the calibration plate from the calibration plate image, etc., which can be used to convert the row pixel position information of the light strip (or the center line of the light strip), including the center pixel coordinates of the light strip, where the center pixel coordinates of the light strip can be regarded as the projection coordinates of the position point on the intersection line of the laser plane emitted by the structured light emitter and the calibration plate in the pixel coordinate system of the camera.
[0044] Furthermore, step A2 controls the displacement of the calibration plate relative to the structured light module, and the calibration plate may be driven by a motor to displace relative to the camera, so that the camera obtains images of the calibration plate at different distances, and each frame of the calibration plate image preferably includes light strips. Generally, the calibration plate moves away from the camera or approaches the camera in one direction, and the camera successively captures at least two frames of calibration plate images, which are images of the calibration plate at two positions at different distances relative to the structured light module at the same position, and both include light strips, that is, each frame of the calibration plate image includes imaging information of the light strips.
[0045] In some embodiments, before the calibration plate is displaced, a certain distance is left between the calibration plate and the camera, so that when the calibration plate is displaced relative to the structured light module, the camera can collect two-dimensional images of the calibration plate at multiple distances, establish a more comprehensive calibration plate image library, and obtain pixel position information at more distances, including the center pixel coordinates of the light strips corresponding to the light strips at different distances, which are used to construct multiple sets of formulas for the pinhole imaging principle, and can be used to combine the equations to obtain parameters that can replace the camera intrinsic parameters, thereby simplifying the camera calibration method.
[0046] In some embodiments, the point cloud that truly expresses the spatial structure of the laser plane can effectively express the real point cloud spatial coordinates through fitting, thereby ensuring the accuracy of the obtained data.
[0047] Step A3, construct a structured light ranging equation based on the pinhole imaging principle, and combine the calibration plate images at different distances to calculate the modeling parameters required to construct the structured light ranging equation, wherein each column of light spot pixel coordinates corresponding to the structured light respectively constructs a corresponding structured light ranging equation, and each column of light spot pixel coordinates corresponds to a set of modeling parameters, that is, the modeling parameters required to construct a structured light ranging equation at a measurement angle correspond to the parameters required for calibration at a measurement angle. In step A3, based on the relationship between the pixel coordinates of the light spot and the test distance constructed under the pinhole imaging principle, the modeling parameters required to construct the structured light ranging relationship are calculated in combination with the calibration plate images at different distances. The test distance is the distance between the calibration plate and the structured light module, specifically the distance between the line between the optical center of the camera and the center of the structured light emitter and the calibration plate (the measured surface of the calibration plate), which is regarded as the ranging distance of the structured light module. If the structured light emitter is regarded as decomposed into multiple single-point structured light emitters, the test distance is configured as the distance between the line between the center of the single-point structured light emitter and the optical center of the camera and the calibration plate. At this time, the structured light emitted by the structured light emitter is a straight-line structured light, which can be approximated as being decomposed into multiple single-point structured lights, each of which is a straight line of light emitted by a single-point structured light emitter.
[0048] For light bars, Figure 3In the two-dimensional light strip image of the intersection of the laser plane and the surface of the object to be measured, the light spot pixel coordinates can be extracted, including the X-axis component and the Y-axis component, which correspond to the U-axis component (expressed by the number of pixels) and the V-axis component (expressed by the number of pixels) of the pixel coordinate system, which are derived from the result of the projection of the one-line structured light or multiple single-point structured lights. In this embodiment, the light spot pixel coordinates can represent the pixel coordinates that are not affected by distortion.
[0049] In this embodiment, in the process of calculating the modeling parameters required to construct the structured light ranging relationship by combining the constructed structured light ranging relationship and the calibration plate images at different distances, a relationship between the light spot pixel coordinates and the test distance is constructed. The modeling parameters required for the relationship are unknown quantities but remain constant, while the light spot pixel coordinates extracted from the calibration plate images at different distances and the test distance (the distance of the calibration plate relative to the structured light module) are both known quantities. Therefore, the known quantities are substituted into the structured light ranging relationship, and a group of equations with the modeling parameters as unknown quantities are established. The modeling parameters required to construct the structured light ranging relationship are solved to establish a mathematical model for structured light module ranging.
[0050] After constructing the structured light ranging relationship, the modeling parameters required to construct the structured light ranging relationship are obtained. The modeling parameters are necessary parameters for constructing the structured light ranging relationship. They can be the result of the deformation of the camera internal parameters according to the pinhole imaging principle (pinhole model, generally used in the single-point structured light reflection light path), which can replace the traditional camera internal parameters to replace or omit the calibration of the camera internal parameters before the ranging operation. There is no need to calibrate the camera's internal parameters, the amount of calculation is less, and the calibration efficiency of the structured light module is improved; in the actual application process, after obtaining the modeling parameters, the modeling and ranging operations can be performed without calibrating the camera's internal parameters, and the calculation efficiency is improved. When the Zhang calibration method disclosed in the prior art is used for camera calibration, a calibration board with a chessboard is used for camera calibration. The detection error of the chessboard corner points is high, resulting in a large error in the subsequent internal parameter matrix. The rotation matrix in the external parameter no longer satisfies the constraints of the special orthogonal group, and the internal and external parameter matrices obtained by the solution are inaccurate. Compared with the prior art, step A disclosed in the present application has low requirements on the structure to be built, and the camera calibration model used is simplified, which reduces the ranging cost. It can be widely used in many scenarios such as rapid measurement of product dimensions and rapid guidance of robot trajectories in various fields such as automobile production and electronic manufacturing, and assists in the implementation of robot navigation and positioning.
[0051] As an embodiment, the step A1 specifically includes: adjusting the emission direction of the structured light emitter to be set at an angle with the optical axis direction of the camera, setting the line between the center of the structured light emitter and the center of the camera to be parallel to the measured surface of the calibration plate, which can be fixed by a special fixture. Note that it is not limited that the camera and the calibration plate must be parallel, nor is it limited that the intersection line of the laser plane emitted by the structured light emitter and the calibration plate must be horizontal. Among them, the angle between the emission direction of the structured light emitter and the optical axis direction of the camera is also fixed by the fixture, which is used to project structured light and collect images on the same calibration plate, regardless of whether the calibration plate is displaced; the relative position between the center of the structured light emitter and the center of the camera, and the angle between the emission direction of the structured light emitter and the optical axis direction of the camera are fed back to the controller of the structured light module, and the controller of the structured light module can fix the relative position and angle by controlling the fixture, so as to keep the relative posture between the camera and the structured light emitter unchanged.
[0052] It should be noted that the above description is only for the convenience of explaining the relative positions of the components in the structured light module. When the relative positions remain unchanged, the structured light module and the calibration plate do not necessarily have to be placed on the horizontal plane, but can also be placed on the vertical plane. When step A2 is subsequently executed, the calibration plate is driven by the motor to move freely or rotate relative to the slide rail, and the placement posture is unconstrained.
[0053] On the basis of the above embodiments, when the surface of the camera's photosensitive element (photosensitive surface) is preferably parallel to the measured surface of the calibration plate, the object plane (from the calibration plate) and the image plane (the camera's photosensitive element) can remain parallel, which is convenient for using the pinhole imaging principle to establish similar triangles for triangular distance measurement calculation, thereby using the triangulation distance measurement method commonly used in single-point line laser ranging to speed up calibration. At this time, it may be necessary to decompose the structured light plane into multiple single-point structured lights (structured light emitted by a single-point structured light emitter). For example, the structured light (straight-line structured light) is divided into single-point structured lights in multiple directions according to a preset angle (just like a straight-line laser is divided into multiple single-point line lasers), and then the triangulation distance measurement method (pinhole imaging principle) is used for each single-point structured light to speed up calibration, so that the triangulation distance measurement methods in different directions can calibrate the corresponding modeling parameters.
[0054] Combination Figure 2 and Figure 3It can be seen that in step A1 of this embodiment, the structured light plane formed by the structured light emitter intersects with the measured surface of the calibration plate to form the light strip, and the controller of the structured light module sets the emission direction of the structured light emitter to intersect with the calibration plate, so that the structured light (generally a single light ray, such as a single-point structured light, which can refer to the optical path of a single-point line laser) emitted by the structured light emitter in step A2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, and the reflected light is imaged into the photosensitive element of the camera through the lens of the camera, and the camera collects the image formed by the reflected light, wherein the camera collects the image of the reflected light of the structured light in the photosensitive element of the camera, and the image of the light strip in the imaging plane of the camera is used to represent that the image of the light strip in the imaging plane of the camera is a part of the image of the calibration plate. The image of the reflected light of the structured light collected by the camera in the photosensitive element of the camera can be regarded as a combination of various imaging positions formed in the imaging plane after multiple single-point structured lights are reflected by the calibration plate.
[0055] When the structured light emitter uses a line laser emitter, the emitted laser line is structured light. For example, a single-point line laser, countless laser lines emitted by the same line laser emitter can form a laser plane after passing through the camera lens, corresponding to Figure 3 The structured light plane shown in the figure, the intersection of the laser plane and the calibration plate forms a light strip, so that the light strip is a linear structured light strip, such as a projection spot of a straight line laser in the calibration plate; it can be extracted from the calibration plate image to perform a triangulation operation (constructing similar triangles for distance calculation), specifically, each pixel point on the light strip (the projection position of each single-point structured light in the pixel plane or the imaging position in the imaging plane) can perform a triangulation operation (using the aforementioned modeling parameters to construct a structured light ranging relationship for triangulation calculation) to obtain the distance information of each pixel position within the angle range covered by the light strip relative to the camera. In this way, on the basis of keeping the relative posture between the camera and the structured light emitter unchanged, the distance information of each position of the projected structured light in the calibration plate can be calculated and restored faster using the pinhole imaging principle, especially when the structured light emitter can be decomposed into multiple single-point structured light emitters, the ranging information at multiple angles can be obtained.
[0056] On the basis of the above embodiment, the step A2 also includes: the controller of the structured light module can drive the calibration plate to slide on the slide rail through a motor; under the drive of the motor, the calibration plate slides a certain distance along the slide rail, the camera collects the calibration plate image once, and the light bar is offset once on the calibration plate, which is an offset in the height direction of the calibration plate, and the offset distance can be detected by the structured light module in real time; the line between the center of the structured light emitter and the optical center of the camera is set as the baseline; the relative position between the structured light emitter and the camera is fixed; the distance between the baseline and the calibration plate is obtained in advance; the calibration device also includes a slide rail and a fixture; the relative position between the structured light emitter and the camera is fixed by the fixture; if the structured light emitter is regarded as being decomposed into a plurality of single-point structured light emitters and each single-point structured light emitter emits a single-point structured light, the fixture fixes the emission direction of each single-point structured light emitter to point to the corresponding measurement angle, and the line between the center of each single-point structured light emitter and the optical center of the camera is set as the baseline on the corresponding measurement angle.
[0057] Each time the calibration plate image is acquired, the calibration plate is located at a different position on the slide rail, and when the structured light emitter is turned on, the camera acquires an image containing the light strip, and the camera detects in real time the pixel coordinates of the light spot formed by the projection of the light strip in the pixel plane. Specifically, when the calibration plate is set on the normal plane of the slide rail, the height direction of the calibration plate can be regarded as the vertical direction of the slide rail, or parallel to the normal plane of the slide rail, and can also be regarded as the longitudinal direction of the imaging plane or pixel plane of the camera; preferably, the calibration plate is slidably mounted on one end of the slide rail, and the structured light module is fixedly mounted on the other end of the slide rail. Figure 2 As shown, the calibration plate is installed on the right side of the photosensitive element and the calibration plate is supported to slide left and right on the slide rail. The calibration plate is installed at the right end of the slide rail, and the structured light module is installed at the left end of the slide rail. Specifically, the structured light emitter and the camera are both installed at the left end of the slide rail, and the relative position between the structured light emitter and the camera is fixed by the fixture. Figure 2 The line between the center A of the structured light emitter and the optical center B of the camera is set as the baseline AB and is fixed, at least in the process of calibrating the parameters of the camera and the structured light; the slide rail does not have to be placed on a horizontal plane, but can also be placed on a vertical plane and can be rotated at will. Every time the calibration plate slides a certain distance along the slide rail, the light bar on the calibration plate is translated once along the vertical direction of the slide rail; combined with Figure 2It can be seen that the calibration plate starts to slide to the right from a starting position of the slide rail (set at a track position at a known distance from the camera). When the calibration plate slides to the position where the structured light emitted by the structured light emitter (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the measured surface of the calibration plate at point O, the reflected light at position point O is imaged to position E on the surface of the camera's photosensitive element through the optical center B of the camera's lens, so that the camera obtains the pixel coordinates at position E in real time, which are expressed as the pixel coordinates of the light point projected by the light bar passing through position point O; at the same time, the vertical distance CO from point O to the baseline AB is also determined, which is expressed as a test distance, which is also the distance between the calibration plate at the current position and the baseline AB, and can be expressed by the displacement of the calibration plate relative to the structured light module along the slide rail. At this time, the angle between the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) and the baseline AB is ɑ, and the baseline AB is the line connecting the optical center B of the camera and the center A of the structured light emitter. When the calibration plate is set on the normal plane of the slide rail, the baseline AB is parallel to the measured surface of the calibration plate (regarded as perpendicular to the slide rail); then, when the angle between the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) and the baseline AB is fixed at ɑ, the calibration plate is only driven to slide away from the camera until the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, When a single-point structured light is regarded as being emitted and intersects the measured surface of the calibration plate at point O', the distance between point O' and point O is equal to y, and the reflected light at point O' is imaged to position F on the surface of the camera's photosensitive element through the optical center B of the camera lens, that is, in the process of the calibration plate deviating from the fixed position of the camera along the slide rail, the imaging position of the reflected light on the surface of the camera's photosensitive element is offset from position E to position F along the height direction, and the camera obtains the pixel coordinates at position F in real time, which are expressed as the pixel coordinates of the light spot projected by the calibration plate at point O'; at the same time, the vertical distance C'O' from point O' to the baseline AB is also determined, which is expressed as a test distance, and is also the distance between the calibration plate at the current position and the baseline AB, which can be expressed by the displacement of the calibration plate relative to the structured light module along the slide rail.
[0058] On this basis, the distance CO and the pixel coordinates at position E are substituted into the structured light ranging relationship described in step A3, and the distance C'O' and the pixel coordinates at position F are substituted into the structured light ranging relationship described in step A3, and a binary equation group is constructed to solve the relevant modeling parameters to complete the calibration process. Therefore, the calibration plate calibrates the structured light module under the drive of the motor, and there is no need to specially set up targets such as chessboards. The calibration can be performed by only controlling the calibration plate to slide along the slide rail by the motor, which is easy to realize automation; thereby, the efficiency of the structured light module calibration is improved while ensuring the reliability of the calibration results of the structured light module.
[0059] In this embodiment, the calibration plate, the slide rail, and the fixture for fixing the sensor in the structured light module can form a calibration device for calibrating the structured light module under the drive of a motor, wherein the structured light module can collect two-dimensional images of the calibration plate at different distances, and the light strips in a certain frame of the two-dimensional image will be clearly imaged in the imaging plane of the camera and will not deviate from the imaging plane of the camera (the surface of the photosensitive element). When the structured light module changes, including at least one of the focal length of the camera, the relative position between the structured light emitter and the camera, the emission direction of the structured light emitter (whose change will affect whether a light strip can be projected on the calibration plate at a certain distance), and the optical axis direction of the camera, the calibration plate needs to be driven by a motor to slide to the corresponding position before the structured light emitted by the structured light emitter can be incident on the calibration plate and the reflected light can be imaged through the camera lens to the surface of the camera's photosensitive element. Only then can the camera capture the image containing the light strip and then calibrate the camera. Therefore, it is necessary to constrain the sliding distance of the calibration plate on the slide rail, such as constraining the calibration plate to slide only one step distance each time under the drive of the motor to meet the camera imaging requirements.
[0060] It should be noted that the size of the calibration plate and the length of the slide rail should be adaptively designed according to the relative position between the camera and the structured light emitter and the range of measurement specified on the slide rail to ensure that the posture information within the effective measurement range of the structured light module is covered. The measured surface of the calibration plate is set as a white area. The white area is conducive to the camera capturing clear structured light, such as line lasers. The measured surface of the calibration plate can also be set to a black area, provided that the camera can capture the light strip. When the measured surface of the calibration plate is set as a black and white area square, if the light strip hits the junction of black and white, there will be an error in finding the center of the light strip from the calibration plate image captured by the camera.
[0061] Preferably, when the camera is a distortion-free camera and the surface of the camera's photosensitive element forms a certain oblique angle with the measured surface of the calibration plate, the distance between the baseline and the calibration plate can be directly calculated according to the pinhole imaging principle without affecting the calibration accuracy of the camera.
[0062] Preferably, when the calibration plate slides along the slide rail, the intersection line between the structured light emitted by the structured light emitter and the calibration plate may be parallel to the horizontal plane, and the optical axis direction of the camera may be parallel to the slide rail.
[0063] Based on the aforementioned embodiment, in step A2, during the sliding of the calibration plate along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures the image of the light strip, and the imaging position of the light strip in the imaging plane is shifted once along the height direction, and each light spot position may be shifted along the height direction by a preset fixed height, that is, relative to the imaging position in the image captured before the current sliding, it is shifted along the height direction by a preset fixed height, and then, the light spot pixel coordinates at the corresponding measurement angle are shifted in the opposite direction of the height direction in the pixel plane by a preset longitudinal offset, specifically, the pixel coordinates of each light spot formed by the projection of the light strip in the pixel plane are shifted in the opposite direction of the height direction by a preset longitudinal offset, expressed in terms of the number of pixels, and sub-pixel level is also allowed, that is, the offset on the pixel is not necessarily an integer, but may also be a decimal, and is detected in real time by the camera.
[0064] When the calibration plate approaches the structured light module, that is, when it approaches the structured light module in steps according to a preset fixed distance, corresponding to the calibration plate sliding along the slide rail from the upper limit of the range to the lower limit of the range with the preset fixed distance as the stepping distance, the imaging position of the light bar in the imaging plane is offset downward along the longitudinal axis of the imaging plane coordinate system with the preset fixed height as the stepping distance, which can be detected by the camera in real time; the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset upward along the longitudinal axis of the pixel coordinate system with the preset longitudinal offset as the stepping distance, and the camera also detects the pixel coordinates of the light spot in the pixel coordinate system or pixel plane, preferably starting from a pixel offset starting point in the pixel plane of the camera, and measured with the preset longitudinal offset as the stepping distance.
[0065] When the calibration plate is away from the structured light module, that is, when it moves away from the structured light module in steps according to a preset fixed distance, the calibration plate slides along the slide rail from the lower limit of the range to the upper limit of the range with a preset fixed distance as the stepping distance. The imaging position of the light bar in the imaging plane is offset upward along the longitudinal axis of the imaging plane coordinate system with a preset fixed height as the stepping distance, and can be detected by the camera in real time; the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset downward along the longitudinal axis of the pixel coordinate system with a preset longitudinal offset as the stepping distance, and the camera also detects the pixel coordinates of the light spot in the pixel coordinate system or pixel plane, preferably starting from a pixel offset starting point in the pixel plane of the camera and measured with a preset longitudinal offset as the stepping distance.
[0066] The upper limit of the range of the slide rail is the position of the calibration plate on the slide rail that is farthest from the structured light module (also considered as the camera), and the distance relative to the structured light module or the baseline is obtained in advance. The lower limit of the range of the slide rail is the position of the calibration plate on the slide rail that is closest to the structured light module (also considered as the camera), and the distance relative to the structured light module or the baseline is obtained in advance.
[0067] When the calibration plate slides to a position on the slide rail, which may be the case where the initial position of the calibration plate is placed at the upper limit or the lower limit of the range, the offset formed by the imaging position of the light bar in the imaging plane relative to the reference position is marked as the light bar offset, specifically, the offset formed by the imaging position of the light bar in the imaging plane toward each measuring angle relative to the reference position at the corresponding measuring angle is marked as the light bar offset at the corresponding measuring angle, that is, the displacement deviating from the reference position is detected in the imaging plane coordinate system along the direction corresponding to the measuring angle, wherein the reference position at the corresponding measuring angle is set as the reference position, within the detectable area of the structured light module, the imaging position of the reflected light on the measured surface of the measured object farthest from the structured light module emitted by the structured light emitter in the camera; the measured object includes the calibration plate; the line connecting the reference position at the corresponding measuring angle and the optical center of the camera is parallel to a single-point structured light at the same measuring angle emitted by the structured light emitter. The line connecting the reference position at the corresponding measuring angle and the optical center of the camera is parallel to a single-point structured light at the same measuring angle emitted by the structured light emitter.
[0068] Since the calibration plate will slide from the initial position to a position, the distance it slides and the position reached by the calibration plate are obtained in real time to determine the distance between the baseline and the calibration plate; since the imaging position of the light bar in the imaging plane is offset along the height direction with a preset fixed height as a step distance, the imaging position of the light bar in the imaging plane is obtained in real time, specifically, it is determined by calculating the coordinate information of the imaging position of the light bar reflected from the calibration plate at the initial position in the imaging plane; since the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are offset along the height direction with a preset longitudinal offset as a step distance, the pixel coordinates of the light spot formed by the projection of the light bar in the pixel plane are obtained in real time, specifically, it is determined by calculating the coordinate information of the pixel position projected from the light bar reflected from the calibration plate at the initial position in the pixel plane. Therefore, when the relative position of the structured light emitter and the camera remains unchanged, when the calibration plate is displaced relative to the baseline, the light bar offset changes, and the pixel coordinates of the light spot change.
[0069] refer to Figure 2It can be known that when the calibration plate slides until the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the calibration plate at position point O, and the intersection line of the structured light plane formed by the structured light emitter and the measured surface of the calibration plate passes through position point O, the offset of the imaging position E of the light bar in the imaging plane relative to the reference position D is the light bar offset DE, represented by x in the figure; wherein, during the sliding of the calibration plate on the slide rail, the line between the center of the structured light emitter and the center of the camera remains parallel to the calibration plate. The motor driving the calibration plate to slide is a stepper motor. At each position to which the calibration plate slides driven by the stepper motor, the distance the calibration plate slides each time is the step distance generated by the stepper motor control. The step distance is a preset fixed distance. Therefore, when the calibration plate slides the same distance along the slide rail, the change in the light bar offset is equal. The reflected light on the calibration plate can project the light bar on the imaging plane of the camera, and at least the pixel coordinates of its light point can be obtained so as to construct multiple sets of the structured light ranging relationship in step A3 to solve the modeling parameters required to construct the structured light ranging relationship.
[0070] In some embodiments, on the slide rail, the initial position of the calibration plate is placed at the lower limit of the range, that is, the structured light module is fixedly mounted at one end of the slide rail, and the calibration plate is slidably mounted on the other end of the slide rail, and is driven by a stepper motor to step toward the upper limit of the range of the slide rail in sequence. Here, the constraint on the range measurement range defined on the slide rail can be determined based on whether the light bar is clearly imaged on the photosensitive element of the camera, at least to ensure that the image coordinate position (the coordinate position in the image coordinate system (i.e., the imaging plane coordinate system)) projected by the light bar does not deviate from the surface area of the photosensitive element, such as the farthest position from the camera that the calibration plate is allowed to slide on the slide rail is the farthest position and the closest position that supports the light bar projected on the calibration plate by the structured light emitted by the structured light emitter for focusing imaging in the camera. When the calibration plate is at the upper limit of the range of the slide rail, the imaging position of the light bar in the imaging plane is set as the reference position, wherein the upper limit of the range of the slide rail is the farthest position from the structured light module (which can also be regarded as the camera) that the calibration plate is allowed to slide on the slide rail.
[0071] In summary, at each measurement angle, the reference position is used to represent the offset reference point of the projection position of the light strip in the imaging plane coordinate system, providing an imaging position offset starting point for the measured object or calibration plate at any distance, which is convenient for measuring and describing the displacement of the light strip in the imaging plane or in the photosensitive element from the reference position.
[0072] As an embodiment, in step A3, the method of constructing a structured light ranging relationship based on the pinhole imaging principle includes: First, the mathematical model between the light point pixel coordinates and the test distance to be established (i.e. the aforementioned structured light ranging relationship) is explained. For ease of understanding, the single-point laser triangulation ranging principle is explained first. From the perspective of a two-dimensional plane, generally the perspective of the normal plane of the measured surface of the calibration plate, the structured light emitted by the structured light emitter is Figure 2 In the figure, a single-point structured light is regarded as being emitted, which is equivalent to a single-point line laser optical path diagram. When the structured light emitted by the structured light emitter in step 2 is incident on the calibration plate at a certain angle to the normal of the measured surface of the calibration plate, the reflected light is imaged into the photosensitive element of the camera through the lens at position B. B is the optical center of the camera and also the pinhole in the pinhole model.
[0073] like Figure 2 As shown, in this embodiment, the line between the center A of the structured light emitter and the optical center B of the camera is set as the baseline, and the length of the baseline is set as the baseline distance AB; the baseline distance AB is predetermined in this embodiment, and is an assembly parameter of the structured light module, and is also an assembly parameter in the height direction. If the structured light emitter is considered to be decomposed into a plurality of single-point structured light emitters and each single-point structured light emitter emits a single-point structured light, the fixture fixes the emission direction of each single-point structured light emitter to point to the corresponding measurement angle, and the line between the center of each single-point structured light emitter and the optical center of the camera is set as the baseline at the corresponding measurement angle.
[0074] In this embodiment, the distance between the baseline and the calibration plate is set as the test distance, specifically, the distance between the baseline and the calibration plate at the corresponding measurement angle is set as the test distance d at the corresponding measurement angle; for example, when the calibration plate slides to the position where the structured light emitted by the structured light emitter (from the perspective of a two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single-point structured light) intersects the measured surface of the calibration plate at point O, the reflected light at position point O is imaged to position E on the surface of the camera's photosensitive element through the optical center B of the camera's lens, so that the camera obtains the pixel coordinates at position E in real time, which are expressed as the pixel coordinates of the light point projected by the light bar passing through position point O. Figure 2 The distance OC represents the test distance at the corresponding measurement angle, and is used as a directly measurable quantity in step A3, that is, configured as a known quantity in the structured light ranging relationship.
[0075] The structured light ranging relationship constructed based on the pinhole imaging principle is: Px=k0 / d+k1, which forms the relationship between the pixel coordinates of the light spot in the direction represented by a column of pixels and the test distance, which is actually converted into the relationship between the pixel coordinates of the light spot and the test distance. Among them, k0 is the parameter to be calibrated in step A3; k1 is the parameter to be calibrated in step A3; Px is the pixel coordinates of the light spot at the corresponding measurement angle, and here the analysis is based on the X-axis component (associated with the baseline distance AB), and the unit is pixel.
[0076] The modeling parameters required to construct the structured light ranging relationship at the corresponding measurement angle include k0 and k1. Specifically, when the calibration plate slides on the slide rail, the light spot pixel coordinates Px, the baseline distance AB, the focal length f of the camera and the test distance OC are all pre-obtained or pre-calculated, and the emission angle of the structured light emitter and the optical axis direction of the camera are both predetermined to determine the reflected light path of the structured light. Both the size of a single pixel and the projection error are unknown quantities, so that k0 and k1 constitute unknown quantities in the structured light ranging relationship and need to be solved by sliding the calibration plate. Figure 2 The triangular ranging relationship constructed by triangle ABO and triangle DEB is simplified, reducing the type and number of parameters that need to be calculated, which is equivalent to reducing the number of camera intrinsic parameters that need to be calibrated, simplifying the camera calibration method.
[0077] It should be added that at the same measurement angle, the ratio of the baseline distance to the test distance is equal to the ratio of the light bar offset to the focal length of the camera. Figure 2 It can be seen that the position D on the surface of the camera's photosensitive element is used as the reference position, where the line connecting the reference position D and the camera's optical center B is parallel to the structured light emitted by the structured light emitter at position A (from a two-dimensional plane perspective, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single point of structured light), and the angle ɑ formed by the structured light emitted by the structured light emitter and the baseline AB (also regarded as the baseline distance AB) also determines the reference position. Figure 2It can be seen that when the calibration plate slides until the structured light emitted by the structured light emitter (from the perspective of the two-dimensional plane, generally from the perspective of the normal plane of the measured surface of the calibration plate, it is regarded as emitting a single point of structured light) intersects with the calibration plate at position point O, and when the intersection line of the structured light plane formed by the structured light emitter and the measured surface of the calibration plate passes through position point O, the offset formed by the imaging position E of the light bar in the imaging plane relative to the reference position D is the light bar offset DE, which is represented by x in the figure; the distance between the baseline AB and the calibration plate, that is, the test distance is represented by OC. Since the baseline AB is parallel to the calibration plate, the surface of the camera's photosensitive element is regarded as parallel to the baseline, so based on the pinhole imaging principle, the triangle ABO and the triangle DEB are similar, so there is: AB / OC=x / f, which simplifies the pinhole imaging model. Therefore, x=(AB*f) / OC, where OC supports real-time measurement during the sliding of the calibration plate on the slide rail. Thus, a calculation model for the light bar offset is established through the baseline distance, focal length and test distance.
[0078] In step A3, the method of converting the structured light ranging relationship into a ranging function with the modeling parameters as known quantities includes: when it is determined that the structured light ranging relationship at the corresponding measurement angle is Px=k0 / d+k1, the modeling parameters required for the structured light ranging relationship at the corresponding measurement angle are all set to known quantities, and the light point pixel coordinates Px at the corresponding measurement angle and the test distance d at the corresponding measurement angle are both set to unknown quantities; then Px=k0 / d+k1 is converted into d=k0 / (Px-k1), and the converted d=k0 / (Px-k1) is recorded as the ranging function, which is used to represent the corresponding measurement angle The relationship between the test distance d and the light spot pixel coordinates Px, wherein k0 and k1 are both changed into known quantities, so that when the light strip center pixel coordinates calculated in step C are configured as the light spot pixel coordinates Px, the test distance is converted by the ranging function, and the test distance is represented in step D as the distance between the object under test and the structured light module at the corresponding measurement angle; the specific implementation is that whenever step C calculates the light strip center pixel coordinates at a measurement angle, step D converts the distance between the object under test and the structured light module at the corresponding measurement angle, which represents the distance information at the orientation of the corresponding measurement angle.
[0079] At the same measurement angle, the sum of the product of the light spot pixel coordinates Px and the single pixel size CellSize and the projection error Dev is equal to the light strip offset x, that is, x=Px*CellSize+Dev. The premise for this relationship is that the imaging of the light strip in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light strip offset are both set in the imaging plane coordinate system, the light spot pixel coordinates are set in the pixel coordinate system, and the pixel coordinate system is the coordinate system of the pixel plane of the camera. The projection error Dev represents a light strip on the surface of the object being measured. The difference between the light strip offset acquired by the camera in real time and the light strip offset formed by the actual projection in the imaging plane of the camera is the result obtained in advance experiments. Figure 2 In the formula, combined with x=(AB*f) / OC, the structured light ranging relationship that can be obtained is Px=[(AB*f / OC)-Dev] / CellSize; then, k0 is represented by (AB*f) / CellSize, that is, the ratio of the product of the baseline distance and the focal length of the camera to the size of a single pixel is equal to k0; at the same time, k0 is represented by -Dev / CellSize, that is, the inverse of the ratio of the projection error of the light strip in the imaging plane of the camera to the size of a single pixel is equal to k1; so far, the structured light ranging relationship can be converted to Px=k0 / d+k1. In this embodiment, k0 and k1 are both modeling parameters to be determined. Therefore, the camera intrinsic parameters that originally need to be calibrated are simplified from Dev, CellSize, AB and f to k0 and k1, which not only simplifies the ranging relationship that requires coordinate system conversion under the pinhole imaging principle, but also reduces the parameters that need to be calibrated. The simplest functional relationship between the pixel coordinates of the light spot in the photosensitive element and the test distance (the distance between the measured surface (target surface) of the calibration plate and the baseline) is formed.
[0080] Therefore, it is only necessary to determine the two parameters k0 and k1 in the above structured light ranging relationship to complete the modeling of the structured light module, that is, to establish a model that uses structured light to measure the distance between the baseline and the object being measured, which is equivalent to completing the camera calibration and has simplified the camera calibration method. Then, on the basis of simplifying the camera internal parameter calibration operation, the functional relationship between the test distance and the light point pixel coordinates d=k0 / (Px-k1) is determined to calculate the test distance when the light point pixel coordinates are known, as the ranging information of the structured light module.
[0081] Based on the above embodiment, in step A3, a method for constructing a structured light ranging relationship based on the pinhole imaging principle and calculating the modeling parameters required for constructing the structured light ranging relationship in combination with the calibration plate images at different distances includes: Under the control of the controller of the aforementioned structured light module, the motor drives the calibration plate to slide on the slide rail. In the process of sliding along the slide rail from the upper limit of the range to the lower limit of the range, the calibration plate slides through at least two positions in sequence according to a preset fixed distance (the step distance driven by the motor). At each position, the camera captures a frame of the calibration plate image including the light bar and extracts the pixel coordinates of the light spot, including the pixel coordinates of the light spot corresponding to a pixel point in the pixel coordinate system; when the relative position of the structured light emitter and the camera remains unchanged, when the calibration plate is displaced relative to the baseline, the light bar offset changes, the pixel coordinates of the light spot change, and k0 and k1 remain unchanged, so that k0 and k1 replace the internal parameters of the camera. Therefore, step A3 specifically contains: The calibration plate slides along the slide rail to the first position, and the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px1, which are obtained by the camera in real time from the surface of the photosensitive element, and the test distance is determined to be d1; then construct equation 1: Px1=k0 / d1+k1, wherein Px1 and d1 are known quantities in equation 1, and k0 and k1 are unknown quantities in equation 1.
[0082] The installation position of the structured light module on the slide rail remains unchanged, and inside the structured light module, the optical axis direction of the camera remains unchanged, the emission direction of the structured light emitter remains unchanged, and the distribution position of the baseline remains unchanged (the relative position between the camera and the structured light emitter remains unchanged); then, the calibration plate slides along the slide rail to the second position, and the camera captures the image of the light strip and determines that the pixel coordinates of the light spot formed in the pixel plane of the camera are Px2, and at the same time determines that the test distance is d2; then construct equation 2: Px2=k0 / d2+k1, wherein Px2 and d2 are known quantities in equation 2, and k0 and k1 are unknown quantities in equation 2.
[0083] Then, equation 1 and equation 2 are combined to solve the modeling parameters k0 and k1 required to construct the structured light ranging relationship at the corresponding measurement angle, wherein the distance between the second position and the baseline is not equal to the distance between the first position and the baseline, that is, d2 is not equal to d1; then the modeling parameters k0 and k1 satisfy the structured light ranging relationship corresponding to all calibration plate images collected by the calibration plate moving along the slide rail, and then extend to satisfy the distance measurement requirements of the structured light module for the measured object outside it, that is, the structured light ranging relationship can be used to measure the distance of the measured object, that is, as the modeling parameters required for the ranging model of the structured light module. It should be noted that the solved invariant modeling parameters are the necessary parameters in the structured light ranging relationship followed by each frame of the calibration plate image and the corresponding light spot pixel coordinates during the sliding process of the calibration plate. Therefore, to construct this structured light ranging relationship, it is necessary to introduce the light spot pixel coordinates projected at multiple distance positions for solution in order to realize the calibration of the modeling parameters. When the measured object is displaced relative to the baseline, x will change, and Px will be displaced. When the calibration plate is displaced relative to the baseline, k0 and k1 remain unchanged at the same measurement angle. Therefore, it is only necessary to determine the two parameters k0 and k1 in the structured light ranging relationship to complete the modeling of the structured light module's ranging model.
[0084] In summary, the pixel coordinates of the light spots disclosed in this embodiment do not need to be converted into an internal parameter matrix according to the internal parameter calibration method disclosed in the prior art, but rely on a motor to drive the calibration plate to slide to different distances to establish a set of equations for the structured light ranging relationship, and solve the modeling parameters to perform camera calibration, thereby simplifying the amount of calculation of the transformation relationship from the camera coordinate system to the pixel coordinate system at each measurement angle; reducing the types and numbers of camera internal parameters that need to be calculated, and improving the calibration efficiency of the structured light module; and accelerating the execution of subsequent ranging operations when it is subsequently actually applied to ranging.
[0085] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.
Claims
1. A distance measurement method based on structured light module, It is characterized in that The structured light module includes a structured light emitter and a camera; The ranging method comprises: Step A, using a calibration device to solve the structured light ranging relationship constructed by the structured light module under the pinhole imaging principle to obtain modeling parameters; then using the modeling parameters as known quantities, converting the structured light ranging relationship into a ranging function; Step B, controlling the structured light emitter to emit structured light to the surface of the object to be measured and form light strips, and controlling the camera to collect an image of the structured light projected onto the surface of the object to be measured, wherein the image of the surface of the object to be measured includes the image of the light strips; Step C, using a light stripe center extraction algorithm to calculate the light stripe center pixel coordinates from the image captured by the structured light module; Step D: input the calculated light strip center pixel coordinates into the distance measurement function to calculate the distance between the measured object and the structured light module.
2. The distance measurement method according to claim 1, It is characterized in that After the light strip is imaged by the camera, it is represented by a number of corresponding columns of pixels in the pixel plane of the camera, and each column of pixels identifies the pixel position formed by each point constituting the light strip in the camera's photosensitive element; Each column of pixels forms a corresponding measurement angle relative to the optical axis of the camera, which is used to represent the orientation characteristics of each point projected by the light strip on the surface of the measured object relative to the optical center of the camera.
3. The distance measurement method according to claim 2, It is characterized in that A column of pixels corresponds to a column of light spot pixel coordinates or a light spot pixel coordinate; a structured light ranging relationship is constructed for each column of pixels in step A and corresponding modeling parameters are obtained, and the structured light ranging relationship constructed for each column of pixels in step A corresponds to a measurement angle.
4. The distance measurement method according to claim 3, It is characterized in that Whenever step C calculates the light strip center pixel coordinates at a measurement angle, step D converts the distance between the measured object and the structured light module at the corresponding measurement angle, representing the distance information at the orientation of the corresponding measurement angle; wherein, there is at least one light strip center pixel coordinate in a column of light spot pixel coordinates, so that when step D uses the ranging function for conversion, the light spot pixel coordinates are configured to be represented by the light strip center pixel coordinates.
5. The distance measurement method according to claim 2, It is characterized in that In step C, the light stripe center extraction algorithm is used to extract the light stripe center line in the image captured by the structured light module, and multiple light stripe center pixel coordinates are obtained to form a center line point set; in the image including the light stripe, a column of pixel points corresponds to at least one light stripe center pixel coordinate; Wherein, the light bar center extraction algorithm includes but is not limited to a grayscale centroid extraction algorithm.
6. The distance measurement method according to claim 3, It is characterized in that The step A comprises: Step A1, setting the positions of a calibration plate, a structured light emitter and a camera; wherein the calibration device includes a calibration plate; Step A2, controlling the structured light emitter to emit structured light, projecting it onto the calibration plate to form a light strip, and controlling the camera to collect an image of the calibration plate; and controlling the calibration plate to move relative to the structured light module, so that the camera obtains images of the calibration plate at different distances; Step A3, construct a structured light ranging relationship based on the pinhole imaging principle, and combine the calibration plate images at different distances to calculate the modeling parameters required to construct the structured light ranging relationship, wherein each column of light spot pixel coordinates corresponding to the structured light respectively constructs a corresponding structured light ranging relationship, and each column of light spot pixel coordinates corresponds to a set of modeling parameters.
7. The distance measurement method according to claim 6, It is characterized in that The step A1 specifically includes: The emission direction of the structured light emitter is adjusted to form an angle with the optical axis direction of the camera, and the line between the center of the structured light emitter and the center of the camera is set to be parallel to the measured surface of the calibration plate; the emission direction of the structured light emitter is set to intersect with the calibration plate, and the structured light plane formed by the structured light emitter intersects with the measured surface of the calibration plate to form the light strip.
8. The distance measurement method according to claim 7, It is characterized in that The step A2 comprises: The calibration plate is driven to slide on the slide rail; each time the calibration plate slides a certain distance along the slide rail, the camera collects an image of the calibration plate, the light bar deviates once on the calibration plate, and the camera detects the pixel coordinates of the light spot formed by the light bar in the pixel plane in real time; The calibration device further includes a slide rail and a fixture; the relative position between the structured light emitter and the camera is fixed by the fixture.
9. The distance measurement method according to claim 8, It is characterized in that In the step A2, during the sliding of the calibration plate along the slide rail, each time the calibration plate slides a preset fixed distance, the camera captures an image of the light strip, the imaging position of the light strip in the imaging plane is offset once along the height direction, and the pixel coordinates of the light point at the corresponding measurement angle are offset in the pixel plane in the opposite direction of the height direction by a preset longitudinal offset; Wherein, when the calibration plate slides to a position on the slide rail, the offset formed by the imaging position of the light strip in the imaging plane at each measuring angle relative to the reference position at the corresponding measuring angle is marked as the light strip offset at the corresponding measuring angle; Among them, the reference position corresponding to the measurement angle is set as the reference position, which is the imaging position in the camera of the reflected light of the structured light emitted by the structured light emitter on the measured surface of the measured object farthest from the structured light module within the detectable area of the structured light module; the measured object includes the calibration plate.
10. The distance measurement method according to claim 9, It is characterized in that In step A3, the method for constructing a structured light ranging relationship based on the pinhole imaging principle includes: The line between the center of the structured light emitter and the optical center of the camera is set as the baseline, and the length of the baseline is set as the baseline distance; The distance between the baseline at the corresponding measurement angle and the calibration plate is set as the test distance d at the corresponding measurement angle; The structured light ranging relationship corresponding to the measurement angle is: Px=k0 / d+k1, forming a relationship between the pixel coordinates of the light point in the direction represented by a column of pixels and the test distance; Wherein, k0 is the parameter to be calibrated in step A3; k1 is the parameter to be calibrated in step A3; Px is the pixel coordinate of the light spot at the corresponding measurement angle; The modeling parameters required to construct the structured light ranging relationship at the corresponding measurement angle include k0 and k1; At the same measurement angle, the ratio of the baseline distance to the test distance is equal to the ratio of the light bar offset to the focal length of the camera; At the same measurement angle, the sum of the product of the light spot pixel coordinates and the single pixel size and the projection error is equal to the light bar offset; the single pixel size and the projection error are both predetermined; the imaging of the light bar in the imaging plane of the camera is set in the imaging plane coordinate system, wherein the reference position and the light bar offset are both set in the imaging plane coordinate system; the light spot pixel coordinates are set in the pixel coordinate system, and the pixel coordinate system is the coordinate system of the pixel plane of the camera.
11. The distance measurement method according to claim 10, It is characterized in that In step A3, a structured light ranging equation is constructed based on the pinhole imaging principle, and the modeling parameters required to construct the structured light ranging equation are calculated by combining the calibration plate images at different distances, including: During the sliding of the calibration plate along the slide rail, at the same measuring angle: The calibration plate slides along the slide rail to the first position, and the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as Px1, and at the same time determines the test distance as d1; then constructs equation 1: Px1=k0 / d1+k1; The calibration plate slides along the slide rail to the second position, and the camera captures the image of the light strip and determines the pixel coordinates of the light spot formed in the pixel plane of the camera as Px2, and at the same time determines the test distance as d2; then constructs equation 2: Px2=k0 / d2+k1; Then, equation 1 and equation 2 are combined to solve the modeling parameters k0 and k1 required to construct the structured light ranging relationship at the corresponding measurement angle; wherein the distance between the second position and the baseline is not equal to the distance between the first position and the baseline; When the calibration plate is displaced relative to the baseline, at the same measurement angle, k0 and k1 remain unchanged.
12. The distance measurement method according to claim 10, It is characterized in that In step A3, the method of converting the structured light ranging relationship into a ranging function by taking the modeling parameters as known quantities includes: When it is determined that the structured light ranging relationship at the corresponding measurement angle is Px=k0 / d+k1, the modeling parameters required for the structured light ranging relationship at the corresponding measurement angle are all set to known quantities, and the light spot pixel coordinate Px at the corresponding measurement angle and the test distance d at the corresponding measurement angle are both set to unknown quantities; Then Px=k0 / d+k1 is converted to d=k0 / (Px-k1), and the converted d=k0 / (Px-k1) is recorded as the ranging function, where k0 and k1 are both known quantities.
