Calibration Method for Line Structured Light Plane Based on Mathematical Statistics Principle

Abstract

The present invention relates to the field of machine vision technology, and in particular to a calibration method for a line structured light plane based on the principle of mathematical statistics. First, a calibration board is photographed by a camera to obtain a calibration board image and multiple pairs of calibration images; multiple pairs of calibration images are randomly selected from the calibration images for structured light plane calibration to obtain corresponding error data; an error outlier rejection operation is performed on the error data; the calibration result corresponding to the smallest error data in the obtained error data is the structured light plane calibration result. Based on the principle of mathematical statistics, the present invention fits the light plane for a relatively large number of light plane feature points and statistically screens the light plane calibration results, improving the extraction accuracy of the center point of the light knife while also improving the calibration accuracy of the light plane.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and particularly relates to a calibration method for a line structured light plane based on mathematical statistics principles. Background Art

[0002] In an online structured light measurement system, the measurement accuracy depends on the calibration accuracy of the system, and the calibration of the line structured light is a key part of the system calibration. Low-quality line structured light calibration will generate obvious systematic errors, directly affecting the pose measurement accuracy of the vision system. Many methods proposed currently are based on the cross-ratio invariance of collinear point imaging. These methods use the points on the light stripe line as the fitting elements of the light plane, and the projection geometric properties used are simple, vulnerable to camera distortion, and involve multiple coordinate transformations and laser straight line equation fitting calculations during the calibration process. The superposition of the two inevitably brings errors to the light plane calibration, thus affecting the measurement accuracy of the vision measurement system. Summary of the Invention

[0003] In view of this, the present invention aims to provide a calibration method for a line structured light plane based on mathematical statistics principles. Based on mathematical statistics principles, the light plane is fitted with more light plane feature points, and the statistical screening of the light plane calibration results is carried out by using the error outlier rejection operation, which improves the extraction accuracy of the center point of the light knife while also improving the calibration accuracy of the light plane.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] A calibration method for a line structured light plane based on mathematical statistics principles, comprising:

[0006] S1: Control the line structured light to irradiate the calibration plate, and use a camera to take pictures of the calibration plate to obtain M pairs of calibration images;

[0007] S2: Randomly select N pairs of calibration images from the M pairs of calibration images for structured light plane calibration to obtain error data;

[0008] S3: Repeat step S2 for P times to obtain P error data; perform an error outlier rejection operation on the P error data;

[0009] S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

[0010] Further, step S1 includes:

[0011] S11: Use a camera to take pictures of the calibration plate to obtain a calibration plate image;

[0012] S12: Keep the pose of the calibration board unchanged, irradiate the calibration board with line structured light, and use the camera to capture the calibration board to obtain a structured light image;

[0013] S13: The structured light image obtained in step S12 and the calibration board image captured in step S11 form a pair of calibration images; adjust the pose of the calibration board, and repeat steps S11 - S12 for M times to obtain M pairs of calibration images.

[0014] Further, step S2 includes:

[0015] S21: Determine the external parameter transformation matrix from the calibration board coordinate system of the calibration board to the camera coordinate system of the camera according to the calibration board image in the calibration images obtained in step S1;

[0016] S22: Use the external parameter transformation matrix obtained in step S21 to calculate the first spatial plane equation of the calibration board in the camera coordinate system;

[0017] S23: Identify and segment the structured light strips in the structured light image in the calibration image, and extract the set of center points from the segmented structured light knives;

[0018] S24: According to the pinhole imaging model and the first spatial plane equation obtained in step S22, calculate the spatial coordinates of the set of center points obtained in step S23 in the camera coordinate system;

[0019] S25: Repeat steps S21 - S24 to process N pairs of calibration images, and correspondingly obtain N sets of spatial coordinates; perform plane fitting on the N sets of spatial coordinates to obtain the second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result;

[0020] S26: Calculate the distance from the points on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as the error data.

[0021] Further, before step S21, it also includes: Determine the camera internal parameter matrix and camera distortion coefficient of the camera according to the calibration board image in the calibration images obtained in step S1.

[0022] Further, in step S22, use the external parameter transformation matrix obtained in step S21 to convert the first spatial coordinates of the calibration points on the calibration board in the calibration board coordinate system into the second spatial coordinates in the camera coordinate system; use the least squares method to fit the second spatial coordinates to obtain the first spatial plane equation.

[0023] Further, step S23 includes:

[0024] S231: Perform object recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light knife as the foreground;

[0025] S232: Use the gray centroid method to extract the skeleton of the structured light knife to obtain a preliminary set of center points;

[0026] S233: Use the RANSAC method to perform linear fitting on the preliminary set of center points obtained in step S232, and then substitute the abscissas in the preliminary set of center points into the linear equation obtained by fitting to obtain the set of center points.

[0027] Further, in step S25, perform least squares plane fitting on N groups of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light.

[0028] Further, the error outlier rejection operation in step S3 includes:

[0029] S31: Calculate the mean of the error data, and count the error data that meets the mean threshold range, as well as the number of error data that meets the mean threshold range;

[0030] S32: Re-execute step S31 with the error data obtained in step S31 until the number of error data that meets the mean threshold range no longer changes.

[0031] Further, the mean threshold range is , where represents the mean of the error data, and represents the boundary of the mean threshold range.

[0032] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0033] (1) In the calibration method of the line structured light plane based on the principle of mathematical statistics of the present invention, based on the principle of mathematical statistics, perform plane fitting on more light plane feature points, and use the error outlier rejection operation to perform statistical screening on the light plane calibration result, which improves the extraction accuracy of the light knife center point while also improving the calibration accuracy of the light plane. The process of the present invention is simple and convenient, and is suitable for engineering applications.

[0034] (2) In the calibration method of the line structured light plane based on the principle of mathematical statistics of the present invention, a linear fitting method based on RANSAC is used to perform secondary fitting correction on the extracted skeleton to reduce the influence of noise points on the calibration accuracy of the light plane. Description of the Drawings

[0035] The accompanying drawings, which form a part of the present invention, are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and shall not unduly limit the present invention. In the drawings:

[0036] Figure 1 It is a schematic flowchart of the calibration method for the line structured light plane based on the principle of mathematical statistics according to the embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the calibration method for the line structured light plane based on the principle of mathematical statistics according to the embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the coordinate system conversion according to the embodiment of the present invention.

[0039] Description of reference numerals:

[0040] 1. Camera; 2. Calibration board; 3. Calibration point; 4. Line structured light. Detailed implementation manners

[0041] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following further details the present invention in combination with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0042] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0043] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0044] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0046] As Figure 1 shown, the calibration method of the line structured light plane based on the principle of mathematical statistics includes:

[0047] S1: Control the line structured light 4 to irradiate the calibration plate 2, and use the camera 1 to photograph the calibration plate 2 to obtain M pairs of calibration images.

[0048] In some embodiments, as Figure 2 shown, step S1 includes:

[0049] S11: Use the camera 1 to photograph the calibration plate 2 to obtain a calibration plate image. In a certain embodiment, the calibration points 3 in the calibration plate 2 are circular dots, and the distance between two adjacent calibration points 3 is 40 mm. Taking the calibration plate 2 in a certain pose as an example, taking the calibration point 3 in the upper left corner of the calibration plate 2 as the coordinate origin, the horizontal right direction as the positive x-axis direction, the vertical downward direction as the positive y-axis direction, and the direction perpendicular to the calibration plate and inward as the positive z-axis direction, a calibration plate coordinate system is established.

[0050] S12: Keep the pose of the calibration plate 2 unchanged, use the line structured light 4 to irradiate the calibration plate 2, and use the camera 1 to photograph the calibration plate 2 to obtain a structured light image.

[0051] S13: The structured light image obtained in step S12 and the calibration plate image photographed in step S11 are a pair of calibration images; adjust the pose of the calibration plate 2, and repeat steps S11 - S12 M times to obtain M pairs of calibration images. In a certain embodiment, repeat steps S11 - S12 100 times to obtain 100 pairs of calibration images.

[0052] S2: Randomly select N pairs of calibration images from M pairs of calibration images for structured light plane calibration to obtain error data.

[0053] In a certain embodiment, randomly select 20 pairs of calibration images from 100 pairs of calibration images for structured light plane calibration to obtain error data.

[0054] In some embodiments, step S2 includes:

[0055] S21: Determine the external parameter transformation matrix from the calibration board coordinate system of the calibration board to the camera coordinate system of the camera based on the calibration board image in the calibration image obtained in step S1. Each pair of calibration images shares one external parameter transformation matrix.

[0056] In a certain embodiment, before step S21, it further includes: determining the camera internal parameter matrix and the camera distortion coefficient of the camera based on the calibration board image in the calibration image obtained in step S1.

[0057] Specifically, the camera internal parameter matrix of the camera is expressed as:

[0058] ;

[0059] Among them, represents the camera internal parameter matrix, and respectively represent the ratio of the focal length of the camera to the pixel size, and represent the principal point coordinates of the camera;

[0060] The camera distortion coefficient of the camera is expressed as:

[0061] ;

[0062] Among them, and represent the radial distortion coefficients of the camera, and represent the tangential distortion coefficients of the camera;

[0063] The external parameter transformation matrix is expressed as:

[0064] ;

[0065] Among them, represents the external parameter transformation matrix, represents a 3×3 rotation matrix, represents a 3×1 translation matrix.

[0066] The actual visual measurement process is a coordinate transformation process from the calibration board coordinate system to the camera coordinate system, from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system. As Figure 3 shown, the first spatial coordinate of the calibration point in the calibration board coordinate system is ; After taking a picture with the camera, at this time, the coordinate of the calibration point in the image coordinate system of the picture is , the pixel coordinate of the calibration point in the pixel coordinate system is , and the second spatial coordinate of the calibration point in the camera coordinate system is , the camera coordinate system takes the optical center of the camera as the coordinate origin, and the X and Y directions of the camera coordinate system are parallel to the X and Y directions of the image coordinate system respectively.

[0067] Specifically, the conversion relationship from the calibration board coordinate system to the pixel coordinate system can be expressed as:

[0068] .

[0069] S22: Use the external parameter transformation matrix obtained in step S21 to calculate the first spatial plane equation of the calibration board in the camera coordinate system.

[0070] In one embodiment, in step S22, use the external parameter transformation matrix obtained in step S21 to convert the first spatial coordinates of the calibration points on the calibration board in the calibration board coordinate system into the second spatial coordinates in the camera coordinate system; use the least squares method to fit the second spatial coordinates to obtain the first spatial plane equation.

[0071] Specifically, use the external parameter transformation matrix and the first spatial coordinates , and obtain the second spatial coordinates through the following formula :

[0072] ;

[0073] Use the least squares method to perform plane fitting on the second spatial coordinates , and the expression of the fitting plane is:

[0074] ;

[0075] Among them, , , and represent the parameters in the expression of the fitting plane, . Transform the above expression:

[0076] ;

[0077] Another , , , at this time the above expression is transformed into:

[0078] ;

[0079] At this time, the corresponding least squares matrix equation is:

[0080] ;

[0081] Among them, Indicates the coordinates on the structured light blade skeleton of the structured light bars in the structured light image in the calibration image. Using the solution formula of the normal equations, the fitting plane parameters can be obtained. .

[0082] S23: Identify and segment the structured light bars in the structured light image in the calibration image, and extract the set of center points from the segmented structured light blades.

[0083] In one embodiment, step S23 includes:

[0084] S231: Perform object recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light blade as the foreground. Specifically, extract the contour of the structured light blade in the structured light image, and use image binarization to extract the foreground image of the structured light blade. This foreground image is the segmented image.

[0085] S232: Use the gray centroid method to extract the skeleton of the structured light blade to obtain a preliminary set of center points.

[0086] Specifically, the gray centroid method is used to extract the gray centroid of each column in the segmented image as the center position of the structured light blade. If the non-zero interval of a certain column in the segmented image is [p, q] (i.e., the p-th to q-th rows in a column of the segmented image), then the ordinate of the gray centroid position of this column is:

[0087] ;

[0088] where represents the ordinate of the gray centroid position, represents the gray value of the i-th row image in this column. The obtained set of gray centroid positions is the preliminary set of center points.

[0089] S233: Use the RANSAC method to perform linear fitting on the preliminary set of center points obtained in step S232, and then substitute the abscissas in the preliminary set of center points into the obtained linear equation to obtain the set of center points.

[0090] It can be understood that since there may be outliers in the process of identifying and segmenting the structured light blade, and the outliers will affect the extraction of the structured light blade skeleton, the present invention uses the RANSAC method to perform secondary fitting on the basis of the result extracted by the gray centroid method to improve the extraction accuracy of the light blade skeleton.

[0091] In one embodiment, the preliminary set of center points is , and using the RANSAC method to perform linear fitting on all points, the linear equation outside the outliers can be obtained. The linear equation is a linear function. Substitute the abscissas of all points in the preliminary set of center points into the linear equation to obtain the corresponding ordinates , the vertical coordinate obtained at this time is the corrected ordinate, and the center point set is .

[0092] S24: Calculate the spatial coordinates of the center point set obtained in step S23 in the camera coordinate system according to the pinhole imaging model and the first spatial plane equation obtained in step S22.

[0093] Specifically, in step S22, the first space plane equation is obtained as:

[0094] ;

[0095] The pinhole imaging model is:

[0096] ;

[0097] in, and They are all intermediate variables in the calculation process and have no actual meaning;

[0098] Combining the first space plane equation with the pinhole imaging model, we get:

[0099] ;

[0100] S25: Repeat steps S21 to S24 to process N pairs of calibration images, and obtain N sets of spatial coordinates; perform plane fitting on the N sets of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result.

[0101] In some embodiments, the N groups of spatial coordinates are subjected to least squares plane fitting to obtain a second spatial plane equation of the spatial plane of the line structured light. The fitting process is the same as the principle of fitting the first spatial plane equation, which will not be repeated here.

[0102] In one embodiment, steps S21 to S24 are repeated to process 20 pairs of calibration images, thereby obtaining 20 sets of spatial coordinates.

[0103] S26: Calculate the distance from the point on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as error data.

[0104] S3: Repeat step S2 P times to obtain P error data; perform error outlier elimination operation on the P error data.

[0105] In some embodiments, the error outlier removal operation of step S3 includes:

[0106] S31: Calculate the mean value of the error data, and count the error data that fall within the mean threshold range, as well as the number of error data that fall within the mean threshold range. Further, the mean threshold range is , where represents the mean value of the error data, and represents the boundaries of the mean threshold range, and the boundaries of the mean threshold range are adaptively adjusted according to the actual situation.

[0107] S32: Re-execute step S31 with the error data obtained in step S31 until the number of error data that fall within the mean threshold range no longer changes.

[0108] In one embodiment, step S2 is repeated 100 times to obtain 100 error data; an outlier rejection operation is performed on the 100 error data. Specifically, the outlier rejection operation on the 100 error data includes:

[0109] Calculate the mean value of the 100 error data, and count the error data that fall within the mean threshold range, as well as the number of error data that fall within the mean threshold range. This process is regarded as one outlier rejection operation. For ease of understanding, assume that the mean value of the 100 error data is , and the boundaries of the mean threshold range are , then the corresponding mean threshold range is , and at this time, the number of error data that fall within the mean threshold range is 60;

[0110] Calculate the mean value of the 60 error data that fall within the mean threshold range. Assume that the mean value of the 60 error data at this time is , then the corresponding mean threshold range is , and once again count the error data that fall within the mean threshold range and their number among the 60 error data, that is, perform another outlier rejection operation;

[0111] Repeat the above operation until the number of error data that fall within the mean threshold range no longer changes.

[0112] S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

[0113] It should be understood that various forms of the flow shown above can be used, and steps can be reordered, added, or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitation is made herein.

[0114] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A calibration method for a line structured light plane based on the principle of mathematical statistics, characterized in that, include: S1: Control the line structured light to illuminate the calibration plate, and use a camera to shoot the calibration plate to obtain M pairs of calibration images; S2: Randomly select N pairs of calibration images from M pairs of calibration images to perform structured light plane calibration to obtain error data; Step S2 includes: S21: determining an extrinsic parameter transformation matrix from a calibration plate coordinate system of the calibration plate to a camera coordinate system of the camera according to the calibration plate image in the calibration image obtained in step S1; S22: Calculate the first space plane equation of the calibration plate in the camera coordinate system using the extrinsic transformation matrix obtained in step S21; S23: identifying and segmenting the structured light strips in the structured light image in the calibration image, and extracting a set of center points from the structured light strips obtained by segmentation; S24: Calculating the spatial coordinates of the center point set obtained in step S23 in the camera coordinate system according to the pinhole imaging model and the first spatial plane equation obtained in step S22; S25: Repeat steps S21 to S24 to process N pairs of calibration images, and obtain N sets of spatial coordinates; perform plane fitting on the N sets of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result; S26: Calculate the distance from the point on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as the error data; S3: Repeat step S2 P times to obtain P error data; perform error outlier elimination operation on the P error data; S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

2. The calibration method of the line structured light plane based on the principle of mathematical statistics according to claim 1, characterized in that Step S1 includes: S11: photographing the calibration plate using the camera to obtain a calibration plate image; S12: keeping the position and posture of the calibration plate unchanged, irradiating the calibration plate with the line structured light, and photographing the calibration plate with the camera to obtain a structured light image; S13: The structured light image obtained in step S12 and the calibration plate image captured in step S11 are a pair of calibration images; the position and posture of the calibration plate are adjusted, and steps S11 to S12 are repeated M times to obtain M pairs of calibration images.

3. The calibration method of the line structured light plane based on the principle of mathematical statistics according to claim 1, characterized in that, Before step S21, the method further includes: determining a camera intrinsic parameter matrix and a camera distortion coefficient of the camera according to the calibration plate image in the calibration image obtained in step S1.

4. The calibration method of the line structured light plane based on the mathematical statistics principle according to claim 1, characterized in that In step S22, the extrinsic parameter transformation matrix obtained in step S21 is used to convert the first spatial coordinates of the calibration point on the calibration plate in the calibration plate coordinate system into the second spatial coordinates in the camera coordinate system; the second spatial coordinates are fitted using the least squares method to obtain the first spatial plane equation.

5. The calibration method of the line structured light plane based on the principle of mathematical statistics according to claim 1, characterized in that, Step S23 includes: S231: performing target recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light knife as the foreground; S232: extracting the skeleton of the structured light knife using a grayscale centroid method to obtain a preliminary center point set; S233: Use the steps of the RANSAC method to perform linear fitting on the preliminary center point set obtained in S232, and then substitute the abscissas in the preliminary center point set into the linear equation obtained by fitting to obtain the center point set.

6. The calibration method of the line structured light plane based on the mathematical statistics principle according to claim 1, characterized in that, In step S25, perform least squares plane fitting on N groups of spatial coordinates to obtain the second spatial plane equation of the spatial plane of the line structured light.

7. The calibration method of the line structured light plane based on the principle of mathematical statistics according to claim 1, characterized in that The error outlier rejection operation in step S3 includes: S31: Calculate the mean of the error data, and count the error data that meets the mean threshold range and the number of error data that meets the mean threshold range; S32: Re-execute step S31 with the error data obtained in step S31 until the number of error data that meets the mean threshold range no longer changes.

8. The calibration method of the line structured light plane based on the mathematical statistics principle according to claim 7, characterized in that, The mean threshold range is , where represents the mean of the error data, represents the boundary of the mean threshold range.

Images