Line structured light plane calibration method based on mathematical statistics principle

Abstract

The invention relates to the technical field of machine vision, in particular to a linear structured light plane calibration method based on a mathematical statistics principle, which comprises the following steps: firstly, shooting a calibration plate by using a camera to obtain a calibration plate image to obtain a plurality of pairs of calibration images; randomly extracting a plurality of pairs of calibration images from the calibration images for structured light plane calibration to obtain corresponding error data; error outlier elimination operation is carried out on the error data; the calibration result corresponding to the minimum error data in the obtained error data is the structured light plane calibration result. According to the method, based on the mathematical statistics principle, light plane fitting is carried out on more light plane feature points, statistical screening is carried out on light plane calibration results, and the light plane calibration precision is improved while the light knife center point extraction precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and in particular relates to a calibration method of a line structured light plane based on the principle of mathematical statistics. Background Art

[0002] In the online structured light measurement system, the measurement accuracy depends on the accuracy of the system calibration, and the calibration of line structured light is a key part of the system calibration. Low-quality line structured light calibration will produce obvious system errors, which directly affect the position and posture measurement accuracy of the visual system. Many methods proposed at present are based on the cross-ratio invariance of collinear point imaging. These methods use points on the straight line of the light strip as the fitting elements of the light plane. The projection geometry used is simple and easily affected by camera distortion. In addition, the calibration process involves multiple coordinate transformations and laser line equation fitting calculations. The superposition of the two will inevitably bring errors to the light plane calibration, thereby affecting the measurement accuracy of the visual measurement system. Summary of the invention

[0003] In view of this, the present invention aims to provide a calibration method for a line structured light plane based on the principle of mathematical statistics. Based on the principle of mathematical statistics, the light plane is fitted for more light plane feature points, and the light plane calibration results are statistically screened by using an error outlier removal operation. While improving the light knife center point extraction accuracy, the light plane calibration accuracy is also improved.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows: A calibration method for a line structured light plane based on mathematical statistics principles, comprising: S1: Control the line structured light to illuminate the calibration plate, and use the camera to shoot the calibration plate to obtain M pairs of calibration images; S2: Randomly select N pairs of calibration images from M pairs of calibration images to perform structured light plane calibration to obtain error data; S3: Repeat step S2 P times to obtain P error data; perform error outlier elimination operation on the P error data; S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

[0005] Further, step S1 includes: S11: photographing the calibration plate using a camera to obtain an image of the calibration plate; S12: Keeping the position and posture of the calibration plate unchanged, irradiating the calibration plate with line structured light, and photographing the calibration plate with a camera to obtain a structured light image; S13: The structured light image obtained in step S12 and the calibration plate image captured in step S11 are a pair of calibration images; adjust the posture of the calibration plate, and repeat steps S11 to S12 M times to obtain M pairs of calibration images.

[0006] Further, step S2 includes: S21: According to the calibration plate image in the calibration image obtained in step S1, determine the extrinsic parameter transformation matrix between the calibration plate coordinate system of the calibration plate and the camera coordinate system of the camera; S22: Calculate the first space plane equation of the calibration plate in the camera coordinate system using the external parameter transformation matrix obtained in step S21; S23: identifying and segmenting the structured light strips in the structured light image in the calibration image, and extracting a set of center points from the structured light blade obtained by segmentation; S24: Calculate the spatial coordinates of the center point set obtained in step S23 in the camera coordinate system according to the pinhole imaging model and the first spatial plane equation obtained in step S22; S25: Repeat steps S21 to S24 to process N pairs of calibration images, and obtain N sets of spatial coordinates; perform plane fitting on the N sets of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result; S26: Calculate the distance from the point on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as error data.

[0007] Furthermore, before step S21, the method further includes: determining a camera intrinsic parameter matrix and a camera distortion coefficient of the camera according to the calibration plate image in the calibration image obtained in step S1.

[0008] Furthermore, in step S22, the extrinsic parameter transformation matrix obtained in step S21 is used to convert the first spatial coordinates of the calibration point on the calibration plate in the calibration plate coordinate system into the second spatial coordinates in the camera coordinate system; the second spatial coordinates are fitted using the least squares method to obtain the first spatial plane equation.

[0009] Further, step S23 includes: S231: performing target recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light knife as the foreground; S232: Use the grayscale centroid method to extract the skeleton of the structured light knife and obtain a preliminary center point set; S233: Use the RANSAC method to perform straight line fitting on the preliminary center point set obtained in step S232, and then substitute the horizontal coordinates in the preliminary center point set into the straight line equation obtained by fitting to obtain the center point set.

[0010] Furthermore, in step S25, the N groups of spatial coordinates are subjected to least square plane fitting to obtain a second spatial plane equation of the spatial plane of the line structured light.

[0011] Furthermore, the error outlier elimination operation of step S3 includes: S31: Calculate the mean of the error data, and count the error data that meet the mean threshold range, and the number of error data that meet the mean threshold range; S32: re-execute step S31 with the error data obtained in step S31 until the number of error data that meets the mean threshold range no longer changes.

[0012] Furthermore, the mean threshold range is ,in, represents the mean of the error data, Indicates the boundaries of the mean threshold range.

[0013] Compared with the prior art, the invention can achieve the following beneficial effects: (1) In the calibration method of the line structure light plane based on the principle of mathematical statistics described in the invention, the light plane is fitted for a large number of light plane feature points based on the principle of mathematical statistics, and the light plane calibration results are statistically screened by using the error outlier elimination operation, which improves the light knife center point extraction accuracy while also improving the light plane calibration accuracy. The process of the invention is simple and convenient, and is suitable for engineering applications.

[0014] (2) In the calibration method of the line structured light plane based on the principle of mathematical statistics described in the present invention, a straight line fitting method based on RANSAC is used to perform a secondary fitting correction on the extracted skeleton to reduce the influence of noise points on the calibration accuracy of the light plane. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of a flow chart of a method for calibrating a line structured light plane based on the principle of mathematical statistics according to an embodiment of the present invention; Figure 2 A schematic diagram of a calibration method for a line structured light plane based on the principle of mathematical statistics according to an embodiment of the present invention; Figure 3 A schematic diagram of the coordinate system conversion described in the embodiment of the present invention.

[0016] Description of reference numerals: 1. Camera; 2. Calibration plate; 3. Calibration points; 4. Line structured light. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0018] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0019] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0020] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0021] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0022] like Figure 1 As shown, the calibration method of the line structured light plane based on the principle of mathematical statistics includes: S1: Control the line structured light 4 to illuminate the calibration plate 2, and use the camera 1 to shoot the calibration plate 2 to obtain M pairs of calibration images.

[0023] In some embodiments, Figure 2As shown, step S1 includes: S11: Use camera 1 to shoot calibration plate 2 to obtain calibration plate image. In a certain embodiment, calibration point 3 in calibration plate 2 is a circle, and the distance between two adjacent calibration points 3 is 40 mm. Take calibration plate 2 in a certain posture as an example, take calibration point 3 in the upper left corner of calibration plate 2 as the origin of coordinates, horizontally to the right as the positive direction of x-axis, vertically downward as the positive direction of y-axis, and vertically inward of calibration plate as the positive direction of z-axis, and establish calibration plate coordinate system.

[0024] S12: Keeping the position and posture of the calibration plate 2 unchanged, irradiating the calibration plate 2 with the line structured light 4, and photographing the calibration plate 2 with the camera 1 to obtain a structured light image.

[0025] S13: The structured light image obtained in step S12 and the calibration plate image captured in step S11 are a pair of calibration images; the position and posture of the calibration plate 2 are adjusted, and steps S11-S12 are repeated M times to obtain M pairs of calibration images. In one embodiment, steps S11-S12 are repeated 100 times to obtain 100 pairs of calibration images.

[0026] S2: Randomly select N pairs of calibration images from M pairs of calibration images to perform structured light plane calibration to obtain error data.

[0027] In a certain embodiment, 20 pairs of calibration images are randomly selected from 100 pairs of calibration images to perform structured light plane calibration to obtain error data.

[0028] In some embodiments, step S2 includes: S21: According to the calibration plate image in the calibration image obtained in step S1, determine the extrinsic parameter transformation matrix between the calibration plate coordinate system of the calibration plate and the camera coordinate system of the camera, and each pair of calibration images shares one extrinsic parameter transformation matrix.

[0029] In one embodiment, before step S21, the method further includes: determining a camera intrinsic parameter matrix and a camera distortion coefficient of the camera according to a calibration plate image in the calibration image obtained in step S1.

[0030] Specifically, the camera intrinsic parameter matrix of the camera is expressed as: ; in, represents the camera intrinsic parameter matrix, and They represent the ratio of the camera’s focal length to the pixel size, and Represents the principal point coordinates of the camera; The camera distortion coefficient of a camera is expressed as: ; in, and represents the radial distortion coefficient of the camera, and Represents the tangential distortion coefficient of the camera; The external parameter transformation matrix is ​​expressed as: ; in, represents the extrinsic transformation matrix, represents a 3×3 size rotation matrix, Represents a 3×1 translation matrix.

[0031] The visual measurement process is actually a coordinate system conversion process from the calibration plate coordinate system to the camera coordinate system, from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system. Figure 3 As shown, the first spatial coordinate of the calibration point in the calibration plate coordinate system is ; After the camera takes the picture, the coordinates of the calibration point in the picture coordinate system are , the pixel coordinates of the calibration point in the pixel coordinate system are , the second space coordinates of the calibration point in the camera coordinate system are The camera coordinate system takes the optical center of the camera as the coordinate origin, and the X direction and Y direction of the camera coordinate system are parallel to the X direction and Y direction of the image coordinate system respectively.

[0032] Specifically, the conversion relationship from the calibration plate coordinate system to the pixel coordinate system can be expressed as: .

[0033] S22: Using the external parameter transformation matrix obtained in step S21, calculate the first space plane equation of the calibration plate in the camera coordinate system.

[0034] In a certain embodiment, in step S22, the extrinsic parameter transformation matrix obtained in step S21 is used to convert the first spatial coordinates of the calibration point on the calibration plate in the calibration plate coordinate system into the second spatial coordinates in the camera coordinate system; the second spatial coordinates are fitted using the least squares method to obtain the first spatial plane equation.

[0035] Specifically, using the external parameter transformation matrix and the first space coordinate , the second space coordinates are obtained by the following formula : ; The second space coordinates are calculated using the least squares method Perform plane fitting, the expression of the fitting plane is: ; in, , , and represents the parameters in the fitting plane expression, . Transform the above expression: ; Other , , , then the above expression is transformed into: ; At this time, the corresponding least squares matrix equation is: ; in, Represents the coordinates of the structured light strips in the structured light image in the calibration image on the structured light knife skeleton. Using the solution formula of the normal equations, the fitting plane parameters can be obtained .

[0036] S23: identifying and segmenting the structured light strips in the structured light image in the calibration image, and extracting a set of center points from the structured light strips obtained by segmentation.

[0037] In one embodiment, step S23 includes: S231: Perform target recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light knife as the foreground. Specifically, perform contour extraction on the structured light knife in the structured light image, and use image binarization to extract the foreground image of the structured light knife, which is the segmented image.

[0038] S232: Use the grayscale centroid method to extract the skeleton of the structured light knife to obtain a preliminary center point set.

[0039] Specifically, the grayscale centroid method is used to extract the grayscale centroid of each column in the segmented image as the center position of the structured light knife. If the non-zero interval of a column in the segmented image is [p, q] (i.e., rows p to q in a column of the segmented image), the ordinate of the grayscale centroid position of the column is: ; in, The vertical coordinate represents the grayscale centroid position. Represents the gray value of the i-th row image in the column. The set of grayscale centroid positions obtained is the preliminary center point set.

[0040] S233: Use the RANSAC method to perform straight line fitting on the preliminary center point set obtained in step S232, and then substitute the horizontal coordinates in the preliminary center point set into the straight line equation obtained by fitting to obtain the center point set.

[0041] It is understandable that there may be wild points in the process of structured light knife recognition and segmentation, which will affect the extraction of structured light knife skeleton. Therefore, the present invention adopts RANSAC method to perform secondary fitting on the basis of the results of grayscale centroid method extraction to improve the extraction accuracy of light knife skeleton. In one embodiment, the initial center point set is , the RANSAC method is used to fit all points to obtain the equation of the line outside the wild points. The equation of the line is a linear function. The horizontal coordinates of all points in the preliminary center point set are Substitute into the equation of the line and get the corresponding ordinate , the vertical coordinate obtained at this time is the corrected ordinate, and the center point set is .

[0042] S24: Calculate the spatial coordinates of the center point set obtained in step S23 in the camera coordinate system according to the pinhole imaging model and the first spatial plane equation obtained in step S22.

[0043] Specifically, in step S22, the first space plane equation is obtained as: ; The pinhole imaging model is: ; in, and They are all intermediate variables in the calculation process and have no actual meaning; Combining the first space plane equation with the pinhole imaging model, we get: ; S25: Repeat steps S21 to S24 to process N pairs of calibration images, and obtain N sets of spatial coordinates; perform plane fitting on the N sets of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result.

[0044] In some embodiments, the N groups of spatial coordinates are subjected to least squares plane fitting to obtain a second spatial plane equation of the spatial plane of the line structured light. The fitting process is the same as the principle of fitting the first spatial plane equation, which will not be repeated here.

[0045] In one embodiment, steps S21 to S24 are repeated to process 20 pairs of calibration images, thereby obtaining 20 sets of spatial coordinates.

[0046] S26: Calculate the distance from the point on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as error data.

[0047] S3: Repeat step S2 P times to obtain P error data; perform error outlier elimination operation on the P error data.

[0048] In some embodiments, the error outlier removal operation of step S3 includes: S31: Calculate the mean of the error data, and count the error data that meet the mean threshold range, and the number of error data that meet the mean threshold range. Further, the mean threshold range is ,in, represents the mean of the error data, Indicates the boundary of the mean threshold range, which is adaptively adjusted according to actual conditions.

[0049] S32: re-execute step S31 with the error data obtained in step S31 until the number of error data that meets the mean threshold range no longer changes.

[0050] In a certain embodiment, step S2 is repeated 100 times to obtain 100 error data; and the error outlier elimination operation is performed on the 100 error data. Specifically, the error outlier elimination operation on the 100 error data includes: Calculate the mean of 100 error data, and count the error data that meet the mean threshold range, as well as the number of error data that meet the mean threshold range. This process is regarded as an error outlier removal operation. For ease of understanding, assume that the mean of 100 error data is , the boundaries of the mean threshold range , then the corresponding mean threshold range is , at this time, the number of error data that meets the mean threshold range is 60; Calculate the mean of 60 error data that meet the mean threshold range. Assume that the mean of 60 error data is , then the corresponding mean threshold range is , count the error data and their number that meet the mean threshold range among the 60 error data again, that is, perform another error outlier elimination operation; Repeat the above operation until the number of error data that meets the mean threshold range no longer changes.

[0051] S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

[0052] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0053] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A calibration method for a line structured light plane based on mathematical statistics, characterized in that: include: S1: Control the line structured light to illuminate the calibration plate, and use a camera to shoot the calibration plate to obtain M pairs of calibration images; S2: Randomly select N pairs of calibration images from M pairs of calibration images to perform structured light plane calibration to obtain error data; S3: Repeat step S2 P times to obtain P error data; perform error outlier elimination operation on the P error data; S4: The calibration result corresponding to the smallest error data among the error data obtained in step S3 is the structured light plane calibration result.

2. The calibration method of line structured light plane based on mathematical statistics principle according to claim 1, characterized in that: Step S1 includes: S11: photographing the calibration plate using the camera to obtain a calibration plate image; S12: keeping the position and posture of the calibration plate unchanged, irradiating the calibration plate with the line structured light, and photographing the calibration plate with the camera to obtain a structured light image; S13: The structured light image obtained in step S12 and the calibration plate image captured in step S11 are a pair of calibration images; the position and posture of the calibration plate are adjusted, and steps S11 to S12 are repeated M times to obtain M pairs of calibration images.

3. The calibration method of line structured light plane based on mathematical statistics principle according to claim 1, characterized in that: Step S2 includes: S21: determining an extrinsic parameter transformation matrix from a calibration plate coordinate system of the calibration plate to a camera coordinate system of the camera according to the calibration plate image in the calibration image obtained in step S1; S22: Calculate the first space plane equation of the calibration plate in the camera coordinate system using the extrinsic transformation matrix obtained in step S21; S23: identifying and segmenting the structured light strips in the structured light image in the calibration image, and extracting a set of center points from the structured light strips obtained by segmentation; S24: Calculating the spatial coordinates of the center point set obtained in step S23 in the camera coordinate system according to the pinhole imaging model and the first spatial plane equation obtained in step S22; S25: Repeat steps S21 to S24 to process N pairs of calibration images, and obtain N groups of spatial coordinates; perform plane fitting on the N groups of spatial coordinates to obtain a second spatial plane equation of the spatial plane of the line structured light, and the second spatial plane equation is the calibration result; S26: Calculate the distance from the point on the structured light strip in each structured light image to the spatial plane calculated in step S25; calculate the average value of all distances as the error data.

4. The calibration method of line structured light plane based on mathematical statistics principle according to claim 3, characterized in that: Before step S21, the method further includes: determining a camera intrinsic parameter matrix and a camera distortion coefficient of the camera according to the calibration plate image in the calibration image obtained in step S1.

5. The calibration method of line structured light plane based on mathematical statistics principle according to claim 3, characterized in that: In step S22, the extrinsic parameter transformation matrix obtained in step S21 is used to convert the first spatial coordinates of the calibration point on the calibration plate in the calibration plate coordinate system into the second spatial coordinates in the camera coordinate system; the second spatial coordinates are fitted using the least squares method to obtain the first spatial plane equation.

6. The calibration method of line structured light plane based on mathematical statistics principle according to claim 3, characterized in that: Step S23 includes: S231: performing target recognition and segmentation processing on the structured light image to obtain a segmented image with only the structured light knife as the foreground; S232: extracting the skeleton of the structured light knife using a grayscale centroid method to obtain a preliminary center point set; S233: Use the RANSAC method to perform straight line fitting on the preliminary center point set obtained in S232, and then substitute the horizontal coordinates in the preliminary center point set into the straight line equation obtained by fitting to obtain the center point set.

7. The calibration method of line structured light plane based on mathematical statistics principle according to claim 3, characterized in that: In step S25, the N groups of spatial coordinates are subjected to least square plane fitting to obtain a second spatial plane equation of the spatial plane of the line structured light.

8. The calibration method of line structured light plane based on mathematical statistics principle according to claim 1, characterized in that: The error outlier elimination operation of step S3 includes: S31: Calculate the mean of the error data, and count the error data that meet the mean threshold range, and the number of error data that meet the mean threshold range; S32: re-execute step S31 with the error data obtained in step S31 until the number of error data that meets the mean threshold range no longer changes.

9. The calibration method of line structured light plane based on mathematical statistics principle according to claim 8, characterized in that: The mean threshold range is ,in, represents the mean of the error data, Represents the boundaries of the mean threshold range.

Images