Method and device for correcting image shot based on structured light

Abstract

The invention discloses a correction method and device based on a structured light shot image. The method comprises the following steps: acquiring image point cloud data and two-dimensional image data through a structured light device; performing denoising processing on the point cloud data; fitting a plane by using all or part of the point cloud data to obtain a plane equation; respectively calculating included angles alpha and beta between the point cloud plane and the X axis and the Y axis of the coordinate system through a plane equation; selecting four coordinate points which are not on the same line on the two-dimensional image, and calculating four coordinate points which are mapped to the horizontal direction according to the included angles alpha and beta; according to the one-to-one correspondence relation between the coordinate points, the perspective matrix of the image is calculated, the two-dimensional image is transformed according to the perspective matrix, correction of the two-dimensional image is achieved, accurate data can be measured, and high-precision repeatability of workpiece measurement is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of industrial visual inspection and measurement, and in particular to a correction method and device based on structured light photographing images. Background Art

[0002] Image correction refers to the restoration of distorted images. There are many reasons for image distortion, such as the imaging system itself, such as the inherent characteristics of the optical lens, which causes barrel distortion and pincushion distortion. Or the image geometric distortion caused by the shooting posture and scanning nonlinearity of the imaging device. These distortions will not only make the captured image unsightly, but also affect the deviation of computer vision measurement. Traditional image distortion processing methods mostly use the method of finding the boundary of the rectangular contour of the image, and then calculate the deformation parameters based on the deformation angle of the rectangular contour.

[0003] In the field of industrial vision, the target workpiece may be warped within a certain tolerance range, such as 0.02 mm. This makes it impossible to ensure that the object being photographed is absolutely parallel to the camera, and there will be a certain degree of pitch angle. In this case, the captured image is distorted, and the distortion is greater at the edge of the object being photographed. This distortion will cause errors in the position measurement based on two-dimensional images. Commonly used methods for correcting distortion, such as correcting a photographed paper document into a rectangle, usually use the outline of the positioning document, use the position relationship of the four vertices, calculate the distortion coefficient, and correct the photographed image. However, in the industrial field, the workpieces being photographed are often irregular objects, and it is impossible to find the rectangular outline boundary, and the measurement accuracy requirements are very high. If only a two-dimensional image of the workpiece is collected, it is impossible to correct the distortion of the warped workpiece. However, in the scheme using structured light imaging, the point cloud data and two-dimensional image of the object are collected at the same time. The angle between the photographed object and the horizontal coordinate system can be calculated through the point cloud, and the angle between the actual photographed object and the horizontal plane can be obtained, so as to fit the distortion model of the two-dimensional image of the photographed object and correct the two-dimensional image. In the field of industrial visual inspection and measurement, image acquisition devices with relatively high precision are often selected, and the distortion caused by the imaging system itself can be ignored. However, the measured product itself will have warping in an uncertain direction within the tolerance range, resulting in the product being unable to be placed parallel to the lens, and there is a certain pitch angle, causing a small distortion of the captured image. In the position measurement scheme, especially under high-precision measurement requirements, two-dimensional images are generally used for measurement because they have higher accuracy in horizontal direction measurement than three-dimensional images. Therefore, when the workpiece itself cannot be kept parallel to the imaging device, the collected two-dimensional image needs to be distorted. Summary of the invention

[0004] The purpose of the present invention is to provide a correction method and device based on structured light photographed images in order to solve the above problems.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] The present invention provides a correction method based on structured light photographing images, comprising the following steps:

[0007] Step S1, collecting a two-dimensional image and corresponding point cloud data through a structured light collection device;

[0008] Step S2, preprocessing the point cloud data, including denoising, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image;

[0009] As a preferred embodiment of the present invention, the denoising process in step S2 to filter out noise and outliers specifically includes: for each point in the point cloud data, calculating the average value and standard deviation of the points within a set radius around it, if the distance from the point to the average value is greater than a set multiple of the standard deviation, then the point is determined to be a noise point and removed;

[0010] As a preferred embodiment of the present invention, the selection of the set radius and the setting multiple parameters need to be specifically adjusted according to the shape and size of the photographed product;

[0011] As a preferred embodiment of the present invention, the coordinate system origin is unified with the two-dimensional image in step S2 specifically including: adjusting the origin of the point cloud coordinate system to the upper left corner, that is, the x coordinate value of each coordinate point of the point cloud is subtracted from the x coordinate value of the first point in the upper left corner, and the y coordinate value is subtracted from the y coordinate value of the first point in the upper left corner, and the z coordinate value remains unchanged. Through this operation, the coordinate system X and Y of the two-dimensional image are ensured to be consistent;

[0012] Step S3, select all or part of the filtered point cloud data, and fit the plane equation of the point cloud by the RANSAC method and the least squares method, specifically including: the RANSAC method randomly selects three points in the point cloud data, calculates the plane model determined by the three points, and then calculates the distances of all other points to the plane, and judges whether these points conform to the plane model according to a preset threshold, repeats this process multiple times, selects the plane model with the most conforming points as the initial plane equation, and then iteratively optimizes the initial plane by the least squares method to obtain the optimal plane equation Ax+By+Cz+D=0;

[0013] Step S4, calculating the angles α and β between the fitting plane and the X and Y axes respectively according to the four parameters A, B, C and D obtained by fitting;

[0014] As the preferred embodiment of the present invention, Figure 3As shown, the calculation of the angles α and β in step S4 is obtained by the following formula: According to the plane equation, the plane normal vector is (A, B, C), and the calculation formulas of the angles α and β are:

[0015]

[0016] Step S5, selecting four points on the edge of the two-dimensional image and calculating their coordinate values ​​corresponding to the coordinate system parallel to the lens;

[0017] As a preferred embodiment of the present invention, the four points in step S5 refer to the abscissa being 0 or the maximum width value of the image, and the ordinate being 0 or the maximum height value of the image;

[0018] As a preferred embodiment of the present invention, the four points in step S5 are not collinear;

[0019] As a preferred embodiment of the present invention, the four points in step S5 may be (0, y1), (0, y2), (x1, 0), and (x2, 0), and the corresponding coordinates of the four points may be calculated using the following formula:

[0020] x′=x / cosα; y′=y / cosβ;

[0021] The corresponding four sets of coordinates are (0, y1'), (0, y2'), (x1', 0), and (x2', 0).

[0022] Step S6, calculating the perspective transformation matrix according to the four sets of corresponding coordinates, and correcting the two-dimensional image according to the transformation matrix.

[0023] A correction device based on structured light image capture, comprising:

[0024] Data acquisition module, which collects point cloud data and two-dimensional images of products through structured light devices;

[0025] Data preprocessing module, including denoising of point cloud data, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image;

[0026] The plane fitting module fits the plane equation of the point cloud Ax+By+Cz+D=0 through the RANSAC method and the least squares method;

[0027] An angle calculation module calculates the angles α and β between the fitting plane and the X and Y axes according to the fitted plane equation;

[0028] The coordinate conversion module selects four points on the edge of the two-dimensional image and calculates their coordinate values ​​corresponding to the coordinate system parallel to the lens;

[0029] The image conversion module calculates the perspective transformation matrix based on the four sets of corresponding coordinates and corrects the two-dimensional image according to the transformation matrix.

[0030] The beneficial effects of the present invention are as follows: the image correction device of the present invention collects images through the data acquisition module to obtain a pair of two-dimensional and three-dimensional images, and then successively operates through the data preprocessing module, the plane fitting module, the angle calculation module, and the coordinate conversion module, and calculates the distortion coefficient of the two-dimensional image by the three-dimensional point cloud, and then corrects the two-dimensional image through the image conversion module. Through such an operation, it is possible to achieve a higher measurement accuracy than that through the three-dimensional image by calculating the position through the two-dimensional image, and it is possible to correct the two-dimensional image through the three-dimensional image to obtain a higher-precision position measurement index. The problem that the distortion of the two-dimensional image caused by the pitch angle of the uncertain direction of the photographed object cannot be corrected is solved. Thereby providing a good input for high-precision x and y direction position measurement, reducing the error caused by shooting distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0032] Figure 2 It is a device block diagram of the present invention.

[0033] Figure 3 Schematic diagram of coordinate transformation of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer, the implementation method of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0035] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0036] Embodiment 1

[0037] The present invention provides a method for image correction, as shown in the attached Figure 1 As shown, the following steps are included:

[0038] Step S1, collecting a two-dimensional image and corresponding point cloud data through a structured light collection device;

[0039] Step S2, preprocessing the point cloud data, including denoising, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image;

[0040] As a preferred embodiment of the present invention, the denoising process in step S2 to filter out noise and outliers specifically includes: for each point in the point cloud data, calculating the average value and standard deviation of the points within a set radius around it, if the distance from the point to the average value is greater than a set multiple of the standard deviation, then the point is determined to be a noise point and removed;

[0041] As a preferred embodiment of the present invention, the selection of the set radius and the setting multiple parameters need to be specifically adjusted according to the shape and size of the photographed product;

[0042] As a preferred embodiment of the present invention, the coordinate system origin is unified with the two-dimensional image in step S2 specifically including: adjusting the origin of the point cloud coordinate system to the upper left corner, that is, the x coordinate value of each coordinate point of the point cloud is subtracted from the x coordinate value of the first point in the upper left corner, and the y coordinate value is subtracted from the y coordinate value of the first point in the upper left corner, and the z coordinate value remains unchanged. Through this operation, the coordinate system X and Y of the two-dimensional image are ensured to be consistent;

[0043] Step S3, select all or part of the filtered point cloud data, and fit the plane equation of the point cloud by the RANSAC method and the least squares method, specifically including: the RANSAC method randomly selects three points in the point cloud data, calculates the plane model determined by the three points, and then calculates the distances of all other points to the plane, and judges whether these points conform to the plane model according to a preset threshold, repeats this process multiple times, selects the plane model with the most conforming points as the initial plane equation, and then iteratively optimizes the initial plane by the least squares method to obtain the optimal plane equation Ax+By+Cz+D=0;

[0044] Step S4, calculating the angles α and β between the fitting plane and the X and Y axes respectively according to the four parameters A, B, C and D obtained by fitting;

[0045] As a preferred embodiment of the present invention, the calculation of the angles α and β in step S4 is obtained by the following formula: According to the plane equation, the plane normal vector is (A, B, C), and the calculation formulas of the angles α and β are:

[0046]

[0047] Step S5, selecting four points on the edge of the two-dimensional image and calculating their coordinate values ​​corresponding to the coordinate system parallel to the lens;

[0048] As a preferred embodiment of the present invention, the four points in step S5 refer to the abscissa being 0 or the maximum width value of the image, and the ordinate being 0 or the maximum height value of the image;

[0049] As a preferred embodiment of the present invention, the four points in step S5 are not collinear;

[0050] As a preferred embodiment of the present invention, the four points in step S5 may be (0, y1), (0, y2), (x1, 0), and (x2, 0), and the corresponding coordinates of the four points may be calculated using the following formula:

[0051] x′=x / cosα; y′=y / cosβ;

[0052] The corresponding four sets of coordinates are (0, y1'), (0, y2'), (x1', 0), and (x2', 0).

[0053] Step S6, calculating the perspective transformation matrix according to the four sets of corresponding coordinates, and correcting the two-dimensional image according to the transformation matrix.

[0054] Embodiment 2

[0055] A correction device based on structured light image capture, such as Figure 2 As shown, including:

[0056] Data acquisition module, which collects point cloud data and two-dimensional images of products through structured light devices;

[0057] Data preprocessing module, including denoising of point cloud data, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image;

[0058] The plane fitting module fits the plane equation of the point cloud Ax+By+Cz+D=0 through the RANSAC method and the least squares method;

[0059] An angle calculation module calculates the angles α and β between the fitting plane and the X and Y axes according to the fitted plane equation;

[0060] The coordinate conversion module selects four points on the edge of the two-dimensional image and calculates their coordinate values ​​corresponding to the coordinate system parallel to the lens;

[0061] The image conversion module calculates the perspective transformation matrix based on the four sets of corresponding coordinates and corrects the two-dimensional image according to the transformation matrix.

[0062] Beneficial effects of the embodiments of the present invention: The image correction device of the present invention collects images through the data acquisition module to obtain a pair of two-dimensional and three-dimensional images, and then successively operates through the data preprocessing module, the plane fitting module, the angle calculation module, and the coordinate conversion module, and the distortion coefficient of the two-dimensional image is calculated by the three-dimensional point cloud, and then the two-dimensional image is corrected by the image conversion module. Through such an operation, it is possible to achieve a higher measurement accuracy than that through the three-dimensional image by calculating the position through the two-dimensional image, and it is possible to correct the two-dimensional image through the three-dimensional image to obtain a higher-precision position measurement index. The problem that the distortion of the two-dimensional image caused by the pitch angle of the uncertain direction of the photographed object cannot be corrected is solved. Thereby providing a good input for high-precision x and y direction position measurement, reducing the error caused by shooting distortion.

[0063] Although the present invention is described herein with reference to the illustrative embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A correction method based on structured light image capture, characterized in that: The steps include: Step S1, collecting a two-dimensional image and corresponding point cloud data through a structured light collection device; Step S2, preprocessing the point cloud data, including denoising, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image; Step S3, selecting all or part of the filtered point cloud data, and fitting the plane equation of the point cloud by the RANSAC method and the least squares method; Step S4, calculating the angles α and β between the fitting plane and the X and Y axes respectively according to the four parameters A, B, C and D obtained by fitting; Step S5, selecting four points on the edge of the two-dimensional image and calculating their coordinate values ​​corresponding to the coordinate system parallel to the lens; Step S6, calculating the perspective transformation matrix according to the four sets of corresponding coordinates, and correcting the two-dimensional image according to the transformation matrix.

2. The correction method based on structured light image capture according to claim 1, characterized in that: The denoising process in step S2 to filter out noise and outliers specifically includes: for each point in the point cloud data, calculating the average value and standard deviation of the points within a set radius around it; if the distance from the point to the average value is greater than a set multiple of the standard deviation, the point is determined to be a noise point and is removed.

3. The correction method based on structured light image capture according to claim 2, characterized in that: The selection of the set radius and the setting of the multiple parameters need to be specifically adjusted according to the shape and size of the product being photographed.

4. The correction method based on structured light image capture according to claim 3, characterized in that: The unification of the origin of the coordinate system with the two-dimensional image in step S2 specifically includes: adjusting the origin of the point cloud coordinate system to the upper left corner, that is, subtracting the x coordinate value of each coordinate point of the point cloud from the x coordinate value of the first point in the upper left corner, and subtracting the y coordinate value of the first point in the upper left corner from the y coordinate value, while keeping the z coordinate value unchanged. Through this operation, the coordinate system X, Y of the two-dimensional image is guaranteed to be consistent.

5. The correction method based on structured light image capture according to claim 4, characterized in that: The plane equation of the point cloud fitted in step S3 specifically includes: the RANSAC method randomly selects three points in the point cloud data, calculates the plane model determined by the three points, then calculates the distances from all other points to the plane, and determines whether these points conform to the plane model based on a preset threshold. This process is repeated many times, and the plane model with the most conforming points is selected as the initial plane equation. Then, the fitted initial plane is iteratively optimized by the least squares method to obtain the optimal plane equation Ax+By+Cz+D=0.

6. The correction method based on structured light image capture according to claim 5, characterized in that: The calculation of the angles α and β in step S4 is obtained by the following formula: According to the plane equation, the plane normal vector is (A, B, C), and the calculation formulas of the angles α and β are:

7. The correction method based on structured light image capture according to claim 6, characterized in that: The four points in step S5 refer to the points whose horizontal coordinate is 0 or the maximum width value of the image, and whose vertical coordinate is 0 or the maximum height value of the image.

8. The correction method based on structured light image capture according to claim 7, characterized in that: The four points in step S5 are not collinear.

9. The correction method based on structured light image capture according to claim 8, characterized in that: The four points selected in step S5 are (0, y1), (0, y2), (x1, 0), and (x2, 0). The corresponding coordinates of the four points can be calculated using the following formula: x′ ＝ x′=x / cosα;y′=y / cosβ; The corresponding four sets of coordinates are (0, y1'), (0, y2'), (x1', 0), and (x2', 0).

10. A correction device based on structured light image capture, used to implement the correction method according to any one of claims 1 to 9, comprising: Data acquisition module, which collects point cloud data and two-dimensional images of products through structured light devices; Data preprocessing module, including denoising of point cloud data, filtering out noise and outliers, and unifying the origin of the coordinate system with the two-dimensional image; The plane fitting module fits the plane equation of the point cloud Ax+By+Cz+D=0 through the RANSAC method and the least squares method; An angle calculation module calculates the angles α and β between the fitting plane and the X and Y axes according to the fitted plane equation; The coordinate conversion module selects four points on the edge of the two-dimensional image and calculates their coordinate values ​​corresponding to the coordinate system parallel to the lens; The image conversion module calculates the perspective transformation matrix based on the four sets of corresponding coordinates and corrects the two-dimensional image according to the transformation matrix.

Images