A method for image distortion correction
By calculating the center coordinates of the dot matrix feature of the calibrated picture and establishing a mapping table, the image distortion problem in industrial automation is solved, and high-precision image correction is achieved. It is suitable for the automatic measurement field of industrial automation, improving the accuracy of measurement and efficiency.
Patent Information
- Application Number
- CN202011216165.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-04
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-11-04
AI Technical Summary
In industrial automation, image distortion problems caused by manufacturing errors in machine vision systems affect the accuracy and accuracy of measurements. Especially in the field of high precision, a fast and high-precision image distortion correction method is needed to improve the reliability of measurement results. .
By calculating the subpixel values of the center coordinates of the dot matrix feature of the calibrated image, an initial image distortion correction mapping table and a grid mapping table are established, and these mapping tables are used to map the pictures taken by the camera and the lens under the same conditions twice. and linear interpolation calculation to obtain high dimensional accuracy dedistortion pictures.
It realizes image distortion correction with high calibration speed and high dimensional accuracy, has good anti-environmental interference capabilities, and is suitable for the field of automatic measurement in industrial automation.
Smart Images

Figure CN112348756B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine vision and automation, and in particular to the field of image distortion correction. Background Art
[0002] With the development of human society, economy and industry, the wave of industrial automation has swept in, and automatic control, automatic adjustment devices and automatic measuring equipment are widely used in industrial manufacturing and production to replace manual operation of machines and machine systems. It can not only improve the safety of the production process and improve production efficiency, but also improve product quality and reduce the loss of raw materials and energy in the production process.
[0003] At the same time, machine vision technology plays a vital role in the development of industrial automation. If automation equipment or machines are compared to human bodies and hands and feet, and software programs are compared to human brains, then there is no doubt that machine vision is equal to human eyes. Nowadays, machine vision has been widely used in a series of processes such as fitting and alignment, dimension measurement, appearance defect detection, 3D modeling, and mechanical grasping in industrial manufacturing and production. On the one hand, it can greatly improve work efficiency. For example, manual measurement of workpieces with multiple sizes may take several minutes, but it can be completed in less than one second using a machine vision measurement system, and the qualification can be judged immediately after the measurement is completed. On the other hand, it can reduce labor costs.
[0004] However, due to manufacturing errors during the manufacturing process of most cameras and lenses, there is a big difference between the pictures taken and the actual objects, which we call distortion. It seriously affects the accuracy and precision of machine vision, especially in the field of high-precision measurement. An error of one pixel is likely to cause measurement errors, resulting in a lot of waste or other problems. Therefore, a method that can quickly and accurately calibrate image distortion correction is needed to fill the flaws of machine vision in industrial automation. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an image distortion correction method, which has the characteristics of high calibration speed and high dimensional accuracy, and also has good resistance to environmental interference in the camera working environment, and is very suitable for the automatic measurement field of industrial automation.
[0006] The method for image distortion correction of the present invention is implemented by the following technical scheme: Step S1: calculating the sub-pixel value of the center coordinates of the characteristic circle of the circular dot matrix of the calibration image; calculating the sub-pixel value of the center coordinates of the characteristic circle in the circular dot matrix of the calibration image by performing contour analysis and block analysis on the calibration image;
[0007] Step S2: Calculate the image initial distortion correction mapping table; use the mapping relationship between the sub-pixel value of the center coordinates and the center coordinate position of the actual calibration plate to calculate the pixel position correspondence mapping table between the original calibration image and the real image;
[0008] Step S3: Calculate the image grid mapping table; use the distortion mapping table to calculate the calibration image for the initial dedistortion, and then use the calibration image for the initial dedistortion to calculate the grid mapping table;
[0009] Step S4: Calculate the dedistorted image; use the image distortion correction mapping table and the image grid mapping table to map other images taken by the camera and lens under the same conditions twice in sequence and perform linear interpolation calculations to obtain a high-size precision dedistorted image.
[0010] 2. The method for image distortion correction according to claim 1, characterized in that:
[0011] In step S1, the sub-pixel value of the center coordinate of the characteristic circle in the circle dot matrix of the calibration image is calculated by performing contour analysis and block analysis on the calibration image, including:
[0012] Step S1.1: Statistically analyze the pixel values of the image to calculate a value as a threshold for image binarization, and use the threshold to binarize the image to obtain a binary image;
[0013] Step S1.2: Calculate all contours in the image, and then filter the contours using the number of contour pixels, contour roundness, and contour area in turn;
[0014] Step S1.3: Calculate the area of the remaining contours and sort them according to the contour area to calculate the approximate radius of the circle in the circle dot matrix, and filter the contours again by radius;
[0015] Step S1.4: Calculate the center points of the remaining contours, fit an optimal matrix, and exclude the contours corresponding to the points that deviate from the matrix by more than a certain threshold;
[0016] Step S1.5: Analyze the block of the original image to obtain the center point coordinates of the block, and find the positional relationship between the center point coordinates and the above outline. Any center point within the outline is the sub-pixel center coordinate of the characteristic circle in the required dot matrix.
[0017] As a further improvement of the above technical solution, in step S2, the position correspondence mapping table between the original image and the corrected image is calculated using the calculated sub-pixel value of the center coordinate and the center coordinate position of the actual calibration plate, including:
[0018] Step S2.1: using the actual distance between the two center points of the calibration plate and the positional relationship of each center point to assign a corresponding coordinate value to each center point;
[0019] Step S2.2: The intrinsic parameter matrix and distortion coefficient of the shooting camera can be calculated using the calculated sub-pixel center coordinate value and the actual calibration plate coordinate value;
[0020] Step S2.3: The primary distortion mapping matrix can be calculated using the camera intrinsic parameter matrix and the distortion coefficient matrix;
[0021] Step S2.4: Calculate the calibration image after the initial dedistortion using the initial distortion mapping matrix and the input calibration image.
[0022] As a further improvement of the above technical solution, in step S3, the dedistorted calibration image is calculated using the initial distortion mapping table, and then the dedistorted calibration image is used to calculate the dedistorted mapping table with higher size accuracy, including:
[0023] Step S3.1: Calculate the first dedistorted calibration image using the original calibration image and the distortion mapping table;
[0024] Step S3.2: Calculate the sub-pixel coordinates of the center of the dot matrix feature circle of the calibration image for the initial dedistortion by using contour analysis and block analysis;
[0025] Step S3.3: Calculate the sub-pixel position of each point in the dedistorted image between adjacent feature points of the new matrix using the principle of reverse mapping, and save it in the new mapping matrix;
[0026] Step S3.4: Reuse the position and distance relationship of adjacent feature points and the principle of linear interpolation to map all other points in the new matrix from the first dedistorted image to obtain a new mapping matrix;
[0027] As a further improvement of the above technical solution, in step S4, the primary distortion mapping table and the grid mapping table are sequentially mapped twice on other pictures taken by the camera and lens under the same conditions and linear interpolation is performed to obtain a high-size precision dedistorted picture, including:
[0028] Step S4.1: using the mapping relationship between the input image and the initial distortion mapping table, and then using a linear interpolation method to initially dedistort the image taken by the camera;
[0029] Step S4.2: Using the mapping relationship between the initial dedistorted image and the grid distortion grid mapping table, a linear interpolation method is used to calculate a dedistorted image with higher accuracy.
[0030] The beneficial effects of the present invention are as follows: the present invention provides an image distortion correction method, which has the characteristics of high calibration speed and high dimensional accuracy, and also has good resistance to environmental interference in the camera working environment, and is very suitable for the automatic measurement field of industrial automation. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 is a flow chart of the present invention;
[0033] Figure 2 The white ceramic dot array calibration plate of the present invention;
[0034] Figure 3 The main flow chart of the sub-pixel coordinates of the center of the circle of the present invention;
[0035] Figure 4 It is the center feature map of the calibrated image feature circle of the present invention;
[0036] Figure 5 The flowchart of the calculation position corresponding mapping table of the present invention;
[0037] Figure 6 A flow chart of a grid mapping table for calculating higher dimensional accuracy according to the present invention;
[0038] Figure 7 It is a regional processing diagram of the present invention;
[0039] Figure 8 This is a flow chart of high dimensional accuracy distortion removal according to the present invention. DETAILED DESCRIPTION
[0040] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.
[0041] Any feature disclosed in this specification (including any additional claims, abstract and drawings), unless otherwise stated, may be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
[0042] In the description of the present invention, it is necessary to understand that the terms "one end", "the other end", "outside", "upper", "inside", "horizontal", "coaxial", "center", "end", "length", "outer end" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0043] In addition, in the description of the present invention, “plurality” means at least two, for example, two, three, etc., unless otherwise clearly and specifically defined.
[0044] Terms such as "upper", "above", "lower", "below", etc. used in the present invention to indicate spatial relative positions are used for the purpose of convenience to describe the relationship of one unit or feature relative to another unit or feature as shown in the accompanying drawings. Terms of spatial relative position may be intended to include different orientations of the device in use or operation other than the orientation shown in the drawings. For example, if the device in the figure is turned over, the unit described as being "below" or "beneath" other units or features will be located "above" the other units or features. Therefore, the exemplary term "below" can encompass both the above and below orientations. The device may be oriented in other ways, and the spatially related descriptors used herein shall be interpreted accordingly.
[0045] In the present invention, unless otherwise clearly specified and limited, the terms "disposed", "socketed", "connected", "through", "plugged" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0046] Figure 1 FIG. 4 is a general flow chart of a camera distortion correction method according to an embodiment. Figure 1 As shown, in this embodiment, the process of image distortion correction includes the following steps:
[0047] Take a picture of the calibration plate;
[0048] The calibration board in this example is a white ceramic dot matrix calibration board such as Figure 2 The characteristic circle is black, the circle radius is 1.5mm, the distance between the two circle centers is 3mm, and the number of characteristic circle arrays is 41*41.
[0049] In step S1, the sub-pixel value of the coordinates of the center point of the characteristic circle in the circle dot matrix of the calibration image is calculated by performing contour analysis and block analysis on the calibration image;
[0050] The main purpose of this step is to quickly and accurately calculate the sub-pixel coordinates of the center of each feature circle. The main flow chart is as follows: Figure 3 shown.
[0051] First, the binarization threshold of the ROI area of the calibration image is calculated by using histogram statistics. The threshold calculated for the calibration image is 118.
[0052] The calibration image is binarized using the threshold, and a total of 1748 contours are calculated, of which the largest contour is the outer contour of the image.
[0053] The minimum pixel count threshold for constituting an outline is set to 10. On the one hand, it can filter out non-circular noise outlines. On the other hand, if the circular outline contains too few pixels, the calibration accuracy will be very low. After filtering, the number of contours calculated is 1734.
[0054] Calculate the contour area, set the maximum and minimum area thresholds of the contour to 50000 and 50 respectively, and further filter the contours. The outer border contour of the point calibration plate and some small spot contours can be filtered out, and the number of remaining contours is 1718.
[0055] The ratio of the contour area to the minimum circumscribed circle area of the contour was calculated, the threshold range was set to 0.7-1.3, and the contours of point, line and rod-shaped objects were filtered. The number of contours after filtering was 1684.
[0056] Sort the remaining contours in ascending order according to the size of the contour area. Using the principle that the areas of characteristic circles are equal, the approximate radius of the characteristic circle can be calculated to be 17.415.
[0057] The upper and lower thresholds of the contour radius were set to 1.3 times and 0.7 times the approximate radius respectively to filter out contours whose areas did not meet the requirements, and the number of remaining contours was 1681.
[0058] The image was analyzed for blocks, and a total of 1694 block center points were obtained.
[0059] Find the center point of the block within the 1681 contours obtained, which is the sub-pixel coordinate of the center of the required characteristic circle of the dot array.
[0060] The final center of the calibration image feature circle is as follows: Figure 4 As shown, green is the center position of the marked characteristic circle.
[0061] In step S2, the position correspondence mapping table between the original image and the first dedistorted image is calculated using the calculated circle center coordinates and the circle center coordinates of the actual calibration plate. The main process is as follows: Figure 5 As shown;
[0062] Calculate the center coordinates of all feature circles of the physical calibration plate. Assume that the center coordinates of the top left feature circle are (0, 0, 0). Since the feature circles are all in the same plane, their Z-axis coordinates are all 0. Then, calculate the center coordinates of all feature circles of the calibration plate based on the distance between the two centers of the calibration plate and the relative position relationship between the feature circles.
[0063] The calculated center coordinates of all characteristic circles of the calibration image and the center coordinates of all characteristic circles of the physical calibration plate calculated above can be used to calculate the camera intrinsic parameter matrix and distortion coefficient.
[0064] The initial distortion correction position mapping table is calculated using the camera intrinsic parameter matrix and the distortion coefficient. The size of the mapping table is the same as the size of the calibration image, and a dual-channel matrix is used to store the x and y coordinates in the same mapping table.
[0065] In step S3, the initial distortion mapping table is used to calculate the initial dedistorted calibration image, and then the initial dedistorted calibration image is used to calculate a grid mapping table with higher size accuracy. The main process is as follows: Figure 6 As shown;
[0066] The obtained mapping table and the calibration image are used to calculate a new image using a bilinear interpolation method. This image is the calibration image for the initial dedistortion.
[0067] The sub-pixel coordinate value of the center of the characteristic circle of the calibration image is obtained by referring to the method for obtaining the sub-pixel coordinate value of the center of the characteristic circle of the calibration image in step S1.
[0068] Through statistical calculation, the average pixel distance between two feature points is rounded to 62 pixels, and a matrix is created. The matrix size is the calibration image size, the number of channels is 2, the matrix data type is single-precision floating-point data, and all positions in the matrix are assigned a value of -1.
[0069] Use 41*41 feature points to divide the image into 40*40 grids, such as Figure 7 As shown in the figure, the entire calibration image is divided into regions for processing, the upper left corner point A (868.158, 242.214) of the rectangle is used as the reference point, and it is mapped to the position of matrix A1 (868, 242) according to the principle of rounding, and B is mapped to B1 (930, 242). Then all feature points are mapped to the corresponding positions of the matrix at a distance of 62 pixels.
[0070] First, process the first grid in the upper left corner. According to the coordinates of points A and B, the pixel length rx in the x direction between AB is calculated to be 62.603 and the pixel length ry in the y direction is calculated to be -0.475. The x-direction proportional coefficient kx is calculated to be 1.009730 and the y-direction proportional coefficient ky is calculated to be -0.007663. The x-coordinate of the nth point between A1 and B1 is 868+n*kx, and the y-coordinate is 242+n*ky.
[0071] Map point C to the position of matrix C1 (868, 304), and use the same method as above to find the positions of the other 61 points between A1 and C1 in the original dedistorted image. Then, take each point between A1 and C1 as the starting point and calculate all the points in the first grid in the same way as the points between A1 and B1.
[0072] By analogy, the positions of all other points in the grid relative to the original distorted image can be calculated, and the image mapped by the matrix mapping table has higher dimensional accuracy.
[0073] In step 4, the primary distortion mapping table and the grid mapping table are used to map other images taken by the camera and lens under the same conditions twice in sequence and perform linear interpolation calculations to obtain a high-precision dedistorted image. The main process is as follows: Figure 8 As shown, including:
[0074] The initial dedistorted image is obtained by using the input original image and the initial distortion correction mapping table.
[0075] The input first dedistorted image and grid mapping table are used to obtain a higher precision dedistorted image.
[0076] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that are not conceived through creative work should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope defined in the claims.
Claims
1. A method for image distortion correction, characterized in that: The correction method comprises the following steps: Step S1: Calculate the sub-pixel value of the center coordinates of the characteristic circle of the circle dot matrix of the calibration image; calculate the sub-pixel value of the center coordinates of the characteristic circle in the circle dot matrix of the calibration image by performing contour analysis and block analysis on the calibration image; Step S2: Calculate the image initial distortion correction mapping table; use the mapping relationship between the sub-pixel value of the center coordinates and the center coordinate position of the actual calibration plate to calculate the pixel position correspondence mapping table between the original calibration image and the real image; Step S3: Calculate the image grid mapping table; use the image primary distortion correction mapping table to calculate the initial dedistorted calibration image, and then use the initial dedistorted calibration image to calculate the image grid mapping table; Step S4: Calculate the dedistorted image; use the image primary distortion correction mapping table and the image grid mapping table to map other images taken by the camera and lens under the same conditions twice in sequence and perform linear interpolation calculation to obtain a high-size precision dedistorted image; In step S1, the sub-pixel value of the center coordinate of the characteristic circle in the circle dot matrix of the calibration image is calculated by performing contour analysis and block analysis on the calibration image, including: Step S1.1: Statistically analyze the pixel values of the image to calculate a value as a threshold for image binarization, and use the threshold to binarize the image to obtain a binary image; Step S1.2: Calculate all contours in the image, and then filter the contours using the number of contour pixels, contour roundness, and contour area in turn; Step S1.3: Calculate the area of the remaining contours and sort them according to the contour area to calculate the approximate radius of the circle in the circle dot matrix, and filter the contours again by the radius; Step S1.4: Calculate the center points of the remaining contours, fit an optimal matrix, and exclude the contours corresponding to the points that deviate from the matrix by more than a certain threshold; Step S1.5: Analyze the block of the original image to obtain the center coordinates of the block, and find the positional relationship between the center coordinates and the above outline. Any center point within the outline is the sub-pixel center coordinate of the characteristic circle in the required dot matrix; In step S2, the position correspondence mapping table between the original calibration image and the real image is calculated using the calculated sub-pixel value of the center coordinate and the center coordinate position of the actual calibration plate, including: Step S2.1: Assign a corresponding coordinate value to each center of the circle using the actual distance between the two center points of the calibration plate and the positional relationship of each center of the circle; Step S2.2: The intrinsic parameter matrix and distortion coefficient of the shooting camera can be calculated using the calculated sub-pixel center coordinate value and the actual calibration plate coordinate value; Step S2.3: The initial distortion correction mapping table can be calculated using the camera intrinsic parameter matrix and the distortion coefficient matrix; Step S2.4: Calculate the calibration image after the initial dedistortion using the initial distortion mapping matrix and the input calibration image; In step S3, the specific steps of the picture grid mapping table include: S3.1 Grid division: Based on the identified feature points, the image is divided into several grids, each grid representing a part of the dedistorted image; S3.2 Matrix mapping: Map feature points into the matrix according to their coordinates, and round their pixel coordinates to the nearest integer value. The process starts from the reference point of the first grid in the upper left corner, and gradually completes the matrix mapping of the entire image by calculating and mapping all points in each grid. S3.3 Calculation of midpoints: Within each grid, the midpoint positions between the mapped points are determined by calculating the pixel distances in the x and y directions and using the scale factor; S3.4 High-precision image mapping: Finally, the entire image is mapped into a matrix, and each grid point corresponds to the exact position in the original dedistorted image, thereby improving the size accuracy of the mapped image; In step S4, the image primary distortion correction mapping table and the picture grid mapping table are used to map other pictures taken by the camera and lens under the same conditions twice in sequence and perform linear interpolation calculations to obtain high-size precision dedistorted pictures, including: Step S4.1: using the mapping relationship between the input image and the image primary distortion correction mapping table, and then using the linear interpolation method to perform primary distortion removal on the image taken by the camera; Step S4.2: Using the mapping relationship between the dedistorted calibration image and the image grid mapping table, a linear interpolation method is used to calculate a dedistorted image with higher accuracy.
Citation Information
Patent Citations
Camera distortion correcting method based on square target model and perspective projection
CN103729841A
Fish-eye image correction method based on multistep correction
CN103996172A
Overlook conversion method, overlook image acquisition method and mapping table construction method of fisheye image
CN106408511A
High-precision camera calibration method based on mixed distortion model
CN106780628A
Image correction method of binocular camera and storage medium
CN109961401A
