A large-format image correction system based on a plane mesh method

Abstract

This invention discloses a large-format image correction system based on a planar mesh method, specifically relating to the field of machine vision. It includes an image data receiving module, an image preprocessing module, a mesh line processing module, an image correction module, and a precision calculation module. The image correction module performs an affine transformation between the extracted distorted triangle and a preset triangle to determine the positional information of the distorted triangle and the preset triangle, enabling rapid and accurate correction of distorted images and improving the efficiency and accuracy of image correction. Based on planar mesh correction, the mesh side length can be adjusted to control the correction accuracy and the time required to generate correction data. It eliminates the need for calibration and calculation of camera intrinsic and extrinsic parameters, directly correcting distortion in planar images. Through the combination and collaboration of these modules, image distortion is corrected, mesh information is accurately extracted, and high-efficiency planar distortion image correction is achieved.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more specifically, to a large-format image correction system based on a planar mesh method. Background Technology

[0002] Most existing image distortion correction methods require taking multiple images of the calibration plate from different angles and positions.

[0003] In the process of planar image correction, it is necessary to determine the relationship between the three-dimensional geometric position of a point on the plane and its corresponding point in the image. This requires knowing the pose parameters of the camera. In most calibration algorithms, the intrinsic parameters, extrinsic parameters, and distortion parameters of the camera must be obtained through experiments and calculations. This process is called camera calibration. Commonly used camera calibration methods include traditional camera calibration methods, active vision camera calibration methods, and camera self-calibration methods.

[0004] However, the above technology still has shortcomings. In the process of using the above technology, it is necessary to take multiple images of the calibration board from different angles and positions. If multiple large-format, high-precision calibration boards are printed, the efficiency is low. In addition, when multiple cameras need to take pictures at the same time, it is not easy to find a suitable calibration board angle and position for all cameras, which can easily lead to missed or incorrect pictures. The angle and position need to be carefully adjusted, otherwise algorithm errors will occur, and the corresponding costs will increase. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, this invention provides a large-format image correction system based on a planar grid method. By combining triangulation, affine transformation, and image interpolation algorithms based on a planar grid image, the system corrects distorted grid images captured by an industrial camera. Subsequently, the average distance between the standard grid points and the grid points of the correction image is calculated to determine the correction accuracy, thereby solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution, comprising: an image data receiving module, an image preprocessing module, a grid line extraction module, an image correction module, and a precision calculation module.

[0007] Image data receiving module: used to receive distorted planar grid images sent by the camera;

[0008] Image preprocessing module: used to perform preprocessing operations such as channel separation, noise reduction, segmentation and enhancement on distorted images, and transmit the processed image to the grid line extraction module;

[0009] Grid line processing module: used to extract grid lines from a grid image and calculate the intersection points of horizontal and vertical lines;

[0010] Image correction module: used to correct grid images and transmit the corrected image data to the database;

[0011] Precision calculation module: used for precision calculation of the image after distortion correction;

[0012] Preferably, the image data receiving module is used to receive distorted planar grid images sent by the camera, and the input module specifically includes:

[0013] Camera unit: Acquires distorted planar grid images through a camera and other image acquisition devices;

[0014] Image acquisition unit: The system connects to the camera, receives distorted planar grid images, stores the images, and transmits them to the image preprocessing module.

[0015] Preferably, the image preprocessing module is used to perform preprocessing operations such as channel separation, noise reduction, segmentation, and enhancement on the corrected image. Specifically, the image preprocessing module includes:

[0016] Uniform size unit: The grid area is scaled and cropped using scaling, cropping and filling techniques. Scaling can shorten the subsequent image processing time, and cropping can prevent the image from being affected by too much noise.

[0017] Denoising Unit: This unit uses a dynamic thresholding algorithm to denoise the input image, removing noise interference. First, it selects regions from the input image whose pixels meet the threshold condition. Let G... O For the input image, G T For a thresholded image, the formula for calculating the image grayscale value is G. O ≤G T -Offset, where Offset is the image grayscale offset value, is applied to G T In general, the threshold image is obtained by filtering the input image. For example, the threshold image G can be obtained through mean filtering, Gaussian filtering, binomial filtering, and other algorithms. T ;

[0018] Preferably, the grid line processing module is used to extract the grid lines of the grid image and calculate the intersection points of the horizontal and vertical lines. The grid line processing module specifically includes:

[0019] Segmentation Unit: Based on the characteristics of the image and the structure of the grid, connected component analysis is used to segment the image into grid regions and non-grid regions;

[0020] Morphological processing unit: Uses morphological operations to further process the mesh area, removing unnecessary details or filling in missing parts;

[0021] Intersection Extraction Unit: Calculates the intersection points of row and column grid lines to obtain the coarse coordinates of the marker points. Since both row and column grid lines are pixel-level curves, and each curve consists of several pixels, the coordinates of a point are taken as the average of the pixel coordinates of the pixels enclosed by it. The calculation formula is as follows: Where P Xi and P Yi Let x and y be the x and y coordinates of the i-th marker point, where x j and y j Let x and y be the x and y coordinates of the j-th pixel enclosed by the marker, and n be the number of pixels enclosed by the marker. This allows us to calculate the accurate coordinates of each marker. The extracted intersection points are shown below. Figure 5 As shown.

[0022] Intersection Sorting Unit: Because the subsequent image correction module requires the intersection points to be sorted from left to right and from top to bottom, a sorting algorithm is used to sort the intersection points. First, the intersection points are sorted. If the first intersection point P... Yi >P Y(i+1) Then swap the two intersection points, and then sort the intersection points by column. If intersection point P Xi >P X(i+1) If the absolute value of the row coordinates of the two intersection points (i and (i+1) is less than devi, then the two intersection points are swapped, where devi is the allowed offset value of the two intersection points in the row direction.

[0023] Preferably, the image correction module is used to correct the grid image and transmit the corrected image data to the database. The image correction module specifically includes:

[0024] Coordinate transformation unit: Using camera parameters, object distance, and image size, the camera resolution (DPI) can be calculated, and then the following formula can be used. The physical coordinate system Grid can be calculated. L How many pixels does a millimeter-long grid occupy in a pixel coordinate system?

[0025] Distortion Triangulation Unit: Since the intersection points are extracted from the distorted grid image and have been sorted in the intersection point sorting unit, every four adjacent intersection points can form a rectangle. Using the three lower left and three upper right intersection points of these four adjacent intersection points, two triangles can be formed. That is, for an intersection point image with P_Columns columns and P_Rows rows, the i-th intersection point can form a lower left triangle with the i+P_Columns and i+P_Columns+1 intersection points, and the i-th intersection point can form an upper right triangle with the i+1 and i+P_Columns+1 intersection points. There are a total of (P_Columns-1)*(P_Rows-1)*2 triangles. The distorted image of each triangle is cropped and transmitted to the affine transformation unit.

[0026] Affine transformation unit: An affine transformation can be represented in the following matrix multiplication form,

[0027] Here, (x, y) are the coordinates of a point in the original image, and (x', y') are the coordinates of a point after the affine transformation. The elements (a, b, c, d, e, f) in the matrix control the scaling, rotation, and translation of the affine transformation. The distorted image of each triangle is corrected using the coordinate transformation unit and its data, and the corrected data is saved.

[0028] Image interpolation unit: Images undergoing affine transformation often exhibit irregular pixel distribution, pixel compression, and uneven density. Therefore, grayscale interpolation is needed to generate regular raster images from these irregular images. Using bilinear interpolation ensures that the computation speed is not too slow while maintaining good image quality. Bilinear interpolation is based on linear interpolation in two directions (horizontal and vertical), estimating the value of the interpolated pixel by weighted averaging of its four nearest neighbors.

[0029] Image stitching unit: The pixel values ​​of two triangular images of the same rectangle are added together to obtain a single correction grid. Then, each correction grid is stitched together to complete the image distortion correction.

[0030] Preferably, the accuracy calculation module is used for accuracy calculation after image distortion correction, and the accuracy calculation module specifically includes:

[0031] Image segmentation unit: The OTSU algorithm is used to segment the foreground and background of the corrected grid image and extract the point set of the corrected grid image.

[0032] Accuracy calculation unit: Calculates the distance between the point set of the corrected image and a preset point set, and takes the average value. The formula is as follows: The average distance accuracy can then be obtained.

[0033] The technical effects and advantages of this invention are as follows: This invention utilizes an image correction module to perform an affine transformation between the extracted distorted triangle and a preset triangle, determining the positional information of the distorted triangle and the preset triangle. This allows for rapid and accurate correction of distorted images, improving the efficiency and accuracy of image correction. Based on planar grid correction, the grid side length can be adjusted to control the correction accuracy and the time required to generate correction data. It can skip the calibration of camera intrinsic and extrinsic parameters and directly perform distortion correction on planar images. Through the combination and collaboration of the above modules, this invention achieves high-efficiency planar distorted image correction, accurately extracts grid information, corrects image distortion, and meets the correction requirements of planar images. Attached Figure Description

[0034] Figure 1 This is a system flowchart of the present invention;

[0035] Figure 2 The denoised image obtained through step S3 of this invention;

[0036] Figure 3 The grid line diagram is obtained by extracting the grid lines in step S4 of this invention;

[0037] Figure 4 A marker point consisting of several pixels;

[0038] Figure 5 The image is a map of marker points extracted and sorted in step S4 of this invention;

[0039] Figure 6 A schematic diagram showing a single mesh divided into two triangles;

[0040] Figure 7 This is a single grid-corrected image obtained through step S5 of the present invention;

[0041] Figure 8 This is the complete corrected image obtained through step S6 of the present invention;

[0042] Figure 9 Image of the leather strap before correction in Example 2;

[0043] Figure 10 The image with leather after correction in Example 2; Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1

[0046] This embodiment provides, for example Figure 1 The present invention discloses a large-format image correction system based on the planar grid method, comprising: an image data receiving module, an image preprocessing module, a grid line extraction module, an image correction module, and an accuracy calculation module.

[0047] The image data receiving module receives the planar grid distortion image sent by the camera and transmits the image to the image preprocessing module. The image preprocessing module performs preprocessing operations such as channel separation, noise reduction, segmentation, and enhancement on the distorted image and transmits the processed image to the grid line extraction module. The grid line extraction module extracts the grid lines of the grid image and calculates the intersection points of the horizontal and vertical lines. It then transmits the extracted intersection point data to the image correction module. The image correction module corrects the grid image and transmits the corrected image data to the database and the accuracy calculation module. The accuracy calculation module calculates the accuracy of the image after distortion correction.

[0048] This embodiment provides a method flow for a large-format image correction system based on the planar grid method, specifically including the following steps:

[0049] 101. The distorted planar grid image is received through the image data receiving module. The specific steps of the image data receiving module are as follows:

[0050] A1. Camera: Acquires distorted planar grid images through cameras and other image acquisition devices;

[0051] A2. Image Acquisition: The system connects to the camera, receives distorted planar grid images, and stores and transmits the images to the image preprocessing module.

[0052] It should be noted here that:

[0053] The image data receiving module can receive images in various formats, has high compatibility, and can acquire images through devices such as scanners and cameras, making it convenient and practical.

[0054] 102. The image preprocessing module is used to perform preprocessing operations such as channel separation, noise reduction, segmentation, and enhancement on distorted images. The specific steps of the image preprocessing module in preprocessing the input image are as follows:

[0055] B1. Uniform Size: The grid area is scaled and cropped using scaling, cropping, and filling techniques. Scaling can shorten the subsequent image processing time, and cropping can prevent the image from being affected by too much noise.

[0056] B2. Image Channel Separation: Based on the received image channels, separate the image of each channel and select the most suitable channel image;

[0057] B3. Denoising Unit: This unit uses a dynamic thresholding algorithm to denoise the input image, removing noise interference. First, it selects regions from the input image whose pixels meet the threshold condition. Let G... O For the input image, G T For a thresholded image, the formula for calculating the image grayscale value is G. O ≤G T -Offset, where Offset is the image grayscale offset value, is applied to G T In general, the threshold image is obtained by filtering the input image. For example, the threshold image G can be obtained through mean filtering, Gaussian filtering, binomial filtering, and other algorithms. T Denoising images such as Figure 2 As shown;

[0058] 103. The grid line processing module is used to extract the grid lines of the grid image and calculate the intersection points of the horizontal and vertical lines. The specific steps of the grid line processing module for extracting image grid features are as follows:

[0059] C1. Image segmentation: Based on the characteristics of the image and the structure of the grid, connected component analysis is used to segment the image into grid regions and non-grid regions;

[0060] C2. Morphological Processing: Morphological operations are used to further process the mesh area, removing unnecessary details or filling in missing parts. The processed result should look like... Figure 3 As shown;

[0061] C3. Intersection Point Extraction: Calculate the intersection points of the row and column grid lines to obtain the coarse coordinates of the marker points. Since both row and column grid lines are pixel-level curves, each curve consists of several pixels, the intersection point of two curves also consists of several pixels. Figure 4 As shown, the coordinates of a point are taken as the average of the pixel coordinates of the pixels enclosed by it, and the calculation formula is: Where P Xi and P Yi Let x and y be the x and y coordinates of the i-th marker point, where x j and y j Let x and y be the x and y coordinates of the j-th pixel enclosed by the marker, and n be the number of pixels enclosed by the marker. This way, the accurate coordinates of each marker can be calculated.

[0062] C4. Intersection Point Sorting: Because the subsequent image correction module requires the intersection points to be sorted from left to right and from top to bottom, a sorting algorithm is used to sort the intersection points. First, the intersection points are sorted. If the first intersection point P... Yi >P Y(i+1) Then swap the two intersection points, and then sort the intersection points by column. If intersection point P Xi >P X(i+1) If the absolute value of the row coordinates of the two intersection points (i and (i+1) is less than devi, then the two intersection points are swapped, where devi is the allowed offset value of the two intersection points in the row direction.

[0063] 104. The image correction module is used to correct the grid image and transmit the corrected image data to the database and the accuracy calculation module. The specific steps of the image correction module are as follows:

[0064] D1. Coordinate Transformation: Using camera parameters, object distance, and image size, the camera resolution (DPI) can be calculated, and then the following formula can be used... The physical coordinate system Grid can be calculated. L How many pixels does a millimeter-long grid occupy in a pixel coordinate system?

[0065] D2. Distortion Triangulation: Since the intersection points are extracted from the distorted grid image and have already been sorted in the intersection point sorting unit, every four adjacent intersection points can form a rectangle. Using the three lower-left and three upper-right intersection points from these four adjacent points, two triangles can be formed. That is, for an intersection point image with P_Columns columns and P_Rows rows, the i-th intersection point can form a lower-left triangle with the (i+P_Columns)-th and (i+P_Columns+1)-th intersection points, and a upper-right triangle with the (i+1)-th and (i+P_Columns+1)-th intersection points. Therefore, there are a total of (P_Columns-1)*(P_Rows-1)*2 triangles. The distorted image of each triangle is extracted and transmitted to the affine transformation unit, as follows: Figure 6 The images shown are a distorted triangle image and a preset standard triangle image;

[0066] D3. Affine Transformation Unit: Affine transformations can be represented as the following matrix multiplication form.

[0067] Here, (x, y) are the coordinates of a point in the original image, and (x', y') are the coordinates of a point after the affine transformation. The elements (a, b, c, d, e, f) in the matrix control the scaling, rotation, and translation of the affine transformation. The distorted image of each triangle is corrected using the coordinate transformation unit and its data, and the corrected data is saved.

[0068] D4. Image Interpolation: Images undergoing affine transformation often exhibit irregular pixel distribution, pixel compression, and uneven density. Therefore, grayscale interpolation is needed to generate regular raster images. Using bilinear interpolation ensures a reasonable computation speed while maintaining good image quality. Bilinear interpolation is based on linear interpolation in two directions (horizontal and vertical), estimating the interpolated pixel value by weighted averaging of four neighboring pixels. An image after interpolation of a single grid is shown below. Figure 7 As shown.

[0069] D5. Image Stitching: Adding the pixel values ​​of two triangular images within the same rectangle yields a single corrected grid. Stitching these corrected grids together completes the image distortion correction. The corrected image looks like this: Figure 8 As shown.

[0070] 105. The precision calculation module is used to calculate the precision of the image after distortion correction. The specific steps of the precision calculation module are as follows:

[0071] E1. Image segmentation: The OTSU algorithm is used to segment the foreground and background of the corrected grid image and extract the point set of the corrected grid image.

[0072] E2, Accuracy Calculation Unit: Calculates the distance between the point set of the corrected image and a preset point set, and takes the average value. The formula is as follows: The average distance accuracy can then be obtained.

[0073] like Figure 1 This embodiment provides a large-format image correction system based on the planar grid method, including the following steps:

[0074] S1. Draw a standard grid;

[0075] S2, Receive the distorted planar grid image;

[0076] S3. Perform preprocessing operations such as channel separation, denoising, segmentation and enhancement on the distorted image;

[0077] S4. Extract the grid lines of the grid image and calculate the intersection points of the horizontal and vertical lines. Sort all intersection points from left to right and from top to bottom.

[0078] S5. Perform mapping correction on individual distorted grid images and preset standard grid images in sequence, and save the correction data;

[0079] S6. Stitch together each calibrated grid image to generate the calibration result and calculate the calibration accuracy.

[0080] Example 2

[0081] The algorithm presented in this paper has been applied to the calibration system of a smart printer for leather shoes and hats. First, a standard grid is drawn, and images are captured using a camera to obtain distorted grid images. The grid intersections of the distorted images are extracted and sorted. Every four adjacent intersections form a rectangle. Using the three intersections at the bottom left and the three at the top right of these four adjacent intersections, two triangles can be formed. The distorted image of each triangle is cropped, and an affine transformation is applied to the distorted triangle. After the affine transformation, the image will exhibit irregular pixel distribution, pixel compression, and uneven density. Therefore, grayscale interpolation is needed to generate a regular raster image. A bilinear interpolation algorithm is used to estimate the interpolated pixel value by weighted averaging of four neighboring pixels. The pixel values ​​of the two triangle images within the same rectangle are added together to obtain a single corrected grid. Then, each corrected grid is stitched together to complete the image distortion correction. Figure 9 This is an image of the front leather belt being corrected. Figure 10 To correct the image with leather.

[0082] As the grid size increases, the accuracy decreases. Conversely, smaller grid sizes result in higher accuracy but take longer to generate correction data.

Claims

1. A large-format image correction system based on a planar grid method, characterized in that: First, a standard grid is drawn. Images are captured using a camera to obtain a grid distortion image. The grid intersections of the distortion image are extracted and sorted. Every four adjacent intersections can form a rectangle. The three intersections at the bottom left and the three intersections at the top right of these four adjacent intersections can form two triangles. The distortion image of each triangle is cropped and subjected to an affine transformation. Gray-scale interpolation is performed on the affine transformed image to generate a regular raster image. The bilinear interpolation algorithm is used to estimate the value of the interpolated pixel by weighted averaging of the four neighboring pixels. The pixel values ​​of the two triangle images of the same rectangle are added together to obtain a single corrected grid. Then, each corrected grid is stitched together to complete the distortion correction of the entire image. The correction system includes the following modules: an image data receiving module for receiving distorted planar grid images sent by a camera and transmitting them to an image preprocessing module; an image preprocessing module for performing channel separation, noise reduction, segmentation, and enhancement preprocessing operations on the distorted image and transmitting the processed image to a grid line extraction module; a grid line processing module for extracting grid lines from the grid image, calculating the intersection points of horizontal and vertical lines, and transmitting the point set to an image correction module; an image correction module for correcting the grid image and transmitting the corrected image data to a database and an accuracy calculation module; and an accuracy calculation module for calculating the accuracy of the image after distortion correction.

2. The large-format image correction system based on the planar grid method according to claim 1, characterized in that: The image data receiving module is used to receive distorted planar grid images sent by the camera. The image data receiving module specifically includes: a camera unit: which acquires distorted planar grid images through a camera and other image acquisition devices; and an image acquisition unit: which connects to the camera, receives distorted planar grid images, stores the images, and transmits them to the image preprocessing module.

3. The large-format image correction system based on the planar grid method according to claim 1, characterized in that: The image preprocessing module is used to perform preprocessing operations such as channel separation, denoising, segmentation, and enhancement on the corrected image. Specifically, the image preprocessing module includes: a uniform size unit: using scaling, cropping, and filling techniques to scale and crop the grid region; and a denoising unit: using a dynamic thresholding algorithm to denoise the input image, removing noise interference. First, it selects regions from the input image whose pixels meet the threshold conditions. For the input image, For a threshold image, the formula for calculating the image grayscale value is: ,in The image grayscale offset value is applied to it. In this process, the threshold image is obtained by filtering the input image.

4. The large-format image correction system based on the planar grid method according to claim 1, characterized in that: The grid line processing module is used to extract the grid lines of the grid image and calculate the intersection points of the horizontal and vertical lines; The grid line processing module specifically includes: a segmentation unit: which segments the image into grid regions and non-grid regions using connected component analysis based on the characteristics of the image and the structure of the grid; and a morphological processing unit: which further processes the grid regions using morphological operations. Intersection Extraction Unit: Calculates the intersection of grid lines in the travel direction and grid lines in the column direction to obtain the coarse coordinates of the marker points. The calculation formula is as follows: , ;in and For the first The x and y coordinates of each marker point and The first pixel of the pixels enclosed by the marker point The x and y coordinates of each pixel are given, and n is the number of pixels enclosed by that marker. This allows us to calculate the accurate coordinates of each marker. The intersection point sorting unit uses a sorting algorithm to sort the intersection points from left to right and from top to bottom. First, the intersection points are sorted. If the intersection points... > Then swap the two intersection points, and then sort the intersection points by column. If the intersection points... > If the absolute value of the row coordinates of the two intersection points (i and (i+1) is less than devi, then the two intersection points are swapped, where devi is the allowed offset value of the two intersection points in the row direction.

5. A large-format image correction system based on a planar grid method according to claim 1, characterized in that: The image correction module is used to correct the grid image and transmit the corrected image data to the database and the accuracy calculation module. Specifically, the image correction module includes: a coordinate transformation unit: using camera parameters, object distance, and image size, it can calculate the camera resolution (DPI), and then use the following formula... The physical coordinate system can be calculated. How many pixels does a millimeter-long grid occupy in the pixel coordinate system? Distortion triangulation unit: Every four adjacent intersection points extracted from the distorted grid image form two triangles, one at the bottom left and one at the top right. That is, for an intersection image with P_Columns columns and P_Rows rows, the i-th intersection point can form a bottom-left triangle with the (i+P_Columns)-th and (i+P_Columns+1)-th intersection points, and a top-right triangle with the (i+1)-th and (i+P_Columns+1)-th intersection points. Therefore, there are a total of (P_Columns-1)*(P_Rows-1)*2 triangles. The distorted image of each triangle is extracted and transmitted to the affine transformation unit. Affine transformation unit: Affine transformation can be represented as the following matrix multiplication form. In this matrix, (x, y) are the coordinates of a point in the original image, and (x', y') are the coordinates of the point after the affine transformation. The elements (a, b, c, d, e, f) in the matrix control the scaling, rotation, and translation of the affine transformation. The coordinate transformation unit and its data are used to correct the distortion of each triangle's image and save the correction data. The image interpolation unit performs grayscale interpolation on irregular images to generate regular raster images. It uses a bilinear interpolation algorithm to estimate the value of the interpolated pixel by weighted averaging of the four neighboring pixels. The image stitching unit adds the pixel values ​​of two triangle images of the same rectangle to obtain a single corrected grid. Then, each corrected grid is stitched together to complete the image distortion correction.

6. The large-format image correction system based on the planar grid method according to claim 1, characterized in that: The accuracy calculation module is used to calculate the accuracy of the image after distortion correction. Specifically, the accuracy calculation module includes: an image segmentation unit that uses the OTSU algorithm to segment the foreground and background of the corrected grid image and extracts the point set of the corrected grid image; and an accuracy calculation unit that calculates the distance between the point set of the corrected image and a preset point set, and takes the average value, using the following formula: The average distance accuracy can then be obtained.

Images