Self-adaptive high-robustness checkerboard image distortion correction method and system and medium

Abstract

The invention discloses a self-adaptive high-robustness checkerboard image distortion correction method and system and a medium, and the method comprises the steps: reading an input image, and carrying out the image preprocessing of the input image, and obtaining an expanded image; performing contour screening based on the expanded image to obtain an initial fitting graph, and traversing the initial fitting graph according to a nearest neighbor principle and a nearest neighborhood principle to obtain a fitting quadrangle; performing coordinate assignment on the fitted quadrangles to generate a coordinate point mask image, entering a circulation judgment mechanism, and exiting circulation until the number of the quadrangles meets the requirement; and after the circulation is quitted, finding out a corresponding world coordinate point according to the coordinate point mask image and the corresponding angular point coordinates so as to calculate an image distortion matrix to complete distortion correction of the image. The method does not need to specify size information of checkerboard corners, greatly improves the success rate of corner positioning, and improves the robustness of an image distortion correction algorithm.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically to an adaptive and highly robust checkerboard image distortion correction method, system and medium. Background Art

[0002] Checkerboard image camera calibration is an important image processing technology and the basis of image distortion correction. It is used to correct possible optical distortion of camera lenses, improve measurement accuracy, unify image coordinate systems, optimize algorithm performance, eliminate system errors, and adapt to camera parameter adjustments in different scenarios. Multiple images containing the calibration plate are taken from different angles using a checkerboard or dot array of known size on the checkerboard calibration plate. The image processing algorithm is used to detect the corner points or center positions of the calibration plate and record their pixel coordinates. Then, the corner point positions are optimized, 3D-2D point pairs are prepared, and the calibration algorithm is used to calculate the camera's intrinsic parameter matrix and distortion coefficients. Finally, the accuracy of the calibration results is verified. This process ensures the accuracy and consistency of the camera imaging system, which is critical for various applications that rely on high-quality image data.

[0003] During the calibration process, the accuracy of corner detection is a key link. Although modern image processing algorithms such as OpenCV's find Chess board Corners can automatically detect the corners of the chessboard, in practical applications, the accuracy of corner detection may be affected by factors such as lighting conditions and the cleanliness of the chessboard. At the same time, the chessboard size needs to be specified before corner detection, which further reduces the success rate. Summary of the invention

[0004] The purpose of the present invention is to provide an adaptive and highly robust checkerboard image distortion correction method, system and medium, which does not require the size information of the checkerboard corner points to be specified, greatly improves the success rate of corner point positioning, and further improves the robustness of the image distortion correction algorithm.

[0005] A first aspect of the present invention provides an adaptive and highly robust checkerboard image distortion correction method, comprising the following steps:

[0006] Reading an input image, and performing image preprocessing on the input image to obtain a dilated image;

[0007] Performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral;

[0008] Assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop;

[0009] After exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

[0010] In this solution, the step of reading the input image and performing image preprocessing on the input image to obtain the dilated image specifically includes:

[0011] Read the input image and convert the input image into a single-channel grayscale image;

[0012] After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image.

[0013] The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

[0014] In this solution, the contour screening based on the dilated image to obtain the initial fitting graph specifically includes:

[0015] The expansion image is subjected to contour extraction and screening, wherein:

[0016] Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area;

[0017] Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

[0018] In this solution, the initial fitting figure is traversed according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral, which specifically includes:

[0019] Traversing the initial fitting graph according to the nearest neighbor principle to obtain a fitting quadrilateral containing nearest neighbor information, wherein the nearest neighbor threshold is the side length of the current quadrilateral;

[0020] Use the nearest neighbor principle to traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, add it to the stack, loop through the stack, take out the top element, and traverse the four corner neighborhoods of the quadrilateral. If the neighborhood exists and the number of neighbors of the neighborhood is greater than zero and has not been grouped, assign the same group number and add it to the stack. Stop traversal when the stack is empty.

[0021] In this solution, assigning coordinates to the fitted quadrilateral to generate a coordinate point mask image specifically includes:

[0022] Assign coordinates to the four corner points of each fitted quadrilateral, where the assignment principle is consistent with the assignment of quadrilateral coordinates;

[0023] At the same time, a coordinate point mask image is generated, where the value 255 in the mask image corresponds to the coordinates of the found corner points, and the value 0 corresponds to the coordinates of the missing corner points.

[0024] In this solution, the loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and then exits the loop, specifically including:

[0025] The loop judgment mechanism includes three loop mechanisms, among which:

[0026] The first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal is repeated for the quadrilaterals that are not grouped.

[0027] The second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, then exit the second loop, otherwise the image is expanded again and then traversed and grouped;

[0028] The third loop mechanism corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned.

[0029] The second aspect of the present invention further provides an adaptive highly robust checkerboard image distortion correction system, comprising a memory and a processor, wherein the memory comprises an adaptive highly robust checkerboard image distortion correction method program, and when the adaptive highly robust checkerboard image distortion correction method program is executed by the processor, the following steps are implemented:

[0030] Reading an input image, and performing image preprocessing on the input image to obtain a dilated image;

[0031] Performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral;

[0032] Assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop;

[0033] After exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

[0034] In this solution, the step of reading the input image and performing image preprocessing on the input image to obtain the dilated image specifically includes:

[0035] Read the input image and convert the input image into a single-channel grayscale image;

[0036] After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image.

[0037] The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

[0038] In this solution, the contour screening based on the dilated image to obtain the initial fitting graph specifically includes:

[0039] The expansion image is subjected to contour extraction and screening, wherein:

[0040] Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area;

[0041] Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

[0042] In this solution, the initial fitting figure is traversed according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral, which specifically includes:

[0043] Traversing the initial fitting graph according to the nearest neighbor principle to obtain a fitting quadrilateral containing nearest neighbor information, wherein the nearest neighbor threshold is the side length of the current quadrilateral;

[0044] Use the nearest neighbor principle to traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, add it to the stack, loop through the stack, take out the top element, and traverse the four corner neighborhoods of the quadrilateral. If the neighborhood exists and the number of neighbors of the neighborhood is greater than zero and has not been grouped, assign the same group number and add it to the stack. Stop traversal when the stack is empty.

[0045] In this solution, assigning coordinates to the fitted quadrilateral to generate a coordinate point mask image specifically includes:

[0046] Assign coordinates to the four corner points of each fitted quadrilateral, where the assignment principle is consistent with the assignment of quadrilateral coordinates;

[0047] At the same time, a coordinate point mask image is generated, where the value 255 in the mask image corresponds to the coordinates of the found corner points, and the value 0 corresponds to the coordinates of the missing corner points.

[0048] In this solution, the loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and then exits the loop, specifically including:

[0049] The loop judgment mechanism includes three loop mechanisms, among which:

[0050] The first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal is repeated for the quadrilaterals that are not grouped.

[0051] The second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, then exit the second loop, otherwise the image is expanded again and then traversed and grouped;

[0052] The third loop mechanism corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned.

[0053] A third aspect of the present invention provides a computer-readable storage medium, which includes a machine's adaptive and highly robust checkerboard image distortion correction method program. When the adaptive and highly robust checkerboard image distortion correction method program is executed by a processor, the steps of an adaptive and highly robust checkerboard image distortion correction method as described in any one of the above items are implemented.

[0054] The present invention discloses an adaptive and highly robust checkerboard image distortion correction method, system and medium, which do not need to specify the size information of the checkerboard corner points, greatly improve the success rate of corner point positioning, and thus improve the robustness of the image distortion correction algorithm. Among them, multiple binary image expansion positioning can ensure that as many corner points as possible are found, and as many fitting quadrilaterals as possible are found in the same binary image, ensuring that as many contours that meet the requirements as possible are found for the original image, and thus as many fitting quadrilaterals as possible can be found. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A method step diagram of an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0056] Figure 2 A schematic flow chart of an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0057] Figure 3 A schematic diagram of an input image in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0058] Figure 4 A schematic diagram of a binarized image in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0059] Figure 5 A schematic diagram of an expanded image in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0060] Figure 6 A schematic diagram of an initial fitting graph in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0061] Figure 7 A schematic diagram showing a fitting quadrilateral containing nearest neighbor information in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0062] Figure 8 A schematic diagram of quadrilateral coordinate assignment of an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0063] Fig. 9 A schematic diagram of assigning coordinates of quadrilateral corner points of an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0064] Fig.10 A schematic diagram of a coordinate point mask image of an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0065] Fig.11 A schematic diagram showing successful corner point positioning in an adaptive and highly robust checkerboard image distortion correction method of the present invention is shown;

[0066] Fig.12 A block diagram of an adaptive and highly robust checkerboard image distortion correction system of the present invention is shown. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0069] Figure 1 A flow chart of an adaptive and highly robust checkerboard image distortion correction method of the present application is shown.

[0070] like Figure 1 As shown, the present application discloses an adaptive and highly robust checkerboard image distortion correction method, comprising the following steps:

[0071] S102, reading an input image, and performing image preprocessing on the input image to obtain a dilated image;

[0072] S104, performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral;

[0073] S106, assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop;

[0074] S108, after exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates, so as to calculate the image distortion matrix and complete the distortion correction of the image.

[0075] It should be noted that, in this embodiment, specific reference is made to Figure 2 , showing a flowchart of the distortion correction of the present application, wherein, firstly, the original image (input image) is read in, that is, the input image is read, and the input image is preprocessed to obtain a dilated image, wherein the preprocessing process includes binarization of the image, and then the contour is screened based on the dilated image to obtain an initial fitting graph, and the initial fitting graph is traversed according to the nearest neighbor principle and the nearest neighbor principle to obtain grouped fitting quadrilaterals, and then the coordinates of the fitting quadrilaterals are assigned to generate a coordinate point mask image, and a loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and the loop is exited, and finally after exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

[0076] According to an embodiment of the present invention, the step of reading an input image and performing image preprocessing on the input image to obtain a dilated image specifically includes:

[0077] Read the input image and convert the input image into a single-channel grayscale image;

[0078] After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image.

[0079] The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

[0080] It should be noted that, in this embodiment, Figure 3As shown, the input image is displayed, specifically corresponding to the original image, the input image is converted into a single-channel grayscale image, and after the grayscale conversion, the image is subjected to graphic filtering processing and binarization processing is performed using local dynamic threshold segmentation to obtain a binary image, specifically as shown Figure 4 As shown, it is displayed as a binary image, and then the binary image is expanded so that the black squares after the binary are not adhered, so as to obtain the following Figure 5 The dilated image is shown.

[0081] According to an embodiment of the present invention, the step of performing contour screening based on the dilated image to obtain an initial fitting graph specifically includes:

[0082] The expansion image is subjected to contour extraction and screening, wherein:

[0083] Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area;

[0084] Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

[0085] It should be noted that, in this embodiment, when extracting and screening the contours of the binary expansion image, the corresponding rules need to be followed. Specifically, Rule 1: contours that are not hole contours need to be filtered out, and Rule 2: contours that are smaller than a certain area threshold are directly filtered out. Therefore, when applied, non-hole contours and hole contours whose threshold area is smaller than a preset area are filtered out; further, the fitting parameters are fine-tuned to perform multiple polygon fittings, retaining the contours that can fit the quadrilateral, retaining the coordinates of the four corner points obtained by fitting and the side length information, and the final initial fitting graph is as follows: Figure 6 shown.

[0086] According to an embodiment of the present invention, traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral specifically includes:

[0087] Traversing the initial fitting graph according to the nearest neighbor principle to obtain a fitting quadrilateral containing nearest neighbor information, wherein the nearest neighbor threshold is the side length of the current quadrilateral;

[0088] Use the nearest neighbor principle to traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, add it to the stack, loop through the stack, take out the top element, and traverse the four corner neighborhoods of the quadrilateral. If the neighborhood exists and the number of neighbors of the neighborhood is greater than zero and has not been grouped, assign the same group number and add it to the stack. Stop traversal when the stack is empty.

[0089] It should be noted that in this embodiment, two principles are described, one is the nearest neighbor principle, and the other is the nearest neighbor principle. Specifically, the nearest neighbor principle is used to obtain each fitting quadrilateral, and the nearest neighbor principle is used to group the obtained fitting quadrilaterals. Specifically, the initial fitting graph is traversed according to the nearest neighbor principle to obtain the fitting quadrilateral containing the nearest neighbor information, as shown in the following example. Figure 7 As shown, the nearest threshold is the side length of the current quadrilateral.

[0090] Furthermore, due to the influence of illumination, adjacent quadrilaterals may be divided into multiple adjacent regions. The fitted quadrilaterals are traversed and the nth adjacent region quadrilateral is found according to the nearest neighbor principle. The specific grouping method is as follows:

[0091] (1) Traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, and add it to the stack;

[0092] (2) Loop through the stack, take out the top element, and traverse the neighbors of the four corners of the quadrilateral. If the neighbors exist, the number of neighbors of the neighbors is greater than zero and has not been grouped, assign the same group number and add them to the stack;

[0093] (3) Stop traversing until the stack is empty and find all elements in this group.

[0094] According to an embodiment of the present invention, assigning coordinates to the fitted quadrilateral to generate a coordinate point mask image specifically includes:

[0095] Assign coordinates to the four corner points of each fitted quadrilateral, where the assignment principle is consistent with the assignment of quadrilateral coordinates;

[0096] At the same time, a coordinate point mask image is generated, where the value 255 in the mask image corresponds to the coordinates of the found corner points, and the value 0 corresponds to the coordinates of the missing corner points.

[0097] It should be noted that in this embodiment, the coordinates of the fitted quadrilaterals of the current group are assigned in the order of the nearest neighbors of the quadrilaterals, where (1) all the fitted quadrilaterals are traversed to find the first quadrilateral with a neighbor of "4", assign the coordinates to (0,0), add it to the stack, traverse the stack in a loop, take out the top element of the stack, and traverse and assign the coordinates of each inner corner according to the principle of adding "1" to the upper left row number, subtracting "1" from the lower left row number, subtracting "1" from the upper right column number, and adding "1" to the lower right column number. Specifically, Figure 8 As shown, it is a schematic diagram of quadrilateral coordinate assignment.

[0098] Furthermore, the coordinates of the four corner points of each fitted quadrilateral are assigned. Since the assignment principle is consistent with the quadrilateral coordinate assignment, it will not be repeated here. The final schematic diagram of the quadrilateral corner point coordinate assignment is as follows: Fig. 9As shown, in addition, a coordinate point mask image is generated at the same time, as shown in Fig.10 As shown, the value "255" in the mask image corresponds to the coordinates of the found corner points, and the value "0" corresponds to the coordinates of the missing corner points.

[0099] According to an embodiment of the present invention, the loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and then exits the loop, specifically including:

[0100] The loop judgment mechanism includes three loop mechanisms, among which:

[0101] The first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal is repeated for the quadrilaterals that are not grouped.

[0102] The second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, then exit the second loop, otherwise the image is expanded again and then traversed and grouped;

[0103] The third loop mechanism corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned.

[0104] It should be noted that, in this embodiment, Figure 2 As shown, there are two obvious loops in the flowchart of distortion correction, corresponding to the first loop mechanism and the second loop mechanism, wherein the first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal of the quadrilaterals without grouping is repeated; the second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein, if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, the second loop is exited, otherwise the image is expanded again and then traversed and grouped. wherein, since the number of quadrilaterals that are first proposed to be looped must meet the requirements of the first loop, that is, the number of quadrilaterals exceeds half of the total number, therefore, in Figure 3 The second loop judgment box in the flowchart has been reduced equivalently, specifically, whether the number of quadrilaterals in the current group exceeds half. Furthermore, the present embodiment also includes a third loop mechanism, which specifically corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned. The purpose of this application is to improve the success rate of corner point positioning. The corner points that are successfully positioned can be highlighted. Fig.11 As shown, it is a schematic diagram showing the effect of successfully locating the corner points corresponding to the current input image. Fig.11 The bright spot in corresponds to Figure 3 The successfully located corner points in the input image are shown.

[0105] Furthermore, it should be noted that after exiting the loop, the corner points are successfully located, and the corresponding world coordinate points can be found according to the corresponding coordinate point mask image and the corner point coordinates, so as to calculate the image distortion matrix. Finally, the distortion correction of the current input image is completed based on the image distortion matrix. Among them, the step of correcting the image using a matrix is ​​a scenario-based application of the prior art and will not be elaborated here.

[0106] Fig.12 A block diagram of an adaptive and highly robust checkerboard image distortion correction system of the present invention is shown.

[0107] like Fig.12 As shown, the present invention discloses an adaptive highly robust checkerboard image distortion correction system, comprising a memory and a processor, wherein the memory comprises an adaptive highly robust checkerboard image distortion correction method program, and when the adaptive highly robust checkerboard image distortion correction method program is executed by the processor, the following steps are implemented:

[0108] Reading an input image, and performing image preprocessing on the input image to obtain a dilated image;

[0109] Performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral;

[0110] Assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop;

[0111] After exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

[0112] It should be noted that, in this embodiment, specific reference is made to Figure 2, showing a flowchart of the distortion correction of the present application, wherein, firstly, the original image (input image) is read in, that is, the input image is read, and the input image is preprocessed to obtain a dilated image, wherein the preprocessing process includes binarization of the image, and then the contour is screened based on the dilated image to obtain an initial fitting graph, and the initial fitting graph is traversed according to the nearest neighbor principle and the nearest neighbor principle to obtain grouped fitting quadrilaterals, and then the coordinates of the fitting quadrilaterals are assigned to generate a coordinate point mask image, and a loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and the loop is exited, and finally after exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

[0113] According to an embodiment of the present invention, the step of reading an input image and performing image preprocessing on the input image to obtain a dilated image specifically includes:

[0114] Read the input image and convert the input image into a single-channel grayscale image;

[0115] After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image.

[0116] The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

[0117] It should be noted that, in this embodiment, Figure 3 As shown, the input image is displayed, specifically corresponding to the original image, the input image is converted into a single-channel grayscale image, and after the grayscale conversion, the image is subjected to graphic filtering processing and binarization processing is performed using local dynamic threshold segmentation to obtain a binary image, specifically as shown Figure 4 As shown, it is displayed as a binary image, and then the binary image is expanded so that the black squares after the binary are not adhered, so as to obtain the following Figure 5 The dilated image is shown.

[0118] According to an embodiment of the present invention, the step of performing contour screening based on the dilated image to obtain an initial fitting graph specifically includes:

[0119] The expansion image is subjected to contour extraction and screening, wherein:

[0120] Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area;

[0121] Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

[0122] It should be noted that, in this embodiment, when extracting and screening the contours of the binary expansion image, the corresponding rules need to be followed. Specifically, Rule 1: contours that are not hole contours need to be filtered out, and Rule 2: contours that are smaller than a certain area threshold are directly filtered out. Therefore, when applied, non-hole contours and hole contours whose threshold area is smaller than a preset area are filtered out; further, the fitting parameters are fine-tuned to perform multiple polygon fittings, retaining the contours that can fit the quadrilateral, retaining the coordinates of the four corner points obtained by fitting and the side length information, and the final initial fitting graph is as follows: Figure 6 shown.

[0123] According to an embodiment of the present invention, traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral specifically includes:

[0124] Traversing the initial fitting graph according to the nearest neighbor principle to obtain a fitting quadrilateral containing nearest neighbor information, wherein the nearest neighbor threshold is the side length of the current quadrilateral;

[0125] Use the nearest neighbor principle to traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, add it to the stack, loop through the stack, take out the top element, and traverse the four corner neighborhoods of the quadrilateral. If the neighborhood exists and the number of neighbors of the neighborhood is greater than zero and has not been grouped, assign the same group number and add it to the stack. Stop traversal when the stack is empty.

[0126] It should be noted that in this embodiment, two principles are described, one is the nearest neighbor principle, and the other is the nearest neighbor principle. Specifically, the nearest neighbor principle is used to obtain each fitting quadrilateral, and the nearest neighbor principle is used to group the obtained fitting quadrilaterals. Specifically, the initial fitting graph is traversed according to the nearest neighbor principle to obtain the fitting quadrilateral containing the nearest neighbor information, as shown in the following example. Figure 7 As shown, the nearest threshold is the side length of the current quadrilateral.

[0127] Furthermore, due to the influence of illumination, adjacent quadrilaterals may be divided into multiple adjacent regions. The fitted quadrilaterals are traversed and the nth adjacent region quadrilateral is found according to the nearest neighbor principle. The specific grouping method is as follows:

[0128] (1) Traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, and add it to the stack;

[0129] (2) Loop through the stack, take out the top element, and traverse the neighbors of the four corners of the quadrilateral. If the neighbors exist, the number of neighbors of the neighbors is greater than zero and has not been grouped, assign the same group number and add them to the stack;

[0130] (3) Stop traversing until the stack is empty and find all elements in this group.

[0131] According to an embodiment of the present invention, assigning coordinates to the fitted quadrilateral to generate a coordinate point mask image specifically includes:

[0132] Assign coordinates to the four corner points of each fitted quadrilateral, where the assignment principle is consistent with the assignment of quadrilateral coordinates;

[0133] At the same time, a coordinate point mask image is generated, where the value 255 in the mask image corresponds to the coordinates of the found corner points, and the value 0 corresponds to the coordinates of the missing corner points.

[0134] It should be noted that in this embodiment, the coordinates of the fitted quadrilaterals of the current group are assigned in the order of the nearest neighbors of the quadrilaterals, where (1) all the fitted quadrilaterals are traversed to find the first quadrilateral with a neighbor of "4", assign the coordinates to (0,0), add it to the stack, traverse the stack in a loop, take out the top element of the stack, and traverse and assign the coordinates of each inner corner according to the principle of adding "1" to the upper left row number, subtracting "1" from the lower left row number, subtracting "1" from the upper right column number, and adding "1" to the lower right column number. Specifically, Figure 8 As shown, it is a schematic diagram of quadrilateral coordinate assignment.

[0135] Furthermore, the coordinates of the four corner points of each fitted quadrilateral are assigned. Since the assignment principle is consistent with the quadrilateral coordinate assignment, it will not be repeated here. The final schematic diagram of the quadrilateral corner point coordinate assignment is as follows: Fig. 9 As shown, in addition, a coordinate point mask image is generated at the same time, as shown in Fig.10 As shown, the value "255" in the mask image corresponds to the coordinates of the found corner points, and the value "0" corresponds to the coordinates of the missing corner points.

[0136] According to an embodiment of the present invention, the loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and then exits the loop, specifically including:

[0137] The loop judgment mechanism includes three loop mechanisms, among which:

[0138] The first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal is repeated for the quadrilaterals that are not grouped.

[0139] The second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, then exit the second loop, otherwise the image is expanded again and then traversed and grouped;

[0140] The third loop mechanism corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned.

[0141] It should be noted that, in this embodiment, Figure 2 As shown, there are two obvious loops in the flowchart of distortion correction, corresponding to the first loop mechanism and the second loop mechanism, wherein the first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal of the quadrilaterals without grouping is repeated; the second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein, if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, the second loop is exited, otherwise the image is expanded again and then traversed and grouped. wherein, since the number of quadrilaterals that are first proposed to be looped must meet the requirements of the first loop, that is, the number of quadrilaterals exceeds half of the total number, therefore, in Figure 3 The second loop judgment box in the flowchart has been reduced equivalently, specifically, whether the number of quadrilaterals in the current group exceeds half. Furthermore, the present embodiment also includes a third loop mechanism, which specifically corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned. The purpose of this application is to improve the success rate of corner point positioning. The corner points that are successfully positioned can be highlighted. Fig.11 As shown, it is a schematic diagram showing the effect of successfully locating the corner points corresponding to the current input image. Fig.11 The bright spot in corresponds to Figure 3 The successfully located corner points in the input image are shown.

[0142] Furthermore, it should be noted that after exiting the loop, the corner points are successfully located, and the corresponding world coordinate points can be found according to the corresponding coordinate point mask image and the corner point coordinates, so as to calculate the image distortion matrix. Finally, the distortion correction of the current input image is completed based on the image distortion matrix. Among them, the step of correcting the image using a matrix is ​​a scenario-based application of the prior art and will not be elaborated here.

[0143] A third aspect of the present invention provides a computer-readable storage medium, which includes an adaptive and highly robust checkerboard image distortion correction method program. When the adaptive and highly robust checkerboard image distortion correction method program is executed by a processor, the steps of an adaptive and highly robust checkerboard image distortion correction method as described in any of the above items are implemented.

[0144] The present invention discloses an adaptive and highly robust checkerboard image distortion correction method, system and medium, which do not need to specify the size information of the checkerboard corner points, greatly improve the success rate of corner point positioning, and thus improve the robustness of the image distortion correction algorithm. Among them, multiple binary image expansion positioning can ensure that as many corner points as possible are found, and as many fitting quadrilaterals as possible are found in the same binary image, ensuring that as many contours that meet the requirements as possible are found for the original image, and thus as many fitting quadrilaterals as possible can be found.

[0145] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0146] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0147] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0148] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and other media that can store program codes.

[0149] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. An adaptive and highly robust checkerboard image distortion correction method, characterized in that: The following steps are involved: Reading an input image, and performing image preprocessing on the input image to obtain a dilated image; Performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral; Assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop; After exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

2. The adaptive and highly robust checkerboard image distortion correction method according to claim 1, characterized in that: The step of reading an input image and performing image preprocessing on the input image to obtain a dilated image specifically includes: Read the input image and convert the input image into a single-channel grayscale image; After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image. The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

3. The adaptive and highly robust checkerboard image distortion correction method according to claim 2, characterized in that: The step of performing contour screening based on the dilated image to obtain an initial fitting graph specifically includes: The expansion image is subjected to contour extraction and screening, wherein: Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area; Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

4. The adaptive and highly robust checkerboard image distortion correction method according to claim 3, characterized in that: The traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain the fitting quadrilateral specifically includes: Traversing the initial fitting graph according to the nearest neighbor principle to obtain a fitting quadrilateral containing nearest neighbor information, wherein the nearest neighbor threshold is the side length of the current quadrilateral; Use the nearest neighbor principle to traverse the fitted quadrilaterals, find the first fitted quadrilateral whose neighborhood is greater than zero and has not been grouped, add it to the stack, loop through the stack, take out the top element, and traverse the four corner neighborhoods of the quadrilateral. If the neighborhood exists and the number of neighbors of the neighborhood is greater than zero and has not been grouped, assign the same group number and add it to the stack. Stop traversal when the stack is empty.

5. The adaptive and highly robust checkerboard image distortion correction method according to claim 4, characterized in that: The step of assigning coordinates to the fitted quadrilateral to generate a coordinate point mask image specifically includes: Assign coordinates to the four corner points of each fitted quadrilateral, where the assignment principle is consistent with the assignment of quadrilateral coordinates; At the same time, a coordinate point mask image is generated, where the value 255 in the mask image corresponds to the coordinates of the found corner points, and the value 0 corresponds to the coordinates of the missing corner points.

6. The adaptive and highly robust checkerboard image distortion correction method according to claim 5, characterized in that: The loop judgment mechanism is entered until the number of quadrilaterals meets the requirement and then exits the loop, specifically including: The loop judgment mechanism includes three loop mechanisms, among which: The first loop mechanism corresponds to calculating the number of quadrilaterals in the current group. If the number exceeds half of the total number, the first loop is exited, otherwise the traversal is repeated for the quadrilaterals that are not grouped. The second loop mechanism corresponds to comparing the number of quadrilaterals before and after the first loop mechanism exits the loop, wherein if the number of quadrilaterals that exit the loop first is greater than or equal to the number of quadrilaterals that exit the loop later, then exit the second loop, otherwise the image is expanded again and then traversed and grouped; The third loop mechanism corresponds to calculating whether the number of corner points is less than the corner point threshold. If the number of corner points is greater than or equal to the corner point threshold, the third loop is exited, otherwise the corner point positioning failure is returned.

7. An adaptive and highly robust checkerboard image distortion correction system, characterized in that: The invention comprises a memory and a processor, wherein the memory comprises an adaptive and highly robust chessboard image distortion correction method program, and the adaptive and highly robust chessboard image distortion correction method program is executed by the processor to implement the following steps: Reading an input image, and performing image preprocessing on the input image to obtain a dilated image; Performing contour screening based on the dilated image to obtain an initial fitting graph, and traversing the initial fitting graph according to the nearest neighbor principle and the nearest neighbor principle to obtain a fitting quadrilateral; Assigning coordinates to the fitted quadrilaterals to generate a coordinate point mask image, and entering a loop judgment mechanism until the number of quadrilaterals meets the requirement and exiting the loop; After exiting the loop, the corresponding world coordinate point is found according to the coordinate point mask image and the corresponding corner point coordinates to calculate the image distortion matrix to complete the distortion correction of the image.

8. The adaptive and highly robust checkerboard image distortion correction system according to claim 7, characterized in that: The step of reading an input image and performing image preprocessing on the input image to obtain a dilated image specifically includes: Read the input image and convert the input image into a single-channel grayscale image; After grayscale conversion, the image is filtered and binarized using local dynamic threshold segmentation to obtain a binary image. The binary image is dilated so that the black squares after the binary image are not adhered to each other, thereby obtaining the dilated image.

9. The adaptive and highly robust checkerboard image distortion correction system according to claim 8, characterized in that: The step of performing contour screening based on the dilated image to obtain an initial fitting graph specifically includes: The expansion image is subjected to contour extraction and screening, wherein: Filter out non-hole contours and filter out hole contours whose threshold area is smaller than a preset area; Polygon fitting is performed based on the fitting parameters to retain the hole contour that can be fitted into a quadrilateral, as well as the corresponding corner point coordinates and side lengths.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an adaptive and highly robust checkerboard image distortion correction method program. When the adaptive and highly robust checkerboard image distortion correction method program is executed by a processor, the steps of an adaptive and highly robust checkerboard image distortion correction method as described in any one of claims 1 to 6 are implemented.

Images