A method and device for correcting image nonlinear distortion and a storage medium

Abstract

The application discloses a method and device for correcting image nonlinear distortion and a storage medium, and is used for eliminating nonlinear distortion and improving detection efficiency and quality. The application comprises the following steps: inputting a checkerboard image into a target image, and dividing the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image with nonlinear distortion; extracting sub-region corner point coordinates of the sub-regions; sorting the sub-region corner point coordinates; calculating a homographic transformation matrix according to the sub-region corner point coordinates, black-and-white grid corner point coordinates of the checkerboard image and a sorting result, wherein the black-and-white grid corner point coordinates are standard corner point coordinates of a plurality of black-and-white grids in the checkerboard image; and converting the target image into a corrected image through the homographic transformation matrix.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a method and device for correcting image non-linear distortion and a storage medium. BACKGROUND

[0002] In the current booming development of virtual reality (VR) technology, creating a highly realistic, comfortable and immersive virtual experience for users has become the core goal of the industry. However, non-linear distortion is a key problem that exists universally in VR scenes and needs to be solved urgently.

[0003] From the optical principle, the structural characteristics of the short-focus lens determine that the curvature of the lens is much larger than that of the normal lens. When light passes through such a large curvature lens, the propagation path of the light will change dramatically. Especially in the edge area of the lens, due to the larger curvature, the light will produce more severe refraction phenomenon, so that the image after imaging through the lens appears obvious deformation in the edge part, which deviates compared with normal imaging, and then forms non-linear distortion.

[0004] At the same time, in the optical system of the VR headset, the display screen is placed close to the lens, and the pixel points at different positions on the screen present a complex non-linear mapping relationship on the final imaging plane after refraction through the lens, that is, there is distortion and deformation. This non-linear mapping relationship aggravates the degree of non-linear distortion of the image, resulting in obvious distortion and deformation of the picture presented in the VR headset.

[0005] This non-linear distortion not only affects the viewing experience of the user, but also in the process of defect detection of the VR display screen, because the optical system of AOI all adopts conventional lenses, when shooting the VR screen body, obvious pillow-shaped distortion will appear, which makes the defect detection unable to proceed normally. SUMMARY

[0006] The present application discloses a method and device for correcting image non-linear distortion and a storage medium, which are used to eliminate non-linear distortion and improve detection efficiency and quality.

[0007] The first aspect of the present application discloses a method for correcting image non-linear distortion, comprising:

[0008] The chessboard image is input into a target image, and the target image is segmented into several sub-regions according to the chessboard image, the target image being an image with nonlinear distortion; sub-region corner point coordinates of the sub-regions are extracted; the sub-region corner point coordinates are sorted; a homographic transformation matrix is calculated according to the sub-region corner point coordinates, black-and-white grid corner point coordinates of the chessboard image, and a sorting result, the black-and-white grid corner point coordinates being standard corner point coordinates of several black-and-white grids in the chessboard image; and the target image is converted into a corrected image through the homographic transformation matrix.

[0009] Optionally, before the sorting of the sub-region corner point coordinates, the correction method further comprises:

[0010] Removing abnormal points in the sub-region corner point coordinates.

[0011] Optionally, the removing of the abnormal points in the sub-region corner point coordinates comprises:

[0012] Setting a row distance threshold and a column distance threshold between the sub-regions;

[0013] Sorting row coordinates of the corner point coordinates, and obtaining a first row coordinate of a first corner point coordinate;

[0014] Calculating a first common row upper and lower limit according to the first row coordinate and the row distance threshold, and obtaining a row coordinate array according to the first common row upper and lower limit;

[0015] According to the row coordinate array, a corresponding column coordinate array is extracted;

[0016] Performing an operation on values in the column coordinate array to obtain a column coordinate difference value array;

[0017] If the column coordinate difference value is less than the column distance threshold, the column coordinate difference value is extracted and summarized as an abnormal point array;

[0018] Analyzing elements in the abnormal point array through a preset condition, and removing corresponding abnormal points according to an analysis result, the elements in the abnormal point array including the column coordinate difference value less than the column distance threshold.

[0019] Optionally, the analyzing of the elements in the abnormal point array through the preset condition and the removing of the corresponding abnormal points according to the analysis result comprise:

[0020] If the element in the abnormal point array is a first element of the column coordinate difference value array, the sub-region corner point coordinate associated with a first element of the row coordinate array is removed;

[0021] If the element in the abnormal point array is not the last element in the column coordinate difference value array, a first difference value between the corresponding element in the row coordinate array and the previous element is obtained;

[0022] If the element in the abnormal point array is not the last element in the column coordinate difference value array, a first difference value between the corresponding element in the row coordinate array and the previous element is obtained;

[0023] If the first difference value is less than the column distance threshold, the sub-region corner point coordinates associated with the corresponding element in the row coordinate array are removed, otherwise the sub-region corner point coordinates associated with the next element of the corresponding element in the row coordinate array are removed.

[0024] Optionally, the sorting of the sub-region corner point coordinates comprises:

[0025] Step one: sort the row coordinates in the row coordinate array and the column coordinates in the column coordinate array respectively, and extract the minimum row coordinate and the minimum column coordinate;

[0026] Step two: according to the row distance threshold and the column distance threshold, obtain the upper and lower limits of the common row and the common column corresponding to the minimum row coordinate and the minimum column coordinate;

[0027] Step three: record and store the row coordinates and the column coordinates that meet the upper and lower limits of the common row and the common column to the common row index set and the common column index set respectively;

[0028] Step four: remove the row coordinates and the column coordinates in the row coordinate array and the column coordinate array that have been stored in the common row index set and the common column index set;

[0029] Step five: repeat steps one to four until all the row coordinates and the column coordinates in the row coordinate array and the column coordinate array are traversed;

[0030] Step six: traverse all common row index sets and common column index sets, and extract index values that appear in both the common row index set and the common column index set;

[0031] Step seven: according to the common row index set and the common column index set where the index value is located, the sub-region corner point coordinates corresponding to the index value are placed in a new coordinate set, and the new coordinate set has a length that is the product of the number of rows and columns of the black and white grid in the checkerboard image;

[0032] Step eight: repeat steps six and seven until all the sub-region corner point coordinates corresponding to the index values are placed in the new coordinate set.

[0033] Optionally, before the step eight, the correction method further comprises:

[0034] determining whether the number of the common row index set or the common column index set is less than the number of black and white grid rows and columns in the checkerboard image;

[0035] if yes, calculating a missing corner point coordinate according to the common row index set and the common column index set;

[0036] putting the missing corner point coordinate into the new coordinate set.

[0037] Optionally, the calculating a missing corner point coordinate according to the common row index set and the common column index set comprises:

[0038] traversing the new coordinate set to record a row and column serial number of the missing corner point coordinate;

[0039] extracting the common row index set and the common column index set in which the missing corner point coordinate is located according to the row and column serial number, respectively;

[0040] calculating a common row second-order fitting coefficient and a common column second-order fitting coefficient according to the common row index set and the common column index set;

[0041] calculating the missing corner point coordinate according to the common row second-order fitting coefficient and the common column second-order fitting coefficient.

[0042] The second aspect of the present application provides a device for correcting image nonlinear distortion, comprising:

[0043] a segmentation unit configured to input a checkerboard image to a target image and segment the target image into a plurality of sub-regions according to the checkerboard image, the target image being an image with nonlinear distortion;

[0044] an extraction unit configured to extract sub-region corner point coordinates of the sub-regions;

[0045] an ordering unit configured to order the sub-region corner point coordinates;

[0046] a calculation unit configured to calculate a homographic transformation matrix according to the sub-region corner point coordinates and black and white grid corner point coordinates of the checkerboard image, the black and white grid corner point coordinates being standard corner point coordinates of a plurality of black and white grids in the checkerboard image;

[0047] a conversion unit configured to convert the target image into a corrected image through the homographic transformation matrix.

[0048] The third aspect of the present application provides a device for correcting image nonlinear distortion, comprising:

[0049] The processor, the memory, the input / output unit and the bus;

[0050] The processor is connected with the memory, the input / output unit and the bus;

[0051] The memory stores a program, and the processor invokes the program to execute the method for correcting as the first aspect and any optional correction method of the first aspect.

[0052] The fourth aspect of the present application provides a computer readable storage medium, and the computer readable storage medium stores a program, and the program executes the method for correcting as the first aspect and any optional correction method of the first aspect when executed on a computer.

[0053] From the above technical solutions, the embodiments of the present application have the following advantages:

[0054] For the non-linear distortion of the display screen, the regular grid of the checkerboard image can fit the distortion form, and the target image is divided into sub-regions adapted to the distortion trend. By decomposing the complex global non-linear distortion into local relatively controllable sub-region distortion, the difficulty of subsequent correction is reduced, and the correction is more targeted. The sub-region corner point coordinates contain spatial position information, which can clearly reflect the arrangement order of the sub-regions in the target image. Extracting the sub-region corner point coordinates helps to quantify the sub-region distortion state and provides data support for subsequent calculation of the correction matrix.

[0055] The sub-region corner point coordinates are sorted to establish an accurate correspondence between the sub-region corner points and the standard checkerboard corner points. At the same time, sorting can correct the confusion of the spatial positions of the corner points caused by non-linear distortion, so that the corner point coordinates strictly follow the row and column logic of the checkerboard. This ensures that the distorted corner points and the standard corner points can be accurately corresponded when calculating the homography matrix, and avoids introducing correction errors due to coordinate mismatch.

[0056] The homographic transformation matrix is obtained by the sub-region corner point coordinates, the standard black and white grid corner point coordinates and the sorting result. The homographic transformation matrix can accurately describe the conversion rule between the distorted corner points and the standard corner points. Using the calculated homographic transformation matrix to convert the target image can effectively eliminate the distortion, and the originally distorted image frame is converted into a normal and clear image, which restores the image closest to the real display screen state. This enables the defect detection system to accurately identify scratches, bright spots, dark spots and other defects on the display screen, and improves the detection efficiency and quality. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor under the premise of these drawings.

[0058] Figure 1 An embodiment of the method for correcting the nonlinear distortion of the image in the present application;

[0059] Figure 2 An embodiment of the method for removing the abnormal points in the present application;

[0060] Figure 3 An embodiment of the method for removing the abnormal points according to the analysis result in the present application;

[0061] Figure 4 An embodiment of the method for sorting the coordinates of the corner points in the sub-regions in the present application;

[0062] Figure 5 An embodiment of the method for supplementing the missing corner point coordinates in the present application;

[0063] Figure 6 An embodiment of the method for calculating the missing corner point coordinates in the present application;

[0064] Figure 7 An embodiment of the structure of the device for correcting the nonlinear distortion of the image in the present application;

[0065] Figure 8 Another embodiment of the structure of the device for correcting the nonlinear distortion of the image in the present application;

[0066] Figure 9 An embodiment of the image generating nonlinear distortion in the present application;

[0067] Figure 10 An embodiment of the checkerboard image in the present application;

[0068] Figure 11 An embodiment of the method for identifying the corner points in the sub-regions in the present application;

[0069] Figure 12 An embodiment of the type of the abnormal corner point coordinates after the serious distortion;

[0070] Figure 13 An embodiment of the corner points not extracted in the present application;

[0071] Figure 14This is a schematic diagram of the intersection of two parabolas with the edge of the image in this application;

[0072] Figure 15 This is a schematic diagram for supplementing the coordinates of missing corner points in this application. Detailed Implementation

[0073] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0074] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0075] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0076] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0077] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0078] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specification are not necessarily all referring to the same embodiment, however, are meant to convey a particular feature, structure, or characteristic that is included in at least one embodiment of the application. The terms "including," "comprising," "having," and variations thereof are meant to encompass the items listed thereafter, but do not exclude other items from also being present. Unless otherwise indicated, the terms "including", "comprising", "having” and variations thereof are meant to encompass non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0079] In the optical system of the VR head-mounted display, the display screen is placed close to the lens, and the pixel points at different positions on the screen present a complex nonlinear mapping relationship on the final imaging plane after refraction through the lens, that is, there is distortion and deformation. This nonlinear mapping relationship aggravates the degree of nonlinear distortion of the image, resulting in obvious distortion and deformation of the picture presented in the VR head-mounted display.

[0080] Nonlinear distortion includes barrel distortion and pincushion distortion, as shown in Figure 9 Figure 9 An illustrative diagram of image nonlinear distortion. This nonlinear distortion not only affects the viewing experience of the user, but also makes it impossible for the defect detection to proceed normally in the process of defect detection of the VR display screen, because the optical system of the AOI adopts a conventional lens, and obvious pincushion distortion occurs when the VR screen body is photographed.

[0081] Based on this, the application discloses a method for correcting image nonlinear distortion, which is used to eliminate nonlinear distortion and improve detection efficiency and quality.

[0082] The technical solutions in the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0083] The method of the application can be applied to a server, a device, a terminal or other devices with logical processing capability, and the application is not limited thereto. For the convenience of description, the following will be described taking the system as an example.

[0084] Please refer to Figure 1 An embodiment of the method for correcting image nonlinear distortion is provided, which includes:

[0085] ​101. Input the checkerboard image into the target image, and divide the target image into several sub-regions according to the checkerboard image. The target image is an image that produces nonlinear distortion.

[0086] checkerboard image as Figure 10 As shown, the checkerboard pattern on the VR display screen is illuminated by the PG (image generator), and the checkerboard image is input into the target image that generates nonlinear distortion. That is, the checkerboard pattern is directly presented on the display screen to be detected as a reference template, forming a reference and detection relationship under the same carrier with the target image of the display screen itself.

[0087] The target image undergoes non-linear distortion due to its display characteristics, and the horizontal and vertical grid lines of the checkerboard pattern deform synchronously with the distortion of the target image. Using the deformed checkerboard grid lines as boundaries, the target image is divided into several sub-regions, each sub-region corresponding to the display area covered by one grid cell in the checkerboard.

[0088] The original overall distorted target image is decomposed into local sub-regions by segmentation, and the degree of distortion in each sub-region is relatively uniform (for example, the central sub-region has less distortion, and the edge sub-regions have a consistent distortion trend).

[0089] 102. Extract the coordinates of the corner points of the sub-region;

[0090] After being divided into a checkerboard pattern, the boundary of each sub-region is formed by the grid lines of the deformed checkerboard pattern, and the corner point of the sub-region is the intersection of two adjacent grid lines.

[0091] like Figure 11 As shown, after acquiring the target image containing the checkerboard pattern, an image processing algorithm is used to identify the corner points of the sub-regions. The algorithm locates the corner points based on the abrupt changes in pixel grayscale values ​​at the corner points, and then, combined with a preset image coordinate system, converts the spatial position of the corner points into specific coordinate values, ultimately obtaining the coordinates of the four corner points of each sub-region.

[0092] 103. Sort the coordinates of the corner points of the sub-region;

[0093] The sorting of corner coordinates in sub-regions is based on the original grid logic of the checkerboard. In a distortion-free state, the checkerboard exhibits a regular row and column distribution (e.g., columns from left to right and rows from top to bottom). Therefore, sorting must refer to this inherent pattern and incorporate the spatial location information of the corner coordinates. For example, for sub-regions in the same row, they are sorted by the x-coordinates of the corners from smallest to largest (corresponding to a left-to-right order); for sub-regions in the same column, they are sorted by the y-coordinates of the corners from smallest to largest (corresponding to a top-to-bottom order).

[0094] Firstly, the reference corner points of the sorting (such as the upper left corner point of each sub-region) are determined, and then the coordinates of the reference corner points of all sub-regions are extracted. Then, the sorting is completed through coordinate comparison: first grouping according to the vertical coordinates (the same row for similar vertical coordinates), and then sorting according to the horizontal coordinates in each group. For the edge sub-regions whose coordinates are offset due to distortion, the sorting is still based on the row and column logic of the original grid, and the slight coordinate deviation caused by local distortion is ignored.

[0095] By sorting, a one-to-one correspondence between the sub-region corner points and the standard corner point positions of the chessboard grid is established, avoiding the confusion of corner point positions caused by distortion, and providing ordered matching data for subsequent calculation of the homography transformation matrix.

[0096] 104、According to the sub-region corner point coordinates, the black and white grid corner point coordinates of the chessboard grid image, and the sorting result, the homography transformation matrix is calculated, and the black and white grid corner point coordinates are the standard corner point coordinates of several black and white grids in the chessboard grid image;

[0097] The homography transformation matrix is used to describe the mapping relationship between the target image (image with nonlinear distortion) and the ideal undistorted image. The sub-region corner point coordinates are the corner point positions in the distorted state of the target image, and the spatial correspondence has been clearly defined through sorting; the black and white grid corner point coordinates of the chessboard grid image are the standard corner point positions in the ideal undistorted image, which are known reference data; the sorting result of the sub-region corner point coordinates can ensure one-to-one correspondence between the two groups of corner point coordinates, avoiding misplacement matching. The sub-region corner point coordinates, the black and white grid corner point coordinates of the chessboard grid image, and the sorting result together constitute the matching pair between the distorted coordinates and the standard coordinates, providing the original input for matrix calculation.

[0098] Firstly, based on the sorting result, the sub-region corner point coordinates are paired with the corresponding black and white grid corner point coordinates of the chessboard. For example, the sub-region corner point coordinates of the 3rd row and 2nd column after sorting will form a pair of matching data with the black and white grid standard corner point coordinates of the 3rd row and 2nd column in the chessboard. Then the homography transformation matrix of each matching pair is solved by least squares method. The homography transformation matrix contains multiple parameters, which can quantitatively describe the transformation rule from the distorted corner point coordinates to the standard corner point coordinates, including translation, rotation, scaling and nonlinear distortion correction, etc.

[0099] The specific homography transformation matrix calculation process is as follows:

[0100] Firstly, two groups of coordinates are determined: the sub-region corner point coordinates {X', Y'} are the actual coordinates of the sub-region corner points in the distorted image; the black and white grid corner point coordinates {X, Y} are the standard coordinates in the ideal undistorted state. The sorting result is used to ensure that the two groups of coordinates can be accurately paired according to the corresponding chessboard row and column order.

[0101] Based on these paired coordinates, the mathematical relationship is constructed by the formula:

[0102]

[0103] Solving each parameter of the homographic transformation matrix M by least square method, the homographic transformation matrix M solved by the homographic transformation matrix M can describe the conversion relationship from the distorted coordinates to the standard coordinates.

[0104] The calculated homographic transformation matrix M is equivalent to dividing each row or each column of the distorted corner points into several sub-regions. The position difference between each other is not the same, but for each row or column sub-region, the change basically meets the linear change relationship, that is, the position difference between adjacent pixels remains consistent.

[0105] 105, converting the target image into a corrected image by the homographic transformation matrix.

[0106] For any one pixel point in the target image (not inputting the checkerboard image), its coordinates {X', Y'} in the image coordinate system, combined with the homographic transformation matrix M solved by the least square method before, the pixel position in the distorted state is mapped to the ideal position without distortion.

[0107] Specifically, by traversing each pixel point of the target image, the homographic transformation matrix M is used to calculate the corrected coordinates in turn, and then the pixel values of the target image are rearranged and combined according to these corrected coordinates to construct a corrected image.

[0108] In this embodiment, for the nonlinear distortion of the display screen, the regular grid of the checkerboard image can fit the distorted form, and the target image can be divided into sub-regions adapted to the distortion trend. By decomposing the complex global nonlinear distortion into local relatively controllable sub-region distortion, the subsequent correction difficulty is reduced, and the correction is more targeted. The sub-region corner point coordinates contain spatial position information and can clearly reflect the arrangement order of the sub-regions in the target image. Extracting the sub-region corner point coordinates helps to quantify the sub-region distortion state and provides data support for subsequent calculation of the correction matrix.

[0109] The sorted corner point coordinates are to establish an accurate correspondence between the sub-region corner points and the standard checkerboard corner points. At the same time, the sorting can correct the confusion of the spatial positions of the corner points caused by the nonlinear distortion, so that the corner point coordinates strictly follow the row and column logic of the checkerboard. This ensures that the distorted corner points and the standard corner points can be accurately corresponded when calculating the homographic matrix, and avoids introducing correction errors due to coordinate mismatch.

[0110] The homographic transformation matrix is obtained by the sub-region corner point coordinates, the standard black and white grid corner point coordinates and the sorting result. The homographic transformation matrix can accurately describe the conversion rule between the distorted corner points and the standard corner points. The target image is converted by using the calculated homographic transformation matrix, which can effectively eliminate the distortion. The originally distorted image is converted into a normal and clear image, which restores the image closest to the state of the real display screen. This makes the defect detection system accurately identify the scratches, bright spots, dark spots and other defects on the display screen, and improves the detection efficiency and quality.

[0111] In the above embodiment, there may be abnormal corner point coordinates in the extracted sub-region corner point coordinates due to image noise, occlusion, local severe distortion and other factors. As shown in Figure 12 Figure 12 is an abnormal corner point coordinate type after severe distortion. Because of severe distortion, the imaging of part of the corner point position is blurred, resulting in multiple sets of coordinates of the same corner point. At the same time, impurities, defects and other phenomena are easy to appear in the white small square, which also easily causes interference. In order to ensure the accuracy and stability of subsequent calculation, the operation of removing abnormal points is needed.

[0112] Referring to Figure 2 , an embodiment of a method for removing abnormal points is provided, comprising:

[0113] 201, setting a row distance threshold and a column distance threshold between sub-regions;

[0114] 202, sorting the row coordinates of the corner point coordinates, and obtaining a first row coordinate of a first corner point coordinate;

[0115] 203, calculating a first common row upper and lower limit according to the first row coordinate and the row distance threshold, and obtaining a row coordinate array according to the first common row upper and lower limit;

[0116] 204, extracting a corresponding column coordinate array according to the row coordinate array;

[0117] 205, performing operation on the values in the column coordinate array to obtain a column coordinate difference value array;

[0118] 206, if there is a column coordinate difference value less than the column distance threshold, extracting and summarizing it as an abnormal point array;

[0119] 207, analyzing the elements in the abnormal point array by a preset condition, and removing the corresponding abnormal points according to the analysis result, wherein the elements in the abnormal point array include the column coordinate difference values less than the column distance threshold.

[0120] ​In the checkerboard image, the row distance and column distance between sub-regions are relatively stable under normal circumstances. However, due to image noise, occlusion, and local severe distortion, the extracted corner point coordinates of the sub-regions may be abnormal, causing the row distance and column distance to deviate from the normal range. Setting the row distance threshold and column distance threshold can define a reasonable distance range to determine which corner point coordinates may be abnormal points. The row distance and column distance between sub-regions in a large number of normal checkerboard images can be counted, and their average values can be determined considering a certain fluctuation range, thereby determining the two thresholds.

[0121] The row coordinates of the corner point coordinates are sorted to arrange the corner points in order of rows, facilitating subsequent row-based processing. The first row coordinate of the first corner point coordinate is obtained to determine the starting row for processing and provide a reference for subsequent calculation of the upper and lower limits of the common row.

[0122] Based on the first row coordinate of the first corner point coordinate and in combination with the row distance threshold, the row coordinate range of the corner point coordinates that may belong to the same row, i.e., the first common row upper and lower limits, can be calculated. The corner points corresponding to the row coordinates within this range may be located in the same row. The row coordinate array is filtered through this range to group the corner points that may be in the same row, facilitating subsequent processing of the column coordinates.

[0123] After obtaining the row coordinate array that may be in the same row, the column coordinates corresponding to these row coordinates are extracted from the original set of corner point coordinates to form a column coordinate array. In this way, the column coordinates of the corner points in the same row can be concentrated together, facilitating subsequent analysis of the relationship between the column coordinates and determination of whether there are abnormal points.

[0124] The adjacent element subtraction operation is performed on the values in the column coordinate array to obtain a column coordinate difference array. The column coordinate difference reflects the column distance between adjacent corner points in the same row. By analyzing these differences, it can be determined whether there are corner points with abnormal column distance, i.e., abnormal points.

[0125] Under normal circumstances, the column distance between adjacent corner points in the same row should be greater than or equal to the column distance threshold. If there is a column coordinate difference less than the column distance threshold, it indicates that the distance between the two adjacent corner points is too close, and there may be abnormalities, such as repeatedly extracted corner points or incorrect corner points due to noise. These abnormal corner points are extracted and summarized as an abnormal point array for subsequent processing.

[0126] The elements in the abnormal point array are analyzed according to the predetermined conditions to determine whether the abnormal points are due to repeated extraction or noise. According to different judgment results, appropriate strategies are used to remove abnormal points. If it is a repeatedly extracted corner point, one of them can be directly deleted; if it is a false corner point due to noise, it can be corrected or deleted based on the information of the surrounding corner points.

[0127] In this embodiment, removing the abnormal points can avoid the interference of these error data on the subsequent calculation of the homography transformation matrix. The row distance threshold and the column distance threshold can be adjusted according to the actual distortion degree, so as to avoid excessive removal of normal points and strictly filter obvious abnormal points.

[0128] The homography transformation matrix is calculated based on accurate corner point coordinates. The existence of abnormal points will lead to inaccurate matrix calculation, and then affect the effect of image rectification. By removing the abnormal points, the homography transformation matrix calculated can more accurately describe the distortion of the image, thereby improving the accuracy of image rectification and making the rectified image closer to the real scene.

[0129] Referring to Figure 3 An embodiment of a method for removing abnormal points according to an analysis result is provided, comprising:

[0130] 301. If the first element of the column coordinate difference value array exists in the element of the abnormal point array, remove the sub-region corner point coordinates associated with the first element of the row coordinate array;

[0131] The first element of the column coordinate difference value array represents the distance difference of the first two corner points (the first and the second) in the row coordinate array in the column direction. When this difference exists in the abnormal point array, it means that the column distance between the two adjacent corner points is abnormal. Generally, in a regular image such as a chessboard, the distribution of corner points is relatively uniform and orderly. If the first difference is abnormal, it is likely that the first corner point itself has a problem, such as being a noise point or an error point extracted repeatedly. Therefore, the sub-region corner point coordinates associated with the first element of the row coordinate array are removed to eliminate the influence of this abnormal factor on the subsequent processing.

[0132] 302. If the last element of the column coordinate difference value array exists in the element of the abnormal point array, remove the sub-region corner point coordinates associated with the element after the corresponding element of the row coordinate array;

[0133] The last element of the column coordinate difference value array represents the distance difference of the last two corner points in the row coordinate array in the column direction. When this difference is abnormal, it means that the distance between the last two corner points does not conform to the normal range. Considering the arrangement rule of the chessboard corner points, it is more likely that the last corner point has a problem, which may be caused by noise interference or partial occlusion at the image edge. Therefore, the sub-region corner point coordinates associated with the element after the corresponding element (i.e., the second last element) of the row coordinate array are removed to correct the abnormality.

[0134] 303、If the element in the abnormal point array is neither the first element of the column coordinate difference value array nor the last element of the column coordinate difference value array, a first difference value between the corresponding element in the row coordinate array and the previous element is obtained;

[0135] 304、If the first difference value is less than the column distance threshold value, the sub-region corner point coordinates associated with the corresponding element in the row coordinate array are removed, otherwise the sub-region corner point coordinates associated with the next element of the corresponding element in the row coordinate array are removed.

[0136] When the element in the abnormal point array is in the middle position of the column coordinate difference value array, it indicates that the column distance between the adjacent corner points in the middle of the row coordinate array is abnormal. By calculating the first difference value between the corresponding element (i.e., the row coordinate with an abnormal difference value) in the row coordinate array and the previous element in the row direction, and combining the distribution rule of the chessboard corner points in the row and column directions, the judgment is assisted.

[0137] When the first difference value is less than the column distance threshold value, it indicates that the corresponding element (i.e., the row coordinate with an abnormal difference value) in the row coordinate array and the previous element in the row direction are relatively close. Combined with the uniformity of the distribution of the chessboard corner points, it is more likely that the corresponding element associated with the corner point has a problem, and the corner point may be a noise point or a position error due to local distortion, so the corner point coordinates are removed.

[0138] When the first difference value is greater than or equal to the column distance threshold value, it indicates that the corresponding element in the row coordinate array and the previous element in the row direction are relatively normal. Then, the abnormal column distance is more likely to be the corner point associated with the next element of the corresponding element, which may be due to the fact that the corner point is shifted too much in the column direction, so the sub-region corner point coordinates associated with the next element are removed.

[0139] In this embodiment, by accurately identifying and removing abnormal points through the above steps, the accuracy of the corner point coordinate data can be significantly improved. Differentiated rules are formulated for the "first, last, and middle" positions of the abnormal points in the column coordinate difference value array to avoid the deletion of normal points. After removing the abnormal points, the proportion of abnormal points is significantly reduced, and the sequence of the retained corner points is continuous and the spacing is compliant.

[0140] Accurate corner point coordinates provide accurate correspondence between distorted corner points and standard corner points for subsequent homography matrix calculation, reduce matrix parameter deviation caused by abnormal points, and help improve the quality and effect of the entire image processing process.

[0141] Please refer to Figure 4 An embodiment of a sub-region corner point coordinate sorting method is provided, which includes:

[0142] Step one: Sort the row coordinates in the row coordinate array and the column coordinates in the column coordinate array respectively, and extract the minimum row coordinate and the minimum column coordinate;

[0143] Step two: According to the row distance threshold and the column distance threshold, obtain the upper and lower limits of the common row corresponding to the minimum row coordinate and the common column corresponding to the minimum column coordinate;

[0144] Step three: Record and store the row coordinates and column coordinates that meet the upper and lower limits of the common row and the common column to the common row index set and the common column index set respectively;

[0145] Step four: Remove the row coordinates and column coordinates in the row coordinate array and the column coordinate array that have been stored in the common row index set and the common column index set;

[0146] Step five: Repeat steps one to four until all row coordinates and column coordinates in the row coordinate array and the column coordinate array are traversed;

[0147] Step six: Traverse all common row index sets and common column index sets, and extract index values that appear in both common row index sets and common column index sets;

[0148] Step seven: According to the common row index set and the common column index set where the index value is located, put the sub-region corner point coordinates corresponding to the index value into a new coordinate set. The new coordinate set has a length equal to the product of the number of rows and columns of the black and white grid in the chessboard image;

[0149] Step eight: Repeat steps six and seven until all index values corresponding to the sub-region corner point coordinates are put into the new coordinate set.

[0150] Sort the row coordinate array and the column coordinate array respectively, and extract the minimum row coordinate (the reference of the topmost row of the chessboard) and the minimum column coordinate (the reference of the leftmost column). The minimum row coordinate and the minimum column coordinate are the starting point of the chessboard sorting, which ensures that the order is built from the top left corner of the chessboard.

[0151] Based on the row distance threshold and the column distance threshold, calculate the common row upper and lower limits of the minimum row coordinate and the common column upper and lower limits of the minimum column coordinate. The common row upper and lower limits of the minimum row coordinate are the minimum row coordinate ± the row distance threshold, which correspond to the normal row coordinate range of the same row as the minimum row. The common column upper and lower limits of the minimum column coordinate are the minimum column coordinate ± the column distance threshold, which correspond to the normal column coordinate range of the same column as the minimum column, providing a judgment standard for selecting the corner points of the same row and the same column.

[0152] Store the corner point index of the row coordinate within the common row upper and lower limits into the common row index set (the corner point index of the same row), and store the corner point index of the column coordinate within the common column upper and lower limits into the common column index set (the corner point index of the same column).

[0153] To avoid repeatedly processing corner points whose positions have already been determined, the row and column coordinates already recorded in the index set are removed from the original array. This gradually narrows down the processing scope and improves the algorithm's efficiency.

[0154] By repeatedly performing the above steps, the positions of all corner points in the chessboard are gradually found and recorded in the corresponding index set. Each repetition uses the minimum row and column coordinates among the remaining corner points as a benchmark to determine new upper and lower limits for the same row and column, filtering out corner points that meet the criteria. This is equivalent to "scanning row by row and column by column," first filtering all corner points in the first row, then the second row, and so on, until all rows and columns of the chessboard are covered.

[0155] Iterate through all sets of common row indices (corner indices for each row) and common column indices (corner indices for each column), extracting the index values ​​that appear in both sets. The corners corresponding to these index values ​​are the actual corners in the chessboard. By iterating through all sets and extracting these index values, the valid corner locations can be determined.

[0156] Based on the extracted index values, the corresponding corner coordinates of the sub-regions are obtained from the original row and column coordinate arrays. These coordinates are then placed into a new coordinate set in a specific order. The length of the new coordinate set is the product of the number of rows and columns of black and white squares in the checkerboard image. This ensures that the new coordinate set can completely store all valid corner coordinates and is arranged according to the checkerboard pattern.

[0157] Repeat the matching process until all corner coordinates are filled into the new coordinate set according to their row and column positions, thus completing the sorting.

[0158] In this embodiment, traditional sorting might mistakenly delete corner points that are severely distorted and deviate from the visual rows and columns. However, by using a method of "extracting the minimum coordinates round by round + iteratively filtering the remaining coordinates," all valid corner coordinates in the chessboard can be accurately found, ensuring that all corner points (including those with edge distortion) can be included in their corresponding row or column sets. Then, index matching is used to assign them to a new coordinate set. In the sorted new coordinate set, the corner coordinates at each position form a clear correspondence with the standard corner points of the chessboard, reducing the processing complexity of subsequent steps.

[0159] like Figure 13 As shown, Figure 13 This is a schematic diagram of corner points that were not extracted. Due to factors such as image blurriness or insufficient local contrast, some corner points were not extracted and are marked as -1 in the sorted coordinate set. In order to ensure that the coordinates of all corner points in the subsequent correction image are matched one-to-one, the coordinates of these missing corner points need to be calculated.

[0160] Please see Figure 5The application provides a method for supplementing missing corner point coordinates. One embodiment of the method comprises:

[0161] 501. Determine whether the number of the common row index set or the common column index set is less than the number of black and white grid rows and columns in the checkerboard image.

[0162] 502. If yes, calculate the missing corner point coordinates according to the common row index set and the common column index set.

[0163] 503. Put the missing corner point coordinates into a new coordinate set.

[0164] The common row index set records the column index information of the successfully extracted corner points in each row of the checkerboard image. The common column index set records the row index information of the successfully extracted corner points in each column.

[0165] The purpose of determining whether the number of the common row index set or the common column index set is less than the number of black and white grid rows and columns in the checkerboard image is to determine whether there is a missing corner point row or column in the checkerboard image. In an ideal case, the number of the common row index set and the common column index set should be equal to the actual number of black and white grid rows and columns in the checkerboard image. By comparing the number of the common row index set or the common column index set with the number of black and white grid rows and columns in the checkerboard image, if the number is less than the actual number, it indicates that there is a missing corner point row or column.

[0166] When it is determined that there is a missing corner point row or column, the existing common row index set and common column index set are used to calculate the coordinates of the missing corner points. Since the checkerboard has a certain regularity, the corner points in the image usually present a regular grid distribution.

[0167] For a row with missing corner points, the coordinate information of the extracted corner points in the row is used to fit the distribution curve of the corner points in the row by using a suitable fitting method (such as least square method to fit a straight line). Assuming that the corner point coordinates in a row have been extracted, a straight line can be found by the least square method, so that the sum of the square errors of these corner points to the straight line is minimized. Similarly, for a column with missing corner points, the coordinate information of the extracted corner points in the column is used to fit the distribution curve of the corner points in the column.

[0168] In order to more accurately obtain the missing corner point coordinates, the fitting results of the common row and the common column can be considered comprehensively. Specifically, the distribution law of the corner points in each row can be obtained by common row fitting, and then the distribution law of the corner points in each column can be obtained by common column fitting, and then the coordinates of the missing corner points can be determined by a certain algorithm.

[0169] According to the obtained fitting co-line and co-column curve equation, the coordinates of the missing corner points can be calculated in combination with the rules of the checkerboard. For example, in a regular checkerboard, the distance between adjacent corner points is fixed. After determining the approximate position of the missing corner point according to the fitting curve, the coordinates of the missing corner point can be accurately calculated in combination with the distance information.

[0170] Finally, the calculated coordinates of the missing corner points are placed in a new coordinate set in a certain format and order.

[0171] In this embodiment, for the missing corner points caused by unclear images and insufficient contrast, the missing coordinates are calculated by fitting curves to avoid the presence of unextracted corner points in the new coordinate set.

[0172] Nonlinear distortion has spatial continuity. The co-line and co-column fitting curve is based on spatial continuity, and the missing values are calculated by using the distribution trend of the existing corner point coordinates, so that the completed corner point coordinates not only conform to the overall distortion law, but also smoothly transition with the coordinates of adjacent effective corner points, avoiding distortion caused by random filling.

[0173] The calculation of the homographic transformation matrix requires one-to-one correspondence (equal number and corresponding position) between the target image corner point coordinates and the standard checkerboard corner point coordinates. After filling the missing coordinates, the number of corner points in the new coordinate set is completely consistent with the standard corner points, and each completed coordinate corresponds to a position, ensuring that the matching relationship is unbiased, providing complete coordinate pairs for matrix calculation, reducing matrix parameter errors caused by data missing, and improving the accuracy of subsequent image correction.

[0174] Please refer to Figure 6 The application provides a method for calculating missing corner point coordinates, one embodiment of which comprises:

[0175] 601. Traverse the new coordinate set and record the row and column numbers of the missing corner point coordinates.

[0176] 602. Extract the co-line index set and the co-column index set in which the missing corner point coordinates are located, respectively, according to the row and column numbers.

[0177] 603. Calculate the co-line second-order fitting coefficients and the co-column second-order fitting coefficients according to the co-line index set and the co-column index set.

[0178] 604. Calculate the missing corner point coordinates according to the co-line second-order fitting coefficients and the co-column second-order fitting coefficients.

[0179] Each element in the new coordinate set is accessed in turn. For each element, it is checked whether it is the identification of a missing corner point. If so, the row number and the column number of the missing corner point are recorded.

[0180] According to the row and column serial number of the missing corner point, a corresponding common row index set (all valid corner point coordinates of the row where the missing point is located) and a common column index set (all valid corner point coordinates of the column where the missing point is located) are extracted. These sets contain valid coordinates in the same row or column as the missing point, and their distribution can reflect the coordinate trend of the missing point.

[0181] For the valid coordinates in the common row index set, the second-order fitting coefficients A of the common row corner points and the second-order fitting coefficients A' of the common column corner points are calculated by polynomial fitting. The second-order fitting coefficients A of the common row corner points describe the nonlinear law of the column coordinates in the same row with the column serial number, and the second-order fitting coefficients A' of the common column corner points describe the nonlinear law of the row coordinates in the same column with the row serial number.

[0182] Polynomial fitting formula:

[0183]

[0184] In the polynomial fitting process, the positions of X and Y need to be swapped, i.e., Y as the independent variable, and then the second-order fitting coefficients A' of the common column corner points are calculated.

[0185] As can be seen from the polynomial fitting formula, the highest order of the independent variable in the fitted common row and common column polynomials Y=A·X and X=A'·Y is 2. If the intersection point of the two equations is calculated, a 4th-order independent variable will be generated. However, there are only two equations in the system, which makes it impossible to calculate the X and Y values of the intersection point.

[0186] 4th-order independent variable formula:

[0187]

[0188] where a, b, c, d, and e are coefficients.

[0189] In order to calculate the intersection point coordinates, the line drawing method is used to find the intersection point:

[0190] According to the length of the checkerboard square, the step size is set to generate the independent variable set. Taking the second-order polynomial in the row direction as an example, assuming that the length of the checkerboard square in the horizontal direction (corresponding to the relative position of the corner point in the row direction) is L, starting from one end of the row where the missing corner point is located, the value of the independent variable x is gradually increased by the step size Δx (generally less than L, for example, L / 10), until it covers the other end of the row where the missing corner point is located, thereby obtaining a row direction independent variable set. Similarly, for the second-order polynomial in the column direction, the step size is set according to the length of the checkerboard square in the vertical direction (corresponding to the relative position of the corner point in the column direction) to generate a column direction independent variable set.

[0191] The row direction coordinate set is obtained by substituting each variable x in the row direction variable set into the row direction polynomial to calculate the corresponding row coordinate. The column direction coordinate set is obtained by substituting each variable y in the column direction variable set into the column direction polynomial to calculate the corresponding column coordinate.

[0192] Two parabolas are drawn according to the coordinate sets calculated in the row direction and the column direction. The row direction coordinate set corresponds to a parabola on the chessboard image plane, which reflects the distribution trend of the corner points in the row direction. The column direction coordinate set corresponds to another parabola, which reflects the distribution trend of the corner points in the column direction.

[0193] The intersection points of the two parabolas and the four edges of the image are calculated. For each parabola, the intersection point coordinates are obtained by solving the simultaneous equations of the parabola equation and the equation of the straight line on which the image edge lies. As shown in Figure 14 , the intersection point serves as the starting point and the ending point of the parabola. The intersection region of the two parabolas on the image plane is found. This intersection region is the common part of the coverage ranges of the two parabolas, which represents the region that satisfies the distribution trends of the corner points in the row and column directions after fitting the second-order polynomials in the row and column directions.

[0194] The coordinates of all the pixel points in the intersection region are counted, and the average values of the coordinates of these pixel points in the x direction and the y direction are calculated, i.e., the missing corner point coordinates. As shown in Figure 15 Figure 15 A diagram for supplementing the missing corner point coordinates.

[0195] In the actual chessboard image, due to the influence of factors such as lens distortion and image perspective, the distribution of the corner points is often not a strict straight line distribution, but has a certain curvature. The second-order fitting can more accurately describe this curvature, so as to calculate more accurate missing corner point coordinates. The second-order fitting information of the common row and the common column is considered, the missing corner point coordinates are solved by solving the simultaneous equations, the correlation of the corner points on the chessboard in the row and column directions is fully utilized, the error caused by single direction fitting is reduced, and the accuracy of the coordinate calculation is improved.

[0196] Referring to Figure 7 , an embodiment of a device for correcting image nonlinear distortion is provided, which comprises:

[0197] The segmentation unit 701 is configured to input the chessboard image into a target image, and segment the target image into a plurality of sub-regions according to the chessboard image, the target image being an image that generates nonlinear distortion.

[0198] The extraction unit 702 is configured to extract sub-region corner point coordinates of the sub-regions.

[0199] ​The sorting unit 703 is configured to sort the sub-region corner point coordinates.

[0200] The computing unit 704 is configured to calculate a homographic transformation matrix according to the sub-region corner point coordinates, the black-and-white grid corner point coordinates of the checkerboard image, and the sorting result, the black-and-white grid corner point coordinates being standard corner point coordinates of a plurality of black-and-white grids in the checkerboard image.

[0201] The converting unit 705 is configured to convert the target image into a corrected image by using the homographic transformation matrix.

[0202] Optionally, before the sorting unit 703, the correction device further comprises:

[0203] The removing unit 706 is configured to remove abnormal points in the sub-region corner point coordinates.

[0204] DETAILED DESCRIPTION Figures 1 to 6 Embodiments, which will not be described herein.

[0205] Reference Figure 8 The present application also provides a correction device for image nonlinear distortion, comprising:

[0206] The processor 801, the memory 802, the input and output unit 803, and the bus 804.

[0207] The processor 801 is connected with the memory 802, the input and output unit 803, and the bus 804.

[0208] The memory 802 stores a program, and the processor 801 invokes the program to perform the correction method in Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 and Figure 6 .

[0209] The present application provides a computer readable storage medium, which stores a program, and the program performs the correction method in Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 and Figure 6 when executed on a computer.

[0210] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0211] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0212] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0213] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0214] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially or substantially, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various other media that can store program codes.

Claims

1. A method of rectifying image non-linear distortion, characterized by, The method comprises the following steps: inputting a chessboard image into a target image, and segmenting the target image into several sub-regions according to the chessboard image, wherein the target image is an image with nonlinear distortion; extracting sub-region corner point coordinates of the sub-regions; sorting the sub-region corner point coordinates, which comprises the following steps: step 1: sorting row coordinates in a row coordinate array and column coordinates in a column coordinate array respectively, and extracting minimum row coordinates and minimum column coordinates; step 2: obtaining common row upper and lower limits and common column upper and lower limits corresponding to the minimum row coordinates and the minimum column coordinates according to row distance thresholds and column distance thresholds; step 3: recording and storing row coordinates and column coordinates meeting the common row upper and lower limits and the common column upper and lower limits into common row index sets and common column index sets respectively; step 4: removing the row coordinates and the column coordinates stored in the common row index sets and the common column index sets from the row coordinate array and the column coordinate array; step 5: repeating steps 1 to 4 until all row coordinates and column coordinates in the row coordinate array and the column coordinate array are traversed; step 6: traversing all common row index sets and common column index sets, and extracting index values appearing in the common row index sets and the common column index sets at the same time; step 7: according to the common row index sets and the common column index sets where the index values are located, putting sub-region corner point coordinates corresponding to the index values into a new coordinate set, wherein the new coordinate set has a length of a product of row and column numbers of black and white grids in the chessboard image; step 8: repeating steps 6 and 7 until all sub-region corner point coordinates corresponding to the index values are put into the new coordinate set; calculating a homography transformation matrix according to the sub-region corner point coordinates, black and white grid corner point coordinates of the chessboard image and a sorting result, wherein the black and white grid corner point coordinates are standard corner point coordinates of several black and white grids in the chessboard image; converting the target image into a corrected image through the homography transformation matrix.

2. The method of correction of claim 1, wherein, Before the sorting of the sub-region corner point coordinates, the correction method further comprises: removing abnormal points in the sub-region corner point coordinates.

3. The method of correction of claim 2, wherein, The removing of the abnormal points in the sub-region corner point coordinates comprises: setting row distance thresholds and column distance thresholds between the sub-regions; sorting row coordinates of the sub-region corner point coordinates, and obtaining a first row coordinate of a first sub-region corner point coordinate; calculating first common row upper and lower limits according to the first row coordinate and the row distance thresholds, and obtaining a row coordinate array according to the first common row upper and lower limits; extracting a corresponding column coordinate array according to the row coordinate array; performing operations on values in the column coordinate array to obtain a column coordinate difference value array; if the column coordinate difference value is less than the column distance threshold, extracting and summarizing the column coordinate difference value as an abnormal point array; analyzing elements in the abnormal point array through a preset condition, and removing corresponding abnormal points according to an analysis result, wherein the elements in the abnormal point array include column coordinate difference values less than the column distance threshold.

4. The method of correction of claim 3, wherein, The analysis of the elements in the abnormal point array through the preset condition and the removal of the corresponding abnormal points according to the analysis result comprise: If the element in the abnormal point array is the first element in the column coordinate difference array, remove the sub-region corner point coordinate associated with the first element in the row coordinate array; If the element in the abnormal point array is the last element in the column coordinate difference array, remove the sub-region corner point coordinate associated with the element after the corresponding element in the row coordinate array; If the element in the abnormal point array is neither the first element in the column coordinate difference array nor the last element in the column coordinate difference array, obtain a first difference value of the corresponding element in the row coordinate array minus the element before the corresponding element; If the first difference value is less than the column distance threshold, remove the sub-region corner point coordinate associated with the corresponding element in the row coordinate array, otherwise remove the sub-region corner point coordinate associated with the element after the corresponding element in the row coordinate array.

5. The method of correction of claim 1, wherein, Before the step eight, the correction method further comprises: determining whether the number of the common row index set or the common column index set is less than the number of rows and columns of the black and white grid in the checkerboard image; if yes, calculating the missing corner point coordinate according to the common row index set and the common column index set; putting the missing corner point coordinate into the new coordinate set.

6. The method of correction of claim 5, wherein, The calculating the missing corner point coordinate according to the common row index set and the common column index set comprises: traversing the new coordinate set to record the row and column serial numbers of the missing corner point coordinate; extracting the common row index set and the common column index set in which the missing corner point coordinate is located according to the row and column serial numbers, respectively; calculating the common row second-order fitting coefficient and the common column second-order fitting coefficient according to the common row index set and the common column index set; calculating the missing corner point coordinate according to the common row second-order fitting coefficient and the common column second-order fitting coefficient.

7. An apparatus for correcting non-linear distortion of an image, characterized by The correction device for performing the correction method according to any one of claims 1 to 6 comprises: a segmentation unit configured to input a checkerboard image into a target image, and segment the target image into a plurality of sub-regions according to the checkerboard image, the target image being an image with non-linear distortion; an extraction unit configured to extract sub-region corner point coordinates of the sub-regions; an ordering unit configured to order the sub-region corner point coordinates; a calculation unit configured to calculate a homographic transformation matrix according to the sub-region corner point coordinates, black and white grid corner point coordinates of the checkerboard image, and the ordering result, the black and white grid corner point coordinates being standard corner point coordinates of a plurality of black and white grids in the checkerboard image; a conversion unit configured to convert the target image into a corrected image through the homographic transformation matrix.

8. An apparatus for correcting non-linear distortion of an image, characterized by The device comprises: a processor, a memory, an input / output unit, and a bus; the processor is connected with the memory, the input / output unit, and the bus; the memory stores a program, and the processor invokes the program to perform the correction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program performs the correction method according to any one of claims 1 to 6 when executed on a computer.

Images